test: define actor retry and transition guarantees

Cover failed-message retries from committed state and stop/restart FIFO preservation. Document in-memory all-or-nothing transitions and their process-local limits.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CHANGELOG.md

@@ -14,6 +14,8 @@ All notable changes to this project will be documented in this file.
 - Runnable two-actor counter demonstration and tests for runaway fairness,
   pending-message isolation, failures, backpressure, and all-or-nothing effects.
 - Execution-model decision and Phase 1 runtime documentation.
+- Lifecycle tests proving failed-message retry from committed state and
+  stop/restart preservation of reserved-first FIFO order.
 
 - Runnable Phase 0 experiment pinned to Rune 0.14.2, covering script execution,
   native calls, async suspension, instruction budgets, frame inspection,

crates/runa-runtime/src/lib.rs

@@ -483,4 +483,109 @@ mod tests {
         assert!(snapshot.reserved.is_some());
         Ok(())
     }
+
+    #[test]
+    fn restart_retries_failed_reservation_from_last_committed_state() -> ExperimentResult<()> {
+        use std::sync::atomic::AtomicBool;
+
+        let fail_once = Arc::new(AtomicBool::new(true));
+        let native_fail_once = Arc::clone(&fail_once);
+        let mut module = rune::Module::new();
+        module
+            .function("fail_once", move || {
+                native_fail_once.swap(false, Ordering::SeqCst)
+            })
+            .build()?;
+        let mut runtime = Runtime::default();
+        let code = runtime.publish(CompiledProgram::compile_with_module(
+            "fn check() { if fail_once() { 1 / 0 } else { 0 } } pub fn handle(state, kind, value) { let _ = check(); (state + value, []) }",
+            module,
+        )?)?;
+        let actor = runtime.spawn(code, 10, capacity(3))?;
+        let first = Message { kind: 1, value: 2 };
+        let second = Message { kind: 1, value: 3 };
+        runtime.send(actor, first)?;
+        runtime.send(actor, second)?;
+        for _ in 0..100 {
+            runtime.tick(capacity(5));
+        }
+
+        let failed = runtime.inspect(actor)?;
+        assert!(matches!(failed.status, Status::Failed(_)));
+        assert_eq!(failed.state, 10);
+        assert_eq!(failed.reserved, Some(first));
+        assert_eq!(failed.queued, vec![second]);
+        assert!(runtime.send(actor, Message { kind: 1, value: 4 }).is_err());
+        assert!(
+            runtime
+                .drain_events()
+                .iter()
+                .any(|event| matches!(event, Event::Failed(id, _) if *id == actor))
+        );
+
+        runtime.restart(actor)?;
+        let restarted = runtime.inspect(actor)?;
+        assert_eq!(restarted.id, failed.id);
+        assert_eq!(restarted.mailbox, failed.mailbox);
+        assert_eq!(restarted.implementation, failed.implementation);
+        assert_eq!(restarted.incarnation, failed.incarnation + 1);
+        assert_eq!(restarted.state, 10);
+        assert_eq!(restarted.queued, vec![first, second]);
+        assert_eq!(restarted.reserved, None);
+
+        for _ in 0..100 {
+            runtime.tick(capacity(5));
+        }
+        let completed = runtime.inspect(actor)?;
+        assert_eq!(completed.state, 15, "{completed:?}");
+        assert_eq!(completed.status, Status::Idle);
+        assert_eq!(completed.queued, Vec::<Message>::new());
+        assert_eq!(completed.reserved, None);
+        Ok(())
+    }
+
+    #[test]
+    fn stop_and_restart_preserve_reserved_and_queued_fifo_order() -> ExperimentResult<()> {
+        let mut runtime = Runtime::default();
+        let code = runtime.publish(CompiledProgram::compile(
+            "pub fn handle(state, kind, value) { (state * 10 + value, []) }",
+        )?)?;
+        let actor = runtime.spawn(code, 0, capacity(3))?;
+        let first = Message { kind: 0, value: 1 };
+        let second = Message { kind: 0, value: 2 };
+        let third = Message { kind: 0, value: 3 };
+        runtime.send(actor, first)?;
+        runtime.send(actor, second)?;
+        runtime.send(actor, third)?;
+
+        runtime.tick(capacity(1));
+        let running = runtime.inspect(actor)?;
+        assert_eq!(running.status, Status::Running);
+        assert_eq!(running.state, 0);
+        assert_eq!(running.reserved, Some(first));
+        assert_eq!(running.queued, vec![second, third]);
+
+        runtime.stop(actor)?;
+        let stopped = runtime.inspect(actor)?;
+        assert_eq!(stopped.status, Status::Stopped);
+        assert_eq!(stopped.state, 0);
+        assert_eq!(stopped.reserved, Some(first));
+        assert_eq!(stopped.queued, vec![second, third]);
+        assert!(runtime.send(actor, Message { kind: 0, value: 4 }).is_err());
+
+        runtime.restart(actor)?;
+        let restarted = runtime.inspect(actor)?;
+        assert_eq!(restarted.state, 0);
+        assert_eq!(restarted.reserved, None);
+        assert_eq!(restarted.queued, vec![first, second, third]);
+        for _ in 0..100 {
+            runtime.tick(capacity(5));
+        }
+        let completed = runtime.inspect(actor)?;
+        assert_eq!(completed.state, 123);
+        assert_eq!(completed.status, Status::Idle);
+        assert_eq!(completed.queued, Vec::<Message>::new());
+        assert!(completed.reserved.is_none());
+        Ok(())
+    }
 }

docs/runtime.md

@@ -45,17 +45,51 @@ runnable; the caller still needs to drive ticks.
 Successful handlers validate every send target and its aggregate queue
 capacity before updating state, acknowledging the reserved message, and
 enqueueing effects. Invalid effects fail the actor without partial sends or a
-state commit. This all-or-nothing behavior is in-memory only, not a durable
-transaction.
+state commit. This is a **logical, single-worker in-memory atomicity guarantee**:
+observers using the runtime see either the transition before commit or after
+commit, not an intermediate state/effect set. It is not crash-safe or durable.
+External side effects performed by native callbacks are outside this boundary
+and cannot be rolled back.
 
 Runtime errors retain the last committed state and reserved message, set an
-observable failed status, and emit a failure event. External sends return errors
+observable failed status, and emit a failure event. The failed message is not
+silently acknowledged or automatically retried. External sends return errors
 for unknown, unavailable, or full actors.
 
 Stopping drops the invocation, retaining committed state, mailbox and reserved
-message. Restarting a stopped or failed actor preserves IDs, increments its
-incarnation, and puts its reserved message at the front of the queue. Restart
-is explicit; there is no automatic poison-message retry or supervision policy.
+message. Restarting a stopped or failed actor preserves its actor ID, mailbox
+ID, implementation ID, committed state, and queued-message order; it increments
+its incarnation and puts the reserved message at the front of the queue.
+Restart is explicit. Repeated failure of the same reserved message can therefore
+block later messages until an operator changes the implementation or otherwise
+resolves it; there is no automatic poison-message policy or supervision.
+
+### Transition and retry guarantees
+
+| Situation | State | Reserved input | Effects | Status |
+| --- | --- | --- | --- | --- |
+| Handler returns a valid transition | New state commits | Acknowledged | All validated sends enqueue | `Idle` |
+| Rune execution or transition decoding fails | Last committed state retained | Retained | None from the returned transition | `Failed` |
+| Any target is unknown, stopped, failed, or over capacity | Last committed state retained | Retained | No sends from the transition enqueue | `Failed` |
+| Stop during an invocation | Last committed state retained | Retained | None from the incomplete transition | `Stopped` |
+| Restart after stop or failure | Retained until retried | Moved to queue front | None until retry completes | `Idle` |
+
+Mailbox capacity counts queued plus reserved messages. Before applying a
+transition, the runtime validates all effects, including aggregate sends to a
+single target. For a self-send, the completing input is acknowledged as part
+of the same transition, freeing its reserved slot before the new effects are
+enqueued. An invalid target or over-capacity effect rejects the entire returned
+transition.
+
+These are process-local guarantees only. A host crash loses state, mailboxes,
+reservations and events. External/native callback effects may already have
+occurred even when the Rune invocation later fails or is stopped.
+
+Lifecycle tests verify that a failed reservation retries from the last
+committed state, that stop/restart preserves reserved-first FIFO order, and
+that a running actor's queued messages are not processed until the reservation
+finishes. Transition tests cover malformed results, runtime failures, invalid
+targets, aggregate capacity, and successful message effects.
 
 ## Limits