23 KiB

Raw Blame History

RunA — Project Specification and Architecture

Status: Initial architecture, revision 2
Primary host language: Rust
Initial embedded language: Rune
Inspirations: Smalltalk/Squeak/Pharo, Morphic, Erlang/OTP
Primary interfaces: Self-contained graphical world and VS Code extension

1. Vision

Build a self-contained, persistent, live programming world. Its fundamental computational entities are actors, not shared mutable objects. The environment itself is programmable and inspectable, with live actor spawning, message sending, state inspection, behavior editing, debugging, and hot code upgrades. A VS Code extension provides an equally capable external interface to the same running image via a programmatic API.

Rust supplies a minimal, reliable substrate; increasingly much of the actor framework, graphical world, and development tools should be implemented in Rune. The end state is a Smalltalk-like image with Erlang-inspired actor isolation and modern tooling.

Core goals

Inspect, spawn, stop, restart, and supervise actors from both graphical world and VS Code.
Inspect state, mailboxes, behavior versions, failures, and execution status.
Send messages and evaluate development operations interactively.
Edit, compile, publish, install, and roll back behaviors without restarting the image.
Support interruptible execution, pause/resume, and eventually source-level stepping.
Preserve stable actor identity and optionally mailbox contents across actor incarnation failures.
Store canonical Rune source and immutable implementations by content identity; cache compilation.
Build self-inspecting, self-modifiable graphical tools, gradually moving implementation from Rust to Rune.
Persist and restore meaningful parts of the image without assuming arbitrary stack serialization.

2. Architectural Principles

One authoritative live image. In-world tools and VS Code are clients of the same semantic development services.
Actor isolation. No direct mutable cross-actor state sharing. All normal interaction occurs through messages.
Separate identity, mailbox, state, handler, and incarnation. A failed handler execution need not destroy an actor's identity or mailbox.
Stateless handlers. Actor-specific mutable state is explicit input/output, not hidden global state.
Immutable code versions. A behavior name points to an exact implementation; active executions retain their own version.
Rust mechanisms, Rune policies. Native kernel for scheduling, execution control, memory/resource ownership, platform I/O; Rune for behaviors, UI, supervision policy, and tools where practical.
Safe-point upgrades and inspection. Never unsafely mutate a currently executing handler.
Honest guarantees. Distinguish cooperative yielding from true preemption, in-memory retention from disk durability, and at-least-once delivery from exactly-once effects.
Replaceable language integration. Keep a narrow runtime adapter, but do not over-generalize before Rune feasibility is tested.

3. Layered Architecture

+-----------------------------------------------------------+
| Live graphical world (Morphic-inspired)                   |
| Windows, widgets, actor browser, inspector, debugger      |
+-------------------------------+---------------------------+
| Rune development services     | VS Code extension         |
| Behavior registry, workspace  | Virtual docs, trees, DAP  |
+-------------------------------+---------------------------+
| Unified image API / development protocol                  |
| Actors, mailboxes, state, code, debugging, events          |
+-----------------------------------------------------------+
| Rune actor framework                                      |
| Stateless handlers, supervision policies, UI behaviors    |
+-----------------------------------------------------------+
| Rune runtime adapter / compilation and execution          |
+-----------------------------------------------------------+
| Rust actor kernel                                         |
| Identity, mailboxes, scheduler, transactions, monitoring  |
+-----------------------------------------------------------+
| Rust platform substrate                                   |
| Graphics, input, timers, filesystem, networking, storage  |
+-----------------------------------------------------------+

The graphical world should use internal service calls to the image API; the VS Code extension uses a versioned external transport. Neither interface should have privileged ad hoc paths that violate runtime invariants.

4. Fundamental Actor Model

An actor comprises distinct components:

Actor {
    id: ActorId,                   // stable logical identity
    mailbox: MailboxId,           // independent queue; may survive restart
    state: StateRecord,           // committed private state
    behavior: ImplementationId,   // immutable handler implementation
    incarnation: Generation,      // changes on restart
    status: ActorStatus,
    supervisor: Option<ActorId>,
}

4.1 Identity

Actor IDs are opaque, never raw pointers. Use generation or other safeguards against stale references. A logical actor may be restarted with the same ID and mailbox but a new incarnation. Distinguish runtime-instance identity from actor identity if remote or persisted references are introduced.

4.2 Mailbox

The mailbox is managed independently of handler execution. Messages have at least queued, reserved, acknowledged, and dead-lettered lifecycle states where durable processing is enabled. Define ordering explicitly; FIFO within a queue does not imply deterministic ordering between concurrent senders.

Start with bounded queues and observable backpressure. For durable mailboxes, initially target at-least-once processing, retry limits, and dead-letter handling. Retention across actor restarts does not imply survival across host process crashes; disk durability requires explicit storage and recovery.

4.3 State

State is owned by the actor runtime, logically private, and supplied to each scheduled handler invocation. The handler works with a controlled draft or value and returns new state; it cannot mutate committed state directly. State has a schema/version where useful and can be inspected via snapshots at safe points.

4.4 Stateless handler

The handler is a stateless Rune function associated with an immutable implementation ID. Conceptually:

handle(state, message, context) -> Transition(new_state, effects)

context provides restricted runtime capabilities, such as actor identity and correlation IDs. Actor-specific state must not be stored in global variables or implicit mutable closures. The handler may await approved asynchronous operations. This API is conceptual, not verified Rune syntax.

4.5 Effects

Prefer explicit effect descriptions for sends, timers, external requests, and lifecycle operations rather than arbitrary irreversible side effects during a state transition.

Transition {
    new_state,
    effects: [Send(...), ScheduleTimer(...), ...],
}

On durable paths, commit the state transition, input-message acknowledgment, and transactional outbox records atomically. Delivery of outbox effects occurs afterward and may be retried. Exactly-once external side effects are not guaranteed; use idempotency keys and deduplication where needed.

5. Scheduling, Awaiting, and Interruption

Each actor processes at most one active handler invocation at a time, including suspended invocations, unless a future explicitly designed concurrency policy says otherwise. Other messages remain queued.

A scheduled invocation receives its committed state, a reserved message, the selected implementation, and execution context. It runs until it:

Completes, producing a transition to commit.
Awaits an asynchronous operation, suspending without blocking a native worker thread.
Is paused/interrupted at a supported safe point.
Fails or is cancelled.
Reaches a configured execution budget.

Await is not commit. State and effects from an incomplete invocation remain uncommitted. On crash, the current invocation is discarded or recovered according to documented policy, and its reserved message can be retried. Long awaits can consume resources and complicate recovery; establish limits and cancellation rules.

5.1 Interruptibility requirement

Interruptible Rune execution is a foundational requirement, not a future nice-to-have. The runtime should eventually support pause, resume, cancel, execution budgets, runaway-code detection, breakpoint locations, and source-level stepping.

Phase 0 must verify actual Rune APIs and limitations: cooperative yield, VM instruction hooks/budgets, native callback blocking, async suspension, frame inspection, and safe interruption of CPU-bound loops. If Rune lacks the necessary mechanisms, evaluate bytecode instrumentation, interpreter modifications, or a controlled fork. Do not claim preemptive scheduling until it is actually implemented and tested. Native Rust callbacks must also define their cancellation behavior.

5.2 Failure and supervision

Separate handler exceptions, invocation cancellation, actor incarnation failure, supervisor restarts, host process failure, and machine failure. Supervisors can restart an actor with retained mailbox, last committed state, and the same or a compatible upgraded handler. Supervision strategies (one-for-one, one-for-all, rest-for-one) can initially live in Rune, atop native monitoring and lifecycle events. Add restart-intensity limits and poison-message handling.

6. Content-Addressed Rune Code

Keep four distinct things:

Original source for editing, diagnostics, and exact text preservation.
Canonical source normalized deterministically and semantics-preservingly.
Source identity derived from canonical bytes with a versioned hash scheme.
Compiled artifact cached using all compilation inputs.

Use a domain-separated hash such as BLAKE3 and a canonicalization version. Never strip arbitrary whitespace/comments without parser-aware analysis; preserve original source and source maps for debugging.

Example identifiers:

rune-source:v1:blake3:<digest>
compiled:<digest-of-source-plus-toolchain-configuration>

A compiled artifact cache key must include source identity, Rune compiler version, compiler settings, enabled native modules, runtime ABI, relevant features, and target constraints. Do not assume artifacts are portable between Rune versions or host configurations.

A mutable behavior registry maps names to immutable implementations:

behaviors/Counter -> implementation:abc123
history: [implementation:def456, implementation:abc123]

Active frames retain references to their exact version. Garbage collection must respect references from active invocations, actor state, persisted snapshots, and retained history. Revisions prevent conflicting editor updates.

6.1 Hot upgrade procedure

Accept source and expected behavior revision.
Canonicalize, hash, and compile (or reuse cached artifact).
Report diagnostics without changing the running behavior on failure.
Publish an immutable implementation.
Validate state compatibility or execute explicit migration.
Install at a safe boundary (initially between message handlers).
Emit change events and retain rollback history.

Do not rewrite suspended execution frames in place.

7. Graphical World

Build a self-contained Morphic-inspired desktop where windows, widgets, and development tools are themselves inspectable and eventually written primarily in Rune.

Required in-world operations: browse actors and behaviors; spawn, stop, restart, pause, and resume actors; inspect state, mailboxes, failures, supervisor relationships, and versions; send messages; edit and compile behavior source; install/rollback versions; inspect execution; view events and diagnostics.

Not every graphical node must be an actor. Use actors for independently scheduled components and ordinary Rune data for geometry, style, and lightweight scene nodes. UI actors update a retained scene representation; a scene manager produces immutable render snapshots consumed by native graphics code. Avoid synchronous arbitrary actor callbacks on the render thread. Native code initially owns windowing, drawing, and input capture; Rune progressively owns composition, layout, event routing, widgets, and tools.

The graphical inspector should support selecting a component, locating its owning actor, inspecting its state and behavior, editing its implementation, and observing live changes. The development environment must be able to inspect itself.

8. Unified Programmatic Image API

The running image exposes a versioned semantic API, used by both the graphical tools and the VS Code extension. It must also be usable by CLI tools, tests, automation, and coding agents.

Suggested operation families (provisional names):

image.info            image.snapshot       image.restore
image.subscribe

actor.list            actor.spawn          actor.inspect
actor.send            actor.call           actor.pause
actor.resume          actor.stop           actor.restart
actor.subscribe

mailbox.inspect       mailbox.peek         mailbox.statistics
mailbox.retry         mailbox.deadLetters

state.inspect         state.snapshot       state.history

behavior.list         behavior.get         behavior.compile
behavior.publish      behavior.install     behavior.rollback
behavior.history

code.getSource        code.getArtifact     code.getMetadata
code.listReferences

debug.attach          debug.detach         debug.pause
debug.resume          debug.step           debug.breakpoints
debug.stack           debug.inspectFrame

These are desired capabilities, not a requirement to implement every endpoint immediately. Define explicit request/response schemas, error codes, authorization, optimistic revision checks, and event subscription semantics. Mutations should return structured outcomes and emit events.

8.1 Security

Normal actors should not automatically receive development privileges. Arbitrary evaluation, state mutation, code replacement, and process control require explicit capabilities. For the initial local bridge, bind to loopback and use a defensible authentication mechanism. Avoid unauthenticated remote code execution endpoints.

9. VS Code Extension

VS Code is an external client of the same live image, not a separate runtime. Begin with JSON-RPC over local WebSocket or an equivalently straightforward versioned transport.

Required extension UX:

Connection management and runtime status.
Actor tree with spawn, stop, restart, pause, and resume actions.
Actor state and mailbox inspector, including message sending.
Behavior tree and source navigation.
Editable virtual documents for mutable behavior references.
Read-only virtual documents for exact content-addressed implementations.
Compilation diagnostics, update conflicts, and runtime event stream.
Later: Debug Adapter Protocol integration for breakpoints, frames, and stepping.

Suggested virtual URIs:

liveworld:/behaviors/Counter/current.rn
liveworld:/implementations/<hash>/source.rn

Saving the first compiles/publishes a new implementation and conditionally updates the behavior reference. The second is immutable and read-only. Avoid silently overwriting edits from another client.

10. Persistence

Implement persistence incrementally:

Source: canonical/original source, immutable implementations, registry revisions.
Actor state: selected actor identities, committed state, handler references, supervision metadata.
Environment: windows, layouts, workspaces, and development session state.
Advanced: coordinated actor/message snapshots and, only if technically feasible, suspended execution state.

Start with SQLite or another simple durable store plus Git-friendly source export. Define snapshot consistency explicitly. Never assume arbitrary Rune stacks, closures, and native resources can be serialized and restored. Preserve mailbox and state transaction invariants across recovery.

11. Proposed Repository

live-world/
├── Cargo.toml
├── README.md
├── docs/
│   ├── architecture.md
│   ├── actor-model.md
│   ├── runtime.md
│   ├── protocol.md
│   └── decisions/
├── crates/
│   ├── world-core/
│   ├── world-actor/
│   ├── world-runtime/
│   ├── world-rune/
│   ├── world-protocol/
│   ├── world-server/
│   ├── world-graphics/
│   └── world-app/
├── scripts/
│   ├── core/
│   ├── actors/
│   ├── ui/
│   └── tools/
├── vscode/
├── examples/
└── tests/

These are intended boundaries, not an instruction to generate empty crates prematurely. Start with the smallest workspace that supports the current experiment.

12. Implementation Roadmap

Phase 0 — Rune feasibility (first priority)

Embed Rune in a Rust workspace and compile/run a minimal script.
Register a native Rust function and call it from Rune.
Demonstrate async suspension and resumption using verified APIs.
Investigate interruption of CPU-bound code, execution budgets, frame inspection, and debugging hooks.
Measure VM/context memory overhead and compiled-code sharing.
Test cross-thread safety and runtime isolation assumptions.
Document actual Rune limitations and potential mitigation strategies.

Exit criterion: Evidence-based choice of execution model and a running example. Do not start GUI implementation until this is assessed.

Phase 1 — Minimal actor kernel

Stable IDs, incarnation generations, bounded mailboxes, state records, stateless handlers.
Spawn/send/receive/terminate, sequential scheduling, basic lifecycle and error events.
Demonstrate two Rune actors exchanging structured messages without shared mutable state.

Phase 2 — Transactional processing

Reserve/ack/retry messages; commit state transitions.
Explicit effects, error handling, and eventually an outbox for durable sends.
Test crash/restart retaining mailbox and last committed state.

Phase 3 — Content-addressed code

Original/canonical source storage, hashing, cache keys, behavior registry, revision history.
Compile or retrieve cached artifacts; retain exact implementation IDs.

Phase 4 — Live code and debugging foundation

Safe-point behavior upgrades, invalid-source rollback, state migration policy.
Pause/resume/cancel and initial execution inspection, subject to Rune feasibility.

Phase 5 — Image API

Headless server, versioned protocol, actor and mailbox operations, source editing, events.
CLI client that can spawn, inspect, send messages, and replace behaviors.

Phase 6 — Graphical world

Native window/rendering, input, scene manager, Rune-driven button and window.
In-world actor browser, state inspector, behavior editor, and message sender.

Phase 7 — VS Code extension

Connection, actor explorer, state/mailbox inspection, spawn/send controls.
Virtual behavior documents, compilation diagnostics, hot upgrades, runtime events.

Phase 8 — Persistence and advanced debugger

Durable mailboxes and state, snapshot/restore, supervised recovery.
Source-level debugging and richer interruption/stepping as supported by runtime.

13. Critical Risks and Open Decisions

Topic	Risk / decision
Rune interruption	Verify whether CPU-bound execution can be paused or budgeted safely.
Async invocation	Define how awaits preserve private state and what is retried after crash.
VM isolation	Determine per-actor VM vs shared compiled code and isolated execution contexts.
Scheduler fairness	Cooperative yield alone does not prevent a runaway CPU-bound handler.
Effects	Commit/retry semantics and idempotency for external side effects.
Mailbox durability	In-memory restart retention vs process-crash durable storage.
Source normalization	Canonicalization must preserve semantics and debugging locations.
Code upgrade	Active frames must retain exact old implementation versions.
UI performance	Avoid an actor per tiny graphical node and render-thread actor calls.
Persistence	Full execution-stack image restoration may require interpreter changes.
API security	Debugging and code installation are privileged operations.

14. Initial Non-Goals

No full BEAM clone, distributed actors, millions-of-actors guarantee, full Morphic compatibility, custom Rune compiler, complete IDE, full stack serialization, production-grade hostile-code sandbox, or complete source-level debugger in the first prototype. Do not build a large abstraction framework before the Rune experiment validates it.

15. Instructions for a Local Coding LLM

Act as the principal systems architect and implementation partner. Work incrementally. Begin only with Phase 0. Verify exact Rune APIs and syntax against the pinned version; never invent working APIs. Distinguish confirmed capabilities from hypotheses. Produce a minimal Cargo workspace, runnable examples, tests where practical, build/run instructions, measurements, and a concise feasibility report. Record architectural decisions and unresolved risks. Do not silently change the mailbox/state/handler decomposition, collapse development tools into separate runtimes, or bypass actor isolation for inspection.

First task: Implement a minimal Rust + Rune experiment that executes a Rune function, invokes a registered native Rust function, demonstrates supported asynchronous suspension, and investigates interruption and compiled-code reuse. Stop after reporting the results and proposed Phase 1 architecture.

16. Long-Term Acceptance Scenario

A developer launches the graphical world, spawns a counter actor, sends it a message, inspects its mailbox and committed state, opens its exact content-addressed Rune handler, edits and publishes a new version, installs it at a safe point, and sees subsequent messages use the new behavior. The same operations are possible through VS Code using the image API. A failed actor incarnation can be restarted with the same logical identity and retained mailbox, and the world can eventually save and restore a meaningful development session.

North star: The world is the program; actors are its living components; development is continuous interaction with that world rather than a separate edit-build-run cycle.

23 KiB Raw Blame History