docs: document the shard cluster, economy and per-shard databases

Describe the gateway and shard processes, transparent transfers, the
economy and console peers in the README. Compose creates one Postgres
database per shard and passes `SHARD_DATABASE_URL` and `SIM_SHARDS`;
the Justfile and Dockerfile use per-shard SQLite files.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Steve Biedermann 2026-10-12 20:43:11 +02:00
parent 00ea5899c1
commit ec861de3ba
6 changed files with 44 additions and 22 deletions

View File

@ -26,3 +26,4 @@ Notable changes to this project are documented here.
- Update Picard to `f88b9cb`.
- The default world is larger: 192×128 tiles with 180 people, 1200 rabbits, 80 wolves, and twice the food and houses.
- `DATABASE_URL` is replaced by `SHARD_DATABASE_URL`; existing `sim.db` snapshots are not migrated.
- Docker Compose creates one Postgres database per shard (`sim_shard_0` to `sim_shard_2`); existing `postgres_data` volumes must be recreated with `docker compose down -v`.

View File

@ -16,7 +16,7 @@ COPY --from=builder /src/target/release/sim /usr/local/bin/sim
COPY --from=builder /src/config.rn /etc/sim/config.rn
ENV SIM_ADDR=0.0.0.0:7878
ENV DATABASE_URL=sqlite:///data/sim.db?mode=rwc
ENV SHARD_DATABASE_URL=sqlite:///data/sim-shard-{shard}.db?mode=rwc
ENV SIM_CONFIG=/etc/sim/config.rn
VOLUME ["/data"]

View File

@ -8,4 +8,4 @@ stop:
docker compose down
dev:
DATABASE_URL="sqlite://sim.db?mode=rwc" cargo run
SHARD_DATABASE_URL="sqlite://sim-shard-{shard}.db?mode=rwc" cargo run

View File

@ -4,7 +4,19 @@ A browser-based world simulation with a Rust server and a Leptos/Wasm UI. The se
Every person, rabbit, wolf, and partnership is a Picard actor that owns its state and applies its own rules: people age, get hungry, eat, and starve; animals spend and gain energy, breed, and die; partnerships track when they last had a child. The world keeps only world-level state (terrain, resources, the shared stockpile, houses, and the calendar) and talks to entities exclusively through messages: each step it asks the actors for their state, decides movement and opportunities with the Rune behavior scripts, and sends the actors the outcome. Behavior decisions stay on the simulation thread because Rune's VM is not `Send`, and replies are processed in entity order so runs stay deterministic. Picard's tracing and runtime statistics features are enabled: actor lifecycle events are logged (configure verbosity with `RUST_LOG`, for example `RUST_LOG=picard=trace`), and cumulative actor/message counters are shown in the UI.
The server enables Picard's console by default and exposes its JSON request/response protocol over `ws://127.0.0.1:7878/api/actors/ws`. Send one JSON request per text frame; each receives one response. For example, `{"id":1,"op":"actors"}` lists actors, and `{"id":2,"op":"inspect","path":"/user/person-1"}` inspects one (actors are named `person-<id>`, `rabbit-<id>`, `wolf-<id>`, and `partnership-<id>`). Other operations include `messages`, `pause`, `resume`, `clear`, `stop`, `restart`, and `send`. The route dispatches console requests through the simulation thread to Picard's actor system. `cargo run --no-default-features` disables the endpoint; `--features console` enables it explicitly.
## Cluster
The world is split into vertical strips of columns called shards. Each shard is a separate process with its own Picard actor system and its own database. `sim` starts as a gateway: it spawns `SIM_SHARDS` (default 3) copies of itself with the `shard` subcommand, steps them in lockstep, and merges their views so the UI sees one world. Edits from the UI are routed to the shard that owns them.
People, animals and carts that cross a shard border are transferred with their whole state and inventory to the neighbouring shard's actor system. They keep their address, and messages to them are forwarded transparently, so actors never know which shard they run on. Shards talk to each other directly over WebSockets (`/remote` for actor messages and transfers, `/control` for the gateway protocol).
## Economy
Every shard founds markets and workplaces: farms, woodcutters, quarries, hunters, toolmakers and cartwrights. Adults are hired, commute to their job site each day, work their shift, sell the goods at a market, get paid and go home. Farmers sow fields that ripen into crops; gatherers fell trees, quarry stone and hunt; crafters buy materials and make tools and carts. Markets are actors with their own coins and stock; prices follow supply and demand, and buyers and sellers haggle. Merchants carry surplus goods to markets on other shards. Markets, workplaces, carts and crop fields are drawn on the map, and the side panel shows each person's job and coins.
## Actor Console
The server enables Picard's console by default and exposes its JSON request/response protocol over `ws://127.0.0.1:7878/api/actors/ws`. The connection reaches shard 0; use `?shard=N` to connect to shard `N`. Every shard registers the others as console peers named `shard-N`: `peers` lists them, `@shard-1 actors` forwards a command, and `send @shard-1 PATH JSON` sends a message to a remote actor. Send one JSON request per text frame; each receives one response. For example, `{"id":1,"op":"actors"}` lists actors, and `{"id":2,"op":"inspect","path":"/user/person-1"}` inspects one (actors are named `person-<id>`, `rabbit-<id>`, `wolf-<id>`, and `partnership-<id>`). Other operations include `messages`, `pause`, `resume`, `clear`, `stop`, `restart`, and `send`. Console requests are answered by the shard's actor system without waiting for the simulation thread. `cargo run --no-default-features` disables the endpoint; `--features console` enables it explicitly.
The simulation script is the root [config.rn](./config.rn), a single Rune file. Its `pub const` items hold defaults and tuning values: map dimensions and terrain generation, initial population and resources, calendar and life-stage defaults, hunger, gathering, construction, partnerships and births, animal behavior, and edit limits. Its `pub fn person`, `pub fn rabbit`, and `pub fn wolf` items are the agent behaviours, and they can use the constants directly. Every constant the server reads must be defined; missing or invalid values are reported at startup. `SIM_CONFIG` can point to a different Rune file. Docker Compose mounts the repository's `config.rn` read-only into the app container.
@ -12,7 +24,7 @@ The in-browser script editor edits the same file contents. Applied edits take ef
The **Calendar and life-stage durations** panel configures ticks per day and the number of days, weeks, and months in larger calendar units. Each human life stage has an inclusive minimum/maximum duration in in-game days; the world samples within that range when a person enters the stage. These runtime settings are stored with the world snapshot and take precedence over the config defaults until the world is reset.
New people, including babies, start as toddlers and age through child, teen, young adult, adult, older adult, elderly, and death. By default, the starting village has nine young and mature adults and a reserve of 60 food so it can form partnerships and sustain its population. People pathfind around blocked terrain when travelling home for food. Unpartnered people from young adulthood onward can form a single partnership with a nearby eligible person. Each partnership can have one toddler per calendar week when both partners are in stages marked fertile, the stockpile has enough food, and housing capacity permits it. By default, the village has capacity for 12 people, with each additional house increasing capacity by three; these values are configurable. Young adult and adult stages are fertile by default; fertility can be changed per stage in the calendar panel. Partnerships are Picard actors as well as people and animals.
New people, including babies, start as toddlers and age through child, teen, young adult, adult, older adult, elderly, and death. By default, the 192×128 world starts with 180 people (60 of them young adults), 1200 rabbits, 80 wolves and 1200 food, shared between the shards. People pathfind around blocked terrain when travelling home for food. Unpartnered people from young adulthood onward can form a single partnership with a nearby eligible person. Each partnership can have one toddler per calendar week when both partners are in stages marked fertile, the stockpile has enough food, and housing capacity permits it. By default, the village has capacity for 12 people, with each additional house increasing capacity by three; these values are configurable. Young adult and adult stages are fertile by default; fertility can be changed per stage in the calendar panel. Partnerships, markets, workplaces and carts are Picard actors as well as people and animals.
The UI is built automatically as part of `cargo run` and embedded in the server binary. There is no separate manual UI build step.
@ -26,7 +38,7 @@ Install Rust with `rustup`. The build script installs the `wasm32-unknown-unknow
just dev
```
Open [http://127.0.0.1:7878](http://127.0.0.1:7878). The database is SQLite at `sim.db` in the current directory, and the initial seed is `42`. You can also run directly with `cargo run`.
Open [http://127.0.0.1:7878](http://127.0.0.1:7878). Each shard keeps a SQLite database `sim-shard-N.db` in the current directory, and the initial seed is `42`. You can also run directly with `cargo run`.
To choose a seed on the first run:
@ -34,7 +46,7 @@ To choose a seed on the first run:
cargo run -- 123
```
An existing database snapshot takes precedence on later runs, including over a new command-line seed. Use the UI's **Reset** button to start a fresh world with its current seed, or **Regenerate** to choose a different seed.
Existing shard snapshots take precedence on later runs, including over a new command-line seed. If the shards' snapshots don't match (for example after changing `SIM_SHARDS`), all shards start fresh. Use the UI's **Reset** button to start a fresh world with its current seed, or **Regenerate** to choose a different seed.
### Run with Just
@ -60,7 +72,7 @@ Install Docker with the Compose plugin, then run `just run`. This stops any exis
docker compose up --build -d
```
Open [http://localhost:7878](http://localhost:7878). Compose waits for Postgres to become healthy before starting the app. Postgres data is stored in the `postgres_data` volume and survives container restarts, `just stop`, and `docker compose down`.
Open [http://localhost:7878](http://localhost:7878). Postgres creates one database per shard (`sim_shard_0` to `sim_shard_2`) from [docker/postgres-init.sql](./docker/postgres-init.sql) when the volume is first created; add databases there when raising `SIM_SHARDS`, and recreate the volume after changing the script. Compose waits for Postgres to become healthy before starting the app. Postgres data is stored in the `postgres_data` volume and survives container restarts, `just stop`, and `docker compose down`.
To remove the database volume and all saved simulation data as well:
@ -72,36 +84,39 @@ The Compose file uses development-only database credentials. Change them before
## Database Configuration
The server selects its database through `DATABASE_URL`:
Each shard selects its database through `SHARD_DATABASE_URL`. `{shard}` in the URL is replaced with the shard index and is required when there is more than one shard:
| Backend | Example URL |
| ---------------- | --------------------------------------- |
| SQLite (default) | `sqlite://sim.db?mode=rwc` |
| Postgres | `postgres://sim:sim@localhost:5432/sim` |
| ---------------- | ------------------------------------------------- |
| SQLite (default) | `sqlite://sim-shard-{shard}.db?mode=rwc` |
| Postgres | `postgres://USER:PASSWORD@localhost:5432/sim_shard_{shard}` |
For example, run locally against a reachable Postgres server with:
For example, run locally against a reachable Postgres server that has the shard databases with:
```sh
DATABASE_URL=postgres://sim:sim@localhost:5432/sim cargo run
SHARD_DATABASE_URL='postgres://USER:PASSWORD@localhost:5432/sim_shard_{shard}' cargo run
```
The simulation is saved as a snapshot about once per second and restored at startup. SQLite is convenient for local testing; Compose configures the app to use Postgres.
Every shard saves its part of the world as a snapshot about once per second and restores it at startup. SQLite is convenient for local testing; Compose configures the app to use Postgres.
`SIM_ADDR` sets the listen address. It defaults to `127.0.0.1:7878`; containers set it to `0.0.0.0:7878`.
`SIM_SHARDS` sets the number of shards (default 3). `SIM_ADDR` sets the gateway's listen address. It defaults to `127.0.0.1:7878`; containers set it to `0.0.0.0:7878`.
## Project Layout
- `src/world.rs` contains the simulation and snapshot serialization.
- `src/main.rs` runs the Axum server, simulation thread, and HTTP/WebSocket API.
- `src/world.rs` contains the simulation and snapshot serialization; `src/world/` holds the actors, shard routing and economy.
- `src/main.rs` runs the gateway's Axum server and HTTP/WebSocket API, and the `shard` subcommand.
- `src/cluster/` contains the gateway, the shard node, the inter-shard transport and the control protocol.
- `src/storage.rs` provides SQLite and Postgres snapshot storage.
- `config.rn` is the Rune simulation script: `pub const` defaults and tuning values plus `pub fn` behaviours.
- `ui/` contains the Leptos frontend, compiled to Wasm by `build.rs`.
- `Dockerfile`, `compose.yaml`, and `Justfile` provide container and developer workflows.
- `Dockerfile`, `compose.yaml`, `docker/postgres-init.sql` and `Justfile` provide container and developer workflows.
## Tests
Run the server tests with:
```sh
cargo test --bin sim
cargo test --workspace --no-default-features
```
`tests/cluster.rs` starts a real multi-process cluster.

View File

@ -4,7 +4,8 @@ services:
environment:
SIM_ADDR: 0.0.0.0:7878
SIM_CONFIG: /etc/sim/config.rn
DATABASE_URL: postgres://sim:sim@db:5432/sim
SIM_SHARDS: "3"
SHARD_DATABASE_URL: postgres://sim:sim@db:5432/sim_shard_{shard}
volumes:
- ./config.rn:/etc/sim/config.rn:ro
ports:
@ -27,6 +28,7 @@ services:
retries: 10
volumes:
- postgres_data:/var/lib/postgresql/data
- ./docker/postgres-init.sql:/docker-entrypoint-initdb.d/shards.sql:ro
restart: unless-stopped
volumes:

4
docker/postgres-init.sql Normal file
View File

@ -0,0 +1,4 @@
-- One database per shard; runs only when the postgres_data volume is created.
CREATE DATABASE sim_shard_0 OWNER sim;
CREATE DATABASE sim_shard_1 OWNER sim;
CREATE DATABASE sim_shard_2 OWNER sim;