docs: document the shard cluster, economy and per-shard databases
Describe the gateway and shard processes, transparent transfers, the economy and console peers in the README. Compose creates one Postgres database per shard and passes `SHARD_DATABASE_URL` and `SIM_SHARDS`; the Justfile and Dockerfile use per-shard SQLite files. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
parent
00ea5899c1
commit
ec861de3ba
|
|
@ -26,3 +26,4 @@ Notable changes to this project are documented here.
|
||||||
- Update Picard to `f88b9cb`.
|
- Update Picard to `f88b9cb`.
|
||||||
- The default world is larger: 192×128 tiles with 180 people, 1200 rabbits, 80 wolves, and twice the food and houses.
|
- The default world is larger: 192×128 tiles with 180 people, 1200 rabbits, 80 wolves, and twice the food and houses.
|
||||||
- `DATABASE_URL` is replaced by `SHARD_DATABASE_URL`; existing `sim.db` snapshots are not migrated.
|
- `DATABASE_URL` is replaced by `SHARD_DATABASE_URL`; existing `sim.db` snapshots are not migrated.
|
||||||
|
- Docker Compose creates one Postgres database per shard (`sim_shard_0` to `sim_shard_2`); existing `postgres_data` volumes must be recreated with `docker compose down -v`.
|
||||||
|
|
|
||||||
|
|
@ -16,7 +16,7 @@ COPY --from=builder /src/target/release/sim /usr/local/bin/sim
|
||||||
COPY --from=builder /src/config.rn /etc/sim/config.rn
|
COPY --from=builder /src/config.rn /etc/sim/config.rn
|
||||||
|
|
||||||
ENV SIM_ADDR=0.0.0.0:7878
|
ENV SIM_ADDR=0.0.0.0:7878
|
||||||
ENV DATABASE_URL=sqlite:///data/sim.db?mode=rwc
|
ENV SHARD_DATABASE_URL=sqlite:///data/sim-shard-{shard}.db?mode=rwc
|
||||||
ENV SIM_CONFIG=/etc/sim/config.rn
|
ENV SIM_CONFIG=/etc/sim/config.rn
|
||||||
|
|
||||||
VOLUME ["/data"]
|
VOLUME ["/data"]
|
||||||
|
|
|
||||||
2
Justfile
2
Justfile
|
|
@ -8,4 +8,4 @@ stop:
|
||||||
docker compose down
|
docker compose down
|
||||||
|
|
||||||
dev:
|
dev:
|
||||||
DATABASE_URL="sqlite://sim.db?mode=rwc" cargo run
|
SHARD_DATABASE_URL="sqlite://sim-shard-{shard}.db?mode=rwc" cargo run
|
||||||
49
README.md
49
README.md
|
|
@ -4,7 +4,19 @@ A browser-based world simulation with a Rust server and a Leptos/Wasm UI. The se
|
||||||
|
|
||||||
Every person, rabbit, wolf, and partnership is a Picard actor that owns its state and applies its own rules: people age, get hungry, eat, and starve; animals spend and gain energy, breed, and die; partnerships track when they last had a child. The world keeps only world-level state (terrain, resources, the shared stockpile, houses, and the calendar) and talks to entities exclusively through messages: each step it asks the actors for their state, decides movement and opportunities with the Rune behavior scripts, and sends the actors the outcome. Behavior decisions stay on the simulation thread because Rune's VM is not `Send`, and replies are processed in entity order so runs stay deterministic. Picard's tracing and runtime statistics features are enabled: actor lifecycle events are logged (configure verbosity with `RUST_LOG`, for example `RUST_LOG=picard=trace`), and cumulative actor/message counters are shown in the UI.
|
Every person, rabbit, wolf, and partnership is a Picard actor that owns its state and applies its own rules: people age, get hungry, eat, and starve; animals spend and gain energy, breed, and die; partnerships track when they last had a child. The world keeps only world-level state (terrain, resources, the shared stockpile, houses, and the calendar) and talks to entities exclusively through messages: each step it asks the actors for their state, decides movement and opportunities with the Rune behavior scripts, and sends the actors the outcome. Behavior decisions stay on the simulation thread because Rune's VM is not `Send`, and replies are processed in entity order so runs stay deterministic. Picard's tracing and runtime statistics features are enabled: actor lifecycle events are logged (configure verbosity with `RUST_LOG`, for example `RUST_LOG=picard=trace`), and cumulative actor/message counters are shown in the UI.
|
||||||
|
|
||||||
The server enables Picard's console by default and exposes its JSON request/response protocol over `ws://127.0.0.1:7878/api/actors/ws`. Send one JSON request per text frame; each receives one response. For example, `{"id":1,"op":"actors"}` lists actors, and `{"id":2,"op":"inspect","path":"/user/person-1"}` inspects one (actors are named `person-<id>`, `rabbit-<id>`, `wolf-<id>`, and `partnership-<id>`). Other operations include `messages`, `pause`, `resume`, `clear`, `stop`, `restart`, and `send`. The route dispatches console requests through the simulation thread to Picard's actor system. `cargo run --no-default-features` disables the endpoint; `--features console` enables it explicitly.
|
## Cluster
|
||||||
|
|
||||||
|
The world is split into vertical strips of columns called shards. Each shard is a separate process with its own Picard actor system and its own database. `sim` starts as a gateway: it spawns `SIM_SHARDS` (default 3) copies of itself with the `shard` subcommand, steps them in lockstep, and merges their views so the UI sees one world. Edits from the UI are routed to the shard that owns them.
|
||||||
|
|
||||||
|
People, animals and carts that cross a shard border are transferred with their whole state and inventory to the neighbouring shard's actor system. They keep their address, and messages to them are forwarded transparently, so actors never know which shard they run on. Shards talk to each other directly over WebSockets (`/remote` for actor messages and transfers, `/control` for the gateway protocol).
|
||||||
|
|
||||||
|
## Economy
|
||||||
|
|
||||||
|
Every shard founds markets and workplaces: farms, woodcutters, quarries, hunters, toolmakers and cartwrights. Adults are hired, commute to their job site each day, work their shift, sell the goods at a market, get paid and go home. Farmers sow fields that ripen into crops; gatherers fell trees, quarry stone and hunt; crafters buy materials and make tools and carts. Markets are actors with their own coins and stock; prices follow supply and demand, and buyers and sellers haggle. Merchants carry surplus goods to markets on other shards. Markets, workplaces, carts and crop fields are drawn on the map, and the side panel shows each person's job and coins.
|
||||||
|
|
||||||
|
## Actor Console
|
||||||
|
|
||||||
|
The server enables Picard's console by default and exposes its JSON request/response protocol over `ws://127.0.0.1:7878/api/actors/ws`. The connection reaches shard 0; use `?shard=N` to connect to shard `N`. Every shard registers the others as console peers named `shard-N`: `peers` lists them, `@shard-1 actors` forwards a command, and `send @shard-1 PATH JSON` sends a message to a remote actor. Send one JSON request per text frame; each receives one response. For example, `{"id":1,"op":"actors"}` lists actors, and `{"id":2,"op":"inspect","path":"/user/person-1"}` inspects one (actors are named `person-<id>`, `rabbit-<id>`, `wolf-<id>`, and `partnership-<id>`). Other operations include `messages`, `pause`, `resume`, `clear`, `stop`, `restart`, and `send`. Console requests are answered by the shard's actor system without waiting for the simulation thread. `cargo run --no-default-features` disables the endpoint; `--features console` enables it explicitly.
|
||||||
|
|
||||||
The simulation script is the root [config.rn](./config.rn), a single Rune file. Its `pub const` items hold defaults and tuning values: map dimensions and terrain generation, initial population and resources, calendar and life-stage defaults, hunger, gathering, construction, partnerships and births, animal behavior, and edit limits. Its `pub fn person`, `pub fn rabbit`, and `pub fn wolf` items are the agent behaviours, and they can use the constants directly. Every constant the server reads must be defined; missing or invalid values are reported at startup. `SIM_CONFIG` can point to a different Rune file. Docker Compose mounts the repository's `config.rn` read-only into the app container.
|
The simulation script is the root [config.rn](./config.rn), a single Rune file. Its `pub const` items hold defaults and tuning values: map dimensions and terrain generation, initial population and resources, calendar and life-stage defaults, hunger, gathering, construction, partnerships and births, animal behavior, and edit limits. Its `pub fn person`, `pub fn rabbit`, and `pub fn wolf` items are the agent behaviours, and they can use the constants directly. Every constant the server reads must be defined; missing or invalid values are reported at startup. `SIM_CONFIG` can point to a different Rune file. Docker Compose mounts the repository's `config.rn` read-only into the app container.
|
||||||
|
|
||||||
|
|
@ -12,7 +24,7 @@ The in-browser script editor edits the same file contents. Applied edits take ef
|
||||||
|
|
||||||
The **Calendar and life-stage durations** panel configures ticks per day and the number of days, weeks, and months in larger calendar units. Each human life stage has an inclusive minimum/maximum duration in in-game days; the world samples within that range when a person enters the stage. These runtime settings are stored with the world snapshot and take precedence over the config defaults until the world is reset.
|
The **Calendar and life-stage durations** panel configures ticks per day and the number of days, weeks, and months in larger calendar units. Each human life stage has an inclusive minimum/maximum duration in in-game days; the world samples within that range when a person enters the stage. These runtime settings are stored with the world snapshot and take precedence over the config defaults until the world is reset.
|
||||||
|
|
||||||
New people, including babies, start as toddlers and age through child, teen, young adult, adult, older adult, elderly, and death. By default, the starting village has nine young and mature adults and a reserve of 60 food so it can form partnerships and sustain its population. People pathfind around blocked terrain when travelling home for food. Unpartnered people from young adulthood onward can form a single partnership with a nearby eligible person. Each partnership can have one toddler per calendar week when both partners are in stages marked fertile, the stockpile has enough food, and housing capacity permits it. By default, the village has capacity for 12 people, with each additional house increasing capacity by three; these values are configurable. Young adult and adult stages are fertile by default; fertility can be changed per stage in the calendar panel. Partnerships are Picard actors as well as people and animals.
|
New people, including babies, start as toddlers and age through child, teen, young adult, adult, older adult, elderly, and death. By default, the 192×128 world starts with 180 people (60 of them young adults), 1200 rabbits, 80 wolves and 1200 food, shared between the shards. People pathfind around blocked terrain when travelling home for food. Unpartnered people from young adulthood onward can form a single partnership with a nearby eligible person. Each partnership can have one toddler per calendar week when both partners are in stages marked fertile, the stockpile has enough food, and housing capacity permits it. By default, the village has capacity for 12 people, with each additional house increasing capacity by three; these values are configurable. Young adult and adult stages are fertile by default; fertility can be changed per stage in the calendar panel. Partnerships, markets, workplaces and carts are Picard actors as well as people and animals.
|
||||||
|
|
||||||
The UI is built automatically as part of `cargo run` and embedded in the server binary. There is no separate manual UI build step.
|
The UI is built automatically as part of `cargo run` and embedded in the server binary. There is no separate manual UI build step.
|
||||||
|
|
||||||
|
|
@ -26,7 +38,7 @@ Install Rust with `rustup`. The build script installs the `wasm32-unknown-unknow
|
||||||
just dev
|
just dev
|
||||||
```
|
```
|
||||||
|
|
||||||
Open [http://127.0.0.1:7878](http://127.0.0.1:7878). The database is SQLite at `sim.db` in the current directory, and the initial seed is `42`. You can also run directly with `cargo run`.
|
Open [http://127.0.0.1:7878](http://127.0.0.1:7878). Each shard keeps a SQLite database `sim-shard-N.db` in the current directory, and the initial seed is `42`. You can also run directly with `cargo run`.
|
||||||
|
|
||||||
To choose a seed on the first run:
|
To choose a seed on the first run:
|
||||||
|
|
||||||
|
|
@ -34,7 +46,7 @@ To choose a seed on the first run:
|
||||||
cargo run -- 123
|
cargo run -- 123
|
||||||
```
|
```
|
||||||
|
|
||||||
An existing database snapshot takes precedence on later runs, including over a new command-line seed. Use the UI's **Reset** button to start a fresh world with its current seed, or **Regenerate** to choose a different seed.
|
Existing shard snapshots take precedence on later runs, including over a new command-line seed. If the shards' snapshots don't match (for example after changing `SIM_SHARDS`), all shards start fresh. Use the UI's **Reset** button to start a fresh world with its current seed, or **Regenerate** to choose a different seed.
|
||||||
|
|
||||||
### Run with Just
|
### Run with Just
|
||||||
|
|
||||||
|
|
@ -60,7 +72,7 @@ Install Docker with the Compose plugin, then run `just run`. This stops any exis
|
||||||
docker compose up --build -d
|
docker compose up --build -d
|
||||||
```
|
```
|
||||||
|
|
||||||
Open [http://localhost:7878](http://localhost:7878). Compose waits for Postgres to become healthy before starting the app. Postgres data is stored in the `postgres_data` volume and survives container restarts, `just stop`, and `docker compose down`.
|
Open [http://localhost:7878](http://localhost:7878). Postgres creates one database per shard (`sim_shard_0` to `sim_shard_2`) from [docker/postgres-init.sql](./docker/postgres-init.sql) when the volume is first created; add databases there when raising `SIM_SHARDS`, and recreate the volume after changing the script. Compose waits for Postgres to become healthy before starting the app. Postgres data is stored in the `postgres_data` volume and survives container restarts, `just stop`, and `docker compose down`.
|
||||||
|
|
||||||
To remove the database volume and all saved simulation data as well:
|
To remove the database volume and all saved simulation data as well:
|
||||||
|
|
||||||
|
|
@ -72,36 +84,39 @@ The Compose file uses development-only database credentials. Change them before
|
||||||
|
|
||||||
## Database Configuration
|
## Database Configuration
|
||||||
|
|
||||||
The server selects its database through `DATABASE_URL`:
|
Each shard selects its database through `SHARD_DATABASE_URL`. `{shard}` in the URL is replaced with the shard index and is required when there is more than one shard:
|
||||||
|
|
||||||
| Backend | Example URL |
|
| Backend | Example URL |
|
||||||
| ---------------- | --------------------------------------- |
|
| ---------------- | ------------------------------------------------- |
|
||||||
| SQLite (default) | `sqlite://sim.db?mode=rwc` |
|
| SQLite (default) | `sqlite://sim-shard-{shard}.db?mode=rwc` |
|
||||||
| Postgres | `postgres://sim:sim@localhost:5432/sim` |
|
| Postgres | `postgres://USER:PASSWORD@localhost:5432/sim_shard_{shard}` |
|
||||||
|
|
||||||
For example, run locally against a reachable Postgres server with:
|
For example, run locally against a reachable Postgres server that has the shard databases with:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
DATABASE_URL=postgres://sim:sim@localhost:5432/sim cargo run
|
SHARD_DATABASE_URL='postgres://USER:PASSWORD@localhost:5432/sim_shard_{shard}' cargo run
|
||||||
```
|
```
|
||||||
|
|
||||||
The simulation is saved as a snapshot about once per second and restored at startup. SQLite is convenient for local testing; Compose configures the app to use Postgres.
|
Every shard saves its part of the world as a snapshot about once per second and restores it at startup. SQLite is convenient for local testing; Compose configures the app to use Postgres.
|
||||||
|
|
||||||
`SIM_ADDR` sets the listen address. It defaults to `127.0.0.1:7878`; containers set it to `0.0.0.0:7878`.
|
`SIM_SHARDS` sets the number of shards (default 3). `SIM_ADDR` sets the gateway's listen address. It defaults to `127.0.0.1:7878`; containers set it to `0.0.0.0:7878`.
|
||||||
|
|
||||||
## Project Layout
|
## Project Layout
|
||||||
|
|
||||||
- `src/world.rs` contains the simulation and snapshot serialization.
|
- `src/world.rs` contains the simulation and snapshot serialization; `src/world/` holds the actors, shard routing and economy.
|
||||||
- `src/main.rs` runs the Axum server, simulation thread, and HTTP/WebSocket API.
|
- `src/main.rs` runs the gateway's Axum server and HTTP/WebSocket API, and the `shard` subcommand.
|
||||||
|
- `src/cluster/` contains the gateway, the shard node, the inter-shard transport and the control protocol.
|
||||||
- `src/storage.rs` provides SQLite and Postgres snapshot storage.
|
- `src/storage.rs` provides SQLite and Postgres snapshot storage.
|
||||||
- `config.rn` is the Rune simulation script: `pub const` defaults and tuning values plus `pub fn` behaviours.
|
- `config.rn` is the Rune simulation script: `pub const` defaults and tuning values plus `pub fn` behaviours.
|
||||||
- `ui/` contains the Leptos frontend, compiled to Wasm by `build.rs`.
|
- `ui/` contains the Leptos frontend, compiled to Wasm by `build.rs`.
|
||||||
- `Dockerfile`, `compose.yaml`, and `Justfile` provide container and developer workflows.
|
- `Dockerfile`, `compose.yaml`, `docker/postgres-init.sql` and `Justfile` provide container and developer workflows.
|
||||||
|
|
||||||
## Tests
|
## Tests
|
||||||
|
|
||||||
Run the server tests with:
|
Run the server tests with:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
cargo test --bin sim
|
cargo test --workspace --no-default-features
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`tests/cluster.rs` starts a real multi-process cluster.
|
||||||
|
|
@ -4,7 +4,8 @@ services:
|
||||||
environment:
|
environment:
|
||||||
SIM_ADDR: 0.0.0.0:7878
|
SIM_ADDR: 0.0.0.0:7878
|
||||||
SIM_CONFIG: /etc/sim/config.rn
|
SIM_CONFIG: /etc/sim/config.rn
|
||||||
DATABASE_URL: postgres://sim:sim@db:5432/sim
|
SIM_SHARDS: "3"
|
||||||
|
SHARD_DATABASE_URL: postgres://sim:sim@db:5432/sim_shard_{shard}
|
||||||
volumes:
|
volumes:
|
||||||
- ./config.rn:/etc/sim/config.rn:ro
|
- ./config.rn:/etc/sim/config.rn:ro
|
||||||
ports:
|
ports:
|
||||||
|
|
@ -27,6 +28,7 @@ services:
|
||||||
retries: 10
|
retries: 10
|
||||||
volumes:
|
volumes:
|
||||||
- postgres_data:/var/lib/postgresql/data
|
- postgres_data:/var/lib/postgresql/data
|
||||||
|
- ./docker/postgres-init.sql:/docker-entrypoint-initdb.d/shards.sql:ro
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
|
|
||||||
volumes:
|
volumes:
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,4 @@
|
||||||
|
-- One database per shard; runs only when the postgres_data volume is created.
|
||||||
|
CREATE DATABASE sim_shard_0 OWNER sim;
|
||||||
|
CREATE DATABASE sim_shard_1 OWNER sim;
|
||||||
|
CREATE DATABASE sim_shard_2 OWNER sim;
|
||||||
Loading…
Reference in New Issue