diff --git a/docs/how/multiplayer-zmachine-design.rst b/docs/how/multiplayer-zmachine-design.rst new file mode 100644 index 0000000..94e6e1f --- /dev/null +++ b/docs/how/multiplayer-zmachine-design.rst @@ -0,0 +1,622 @@ +========================================== +multiplayer z-machine — shared world design +========================================== + +level 4 means multiple MUD players sharing one z-machine instance. one game +world, multiple explorers. MojoZork proved this works for Zork 1 with 4 +concurrent players over telnet. this doc designs how it works in our Python +interpreter, using MojoZork's lessons but improving on its limitations. + +see ``docs/how/mojozork-audit.rst`` for the full MojoZork technical analysis. +see ``docs/how/if-journey.rst`` for the five-level vision (this is level 4). + + +why this is hard +================ + +the z-machine is a single-player virtual machine. one program counter, one +stack, one set of global variables. the game assumes there's one "you" in +the world. multiplayer means faking multiple players inside a system that +was never designed for it. + +MojoZork solved this with brute force: swap the entire execution context +(PC, stack, globals) every time a different player takes a turn. it works, +but it's hardcoded to one build of Zork 1 — 10 specific global variable +indices, 6 bytecode patch addresses, one known object layout. + +our design keeps the same core trick (context swapping) but wraps it in +a config-driven system that can support multiple games. + + +the core trick: context swapping +================================ + +the z-machine blocks on input (``op_sread`` in V3, ``op_aread`` in V5+). +when it blocks, the entire interpreter state is frozen: PC points past the +read instruction, the stack is stable, globals hold current values. this is +the natural swap point. + +the multiplayer loop:: + + 1. player sends a command + 2. restore that player's saved context (PC, stack, globals, object data) + 3. feed the command to the interpreter + 4. run until the next READ (interpreter blocks again) + 5. capture output, save context back out + 6. repeat for next player with input + +this is what MojoZork does. it's also what we'll do. the z-machine is +inherently single-threaded — no way around processing one player at a time. + + +shared vs per-player state +========================== + +which z-machine state is shared, and which needs per-player copies? + +======================== ========= ================ ================================ +state shared? per-player copy? notes +======================== ========= ================ ================================ +static/high memory yes no code and strings, immutable +dynamic memory yes no world state (object tree, etc) +program counter no yes each player paused at own READ +call stack + locals no yes different execution contexts +data stack no yes operand stack diverges +player-specific globals no yes location, deaths, flags, etc +player object entry no yes parent/attrs/properties +touchbits (visited rooms) no yes each player's exploration history +PRNG state yes no shared randomness is fine +======================== ========= ================ ================================ + +this matches MojoZork's split exactly. the key insight: dynamic memory is ONE +copy shared by all players. most of the game world (objects, rooms, items, +puzzle state) is shared. only the player's execution state and a handful of +"this is about me specifically" globals need per-player copies. + +the consequence: when player A picks up the brass lantern, player B can't pick +it up either. the object tree is shared. this is correct — it's the same world. +but when player A is in the kitchen and player B is in the cellar, they each +have their own location global. the game "sees" each player in their own room +during their turn. + + +player object strategy +====================== + +each player needs a z-machine object to exist in the world — to have a +location, to hold inventory, to be visible to other players. + +**V3 (Zork 1):** 255 max objects, Zork uses ~250. objects 251-255 are +available as player slots. this limits V3 to ~5 concurrent players. + +**V5/V8 (Lost Pig, etc):** 65535 max objects. even a large game uses a few +thousand at most. tens of thousands of spare slots. + +MojoZork virtualizes player objects — stores them outside the real object +table and intercepts all object access with function pointer hooks. this is +clever but adds complexity and tight coupling to every object opcode. + +**our approach: use real object table slots.** + +for V3, claim spare slots at the top of the range (251+). for V5/V8, claim +a block starting well above the game's highest object. player objects live +in the real object table, initialized with appropriate parent (starting room), +attributes (visible), and properties (name, description). + +advantages over MojoZork's virtualization: + +- no function pointer hooks on every object access +- standard object opcodes work unmodified +- properties live in real memory (no fake address ranges) +- simpler code, fewer places for bugs + +disadvantages: + +- requires knowing the game's max object count (to avoid collisions) +- V3 is limited to ~5 players (real constraint, but acceptable) +- must write object data into dynamic memory during initialization + +to determine available slots, the game config specifies a safe range, or we +scan the object table at startup to find the highest used object. + +comparison with MojoZork:: + + MojoZork our approach + ───────────────────────────────── ────────────────────────────────── + virtual objects outside table real objects in table + function pointer hooks no hooks needed + fake address range (0xFE00+) real memory addresses + hardcoded to Zork objects 251-254 config-driven range per game + 4 player limit (hardcoded) V3: ~5, V8: effectively unlimited + + +identifying player-specific state +================================= + +the hardest part of supporting a new game: figuring out which global variables +need per-player copies. + +MojoZork hardcodes 10 Zork 1 globals (location, deaths, lit, verbose, etc). +that works for one game but doesn't scale. + +**our approach: per-game TOML config files.** + +each supported game gets a multiplayer config that maps globals to meanings:: + + # content/stories/zork1.multiplayer.toml + + [game] + name = "zork1" + version = 3 + story_file = "zork1.z3" + + [player_object] + original_id = 4 # the game's built-in "player" object + start_room = 180 # West of House (object ID) + room_container = 82 # parent of all rooms + + [player_globals] + # global variable indices that need per-player copies + # index = human-readable name (for debugging/logging) + 0 = "location" + 60 = "lucky" + 61 = "deaths" + 62 = "dead" + 66 = "lit" + 70 = "superbrief" + 71 = "verbose" + 72 = "alwayslit" + 133 = "loadallowed" + 139 = "coffin_held" + + [attributes] + invisible = 7 # attribute to hide player during swap + touchbit = 3 # room-visited flag (per-player) + + [player_slots] + start = 251 # first available object ID for players + max = 255 # last available (V3 ceiling) + + [bytecode_patches] + # addresses where the game hardcodes the player object ID. + # during swap, these bytes get patched to the current player's object ID. + # found by disassembling and searching for references to object 4. + addresses = [0x6B3F, 0x93E4, 0x9411, 0xD748, 0xE1AF, 0x6B88] + +how to create these configs for new games: + +1. **location global** — always global 0 (``read_global(0x10)``). this is + standard across all z-machine games. +2. **other player globals** — requires disassembly and analysis. look for + globals that change on room entry, death, or mode switches. the + ``scripts/zmachine_inspect.py`` tool can help, and we could build a + tracing tool that logs global writes during single-player play. +3. **bytecode patches** — search disassembly for literal references to the + player object ID. not all games will need these (some reference the + player via a global, not a literal). +4. **object slots** — load the game, count objects, pick a safe range above + the highest used object. + +this is manual work per game. but multiplayer is inherently game-specific — +each game makes different assumptions about the player. a config file is the +honest way to capture those assumptions. + +future tooling (not in scope for initial implementation): + +- a tracing mode that logs all global writes during play, flagging candidates + for per-player globals +- a disassembler that searches for literal object ID references +- a test harness that runs two contexts against the same game and detects + state conflicts + + +the turn model +============== + +MojoZork uses implicit round-robin: whoever sends input next gets their turn. +there's no explicit queue or scheduling. + +**our approach: input queue, process one at a time.** + +:: + + SharedIFSession + ├── input_queue: asyncio.Queue[(player_name, command)] + ├── processing_lock: asyncio.Lock + └── tick(): + while not input_queue.empty(): + player, command = input_queue.get() + swap_in(player) + feed(command) + run_until_read() + capture_output(player) + swap_out(player) + +when a MUD player types a command in IF mode, it goes into the shared input +queue. the session processes queued commands one at a time in FIFO order. + +latency: in a MUD, players expect immediate response. with context swapping, +each player's command takes one z-machine "turn" to process (typically +milliseconds — our interpreter runs thousands of instructions per ms). even +with 4-5 players, the queue drains in well under 100ms. this is fast enough +that players won't notice the serialization. + +if latency becomes a problem (unlikely), we could process the queue on every +MUD tick rather than waiting for it to drain, giving each player one turn per +tick in round-robin order. + +comparison with MojoZork:: + + MojoZork our approach + ───────────────────────────────── ────────────────────────────────── + implicit (whoever sends input) explicit FIFO queue + blocking telnet read per player async queue, non-blocking + single-threaded C async Python (event loop) + no fairness guarantee FIFO ordering + + +data structures +=============== + +the core types for multiplayer:: + + @dataclass + class PlayerContext: + """per-player execution state for a shared z-machine.""" + player_name: str + mud_player: Player # reference to MUD player entity + + # execution state (saved when swapped out) + program_counter: int + call_stack: list[ZRoutine] # full call frame chain + data_stack: list[int] # operand stack + + # player-specific z-machine state + player_globals: dict[int, int] # global index -> value + player_object_id: int # our object in the z-machine + object_entry: bytes # saved object table entry + property_data: bytes # saved property table + touchbits: bytearray # visited-room flags + + # i/o + output_buffer: list[str] # accumulated output this turn + + # session + hash_code: str # for reconnection (6 chars) + joined_at: float # timestamp + + + class SharedIFSession: + """multiplayer z-machine session — one game, many players.""" + zmachine: ZMachine # single shared interpreter + story_name: str + game_config: GameMultiplayerConfig + + contexts: dict[str, PlayerContext] # player_name -> context + current_player: str | None # who's executing right now + input_queue: asyncio.Queue # (player_name, command) pairs + spectators: set[Player] # watchers who aren't playing + + # persistence + save_path: Path + moves_since_save: int + +the ``GameMultiplayerConfig`` is loaded from the TOML file described above. +it holds the per-game mappings (player globals, object slots, patches, etc). + + +the swap in detail +================== + +**swap_in(player_name)** — restore a player's context before their turn:: + + 1. load PlayerContext for this player + 2. restore PC: + opdecoder.program_counter = ctx.program_counter + 3. restore stack: + stackmanager.restore(ctx.call_stack, ctx.data_stack) + 4. restore player-specific globals: + for idx, value in ctx.player_globals.items(): + memory.write_global(idx + 0x10, value) + 5. restore player object entry: + write ctx.object_entry to object table at ctx.player_object_id + write ctx.property_data to property table address + 6. restore touchbits: + for each room object, set/clear touchbit attribute from ctx.touchbits + 7. apply bytecode patches: + for each patch address in game_config: + memory[addr] = ctx.player_object_id (low byte) + 8. mark other players' objects as invisible (set invisible attribute) + +**swap_out(player_name)** — save a player's context after their turn:: + + 1. save PC: + ctx.program_counter = opdecoder.program_counter + 2. save stack: + ctx.call_stack, ctx.data_stack = stackmanager.snapshot() + 3. save player-specific globals: + for idx in game_config.player_globals: + ctx.player_globals[idx] = memory.read_global(idx + 0x10) + 4. save player object entry: + ctx.object_entry = read object table at ctx.player_object_id + ctx.property_data = read property table + 5. save touchbits: + for each room, ctx.touchbits[i] = get_attr(room, touchbit_attr) + 6. clear visibility hacks (remove invisible from other players) + +the swap is fast. it's mostly memcpy-equivalent operations. the touchbit +loop is O(rooms) but rooms are typically < 300. + + +output routing +============== + +during a player's turn, the z-machine generates text via the UI callback. +this text should go to that player. but other players might also need to +see certain things (someone entering their room, items being taken, etc). + +**basic output routing:** all z-machine output during player A's turn goes +to player A's output buffer. after the turn, flush the buffer to their MUD +connection. + +**broadcast messages:** when a player takes an action visible to others +(enters a room, picks up an item), the MUD layer generates broadcast messages +for nearby players. this happens at the MUD level, not the z-machine level. + +:: + + after swap_out(player_a): + old_room = previous_location + new_room = ctx.player_globals[0] # location + if old_room != new_room: + broadcast_to_room(old_room, f"{player_a} leaves.") + broadcast_to_room(new_room, f"{player_a} arrives.") + +**spectators:** non-playing watchers see all output from all players, +prefixed with the player's name. same pattern as current single-player +spectator support, extended to multiplex output streams. + + +MUD integration +=============== + +how does a multiplayer IF session look from the MUD side? + +**entering a shared session:** + +a terminal object in the MUD world hosts the shared game. players "use" the +terminal to join. the first player creates the session; subsequent players +join the existing one:: + + > use terminal + [joining Zork I — 2 players already exploring] + [you are player 3, starting at West of House] + +the player's mode stack pushes "shared_if". input routes to the shared +session's input queue instead of the MUD command dispatcher. + +**leaving and rejoining:** + +players can leave with ``::quit`` (same as single-player). their context is +preserved — the player object stays in the world (marked inactive/invisible), +and their state is saved. they can rejoin later and resume where they left off. + +MojoZork uses a 6-char hash code for reconnection. we can do the same, or +just use the player's MUD identity (they're already authenticated). + +**when the last player leaves:** + +the session enters a dormant state. the shared z-machine state (dynamic +memory) is saved to disk along with all player contexts. when someone +rejoins, the session is restored. + +option: keep the session alive for some grace period (5 minutes?) before +saving and tearing down, in case players are just briefly disconnecting. + +**spectators:** + +work the same as single-player spectators. watchers see multiplexed output +from all active players, prefixed with player names. spectators can't send +commands to the z-machine. + + +persistence +=========== + +MojoZork saves to SQLite (instance state + per-player state + transcripts). +we already have Quetzal save/restore for single-player. for multiplayer +we need both the shared state and per-player contexts. + +**what to save:** + +- shared dynamic memory (one copy, same as Quetzal CMem/UMem) +- per-player: PC, call stack, data stack, player globals, object data, + touchbits +- session metadata: game name, player list, creation time + +**format options:** + +1. **extend Quetzal** — add custom chunks for per-player data. Quetzal + is extensible (IFF format). but it's designed for single-player saves. +2. **separate files** — one Quetzal-like file for shared state, one file + per player for their context. simpler but more files to manage. +3. **SQLite** — like MojoZork. one row per session, BLOBs for state. + integrates with our existing player database. + +**recommendation:** SQLite, following MojoZork's proven approach. we already +use SQLite for player accounts. add tables for multiplayer sessions and +player contexts. BLOBs for binary state (memory, stacks). + +**autosave:** every N moves (MojoZork uses 30). also save on player +join/leave and session dormancy. + + +game candidates +=============== + +not all IF games make sense for multiplayer. criteria: + +- exploration-heavy (many rooms to discover) +- items and puzzles that can be shared or divided +- enough physical space for multiple players +- not deeply single-protagonist (tight narrative, personal journey) + +**good candidates:** + +1. **Zork I** (V3) — the original. large map, many items, exploration-focused. + MojoZork already proved this works. our first target. + +2. **Lost Pig** (V8) — single location focus (farm/caves), but funny and + interactive. orc protagonist gives natural "party of adventurers" vibe. + V8 format means plenty of object slots. + +3. **Zork II / Zork III** (V3) — similar to Zork I. would need their own + multiplayer configs but the gameplay pattern works. + +4. **Enchanter trilogy** (V3/V5) — magic system could make cooperative play + interesting (different players learn different spells). + +**poor candidates:** + +- tightly scripted narrative (Photopia, Shade) +- real-time or timing-sensitive games +- games with one-protagonist arcs (most modern literary IF) +- Glulx games (out of scope — different VM entirely) + +**what we have now:** ``content/stories/zork1.z3`` and +``content/stories/LostPig.z8``. Zork I is the obvious first target since +MojoZork provides a reference implementation to validate against. + + +implementation plan +=================== + +in rough dependency order: + +**phase 1: game config and analysis tools** + +- define the multiplayer TOML config format +- write ``content/stories/zork1.multiplayer.toml`` (translating MojoZork's + hardcoded values) +- extend ``scripts/zmachine_inspect.py`` with object counting and + global-tracing modes +- analyze Lost Pig for multiplayer config (object count, candidate globals) + +**phase 2: context swapping** + +- implement ``PlayerContext`` dataclass +- implement ``swap_in()`` / ``swap_out()`` on the interpreter +- add ``stackmanager.snapshot()`` and ``stackmanager.restore()`` methods +- add player object initialization (write new objects into table) +- test: two contexts alternating on Zork 1, verify independent locations + +**phase 3: shared session** + +- implement ``SharedIFSession`` (the multiplayer equivalent of + ``EmbeddedIFSession``) +- input queue and FIFO processing +- output routing (per-player buffers) +- broadcast messages for cross-player events (room entry/exit) + +**phase 4: MUD integration** + +- shared terminal object that hosts multiplayer sessions +- join/leave/rejoin flow +- mode stack integration (``shared_if`` mode) +- spectator support (multiplexed output) + +**phase 5: persistence** + +- SQLite schema for multiplayer sessions +- save/restore shared state + per-player contexts +- autosave logic +- dormant session management (save on last leave, restore on rejoin) + +**phase 6: Lost Pig support** + +- analyze Lost Pig for multiplayer config +- write ``LostPig.multiplayer.toml`` +- test with V8 object slots (high-range allocation) +- validate context swapping works for V8/V5 (aread vs sread differences) + + +comparison table +================ + +================================ ============================== ============================== +concern MojoZork our design +================================ ============================== ============================== +player objects virtual (outside object table) real (in object table) +object access function pointer hooks standard opcodes, no hooks +player globals 10 hardcoded indices TOML config per game +bytecode patches 6 hardcoded addresses TOML config per game +player limit 4 (hardcoded) V3: ~5, V8: hundreds +game support one build of Zork 1 config-driven, multiple games +turn model implicit (whoever sends input) explicit FIFO queue +output routing callback to current connection per-player buffer + broadcast +persistence SQLite (memory BLOBs) SQLite (same approach) +reconnection 6-char hash code MUD identity (already authed) +error recovery longjmp + instance destroy exception handling + save state +language C (single file, 2500 lines) Python (multiple modules) +================================ ============================== ============================== + + +known limitations +================= + +things we're explicitly not solving in the initial implementation: + +- **no generic multiplayer** — each game needs manual analysis and a config + file. there's no way around this; games make game-specific assumptions. + +- **no concurrent execution** — one player at a time. the z-machine is + single-threaded. serialized turns are the only safe model. + +- **V3 player limit** — roughly 5 players max due to the 255-object ceiling. + acceptable for the MUD use case (small group exploration). + +- **no cross-player item interaction** — if player A drops an item in a room + and player B is in that room, B won't see it until their next turn (when + the shared object tree is visible to their context). this is a fundamental + consequence of turn-based context swapping. + +- **game-specific bytecode patches** — some games hardcode the player object + ID in z-code instructions. these patch addresses must be found manually + via disassembly. not all games will need them. + +- **save_undo** — currently stubbed as "not available" in our interpreter. + multiplayer makes undo even harder (whose undo?). punting entirely. + +- **dictionary injection** — level 3 feature (adding new words to the game). + orthogonal to multiplayer but would enhance it. not in scope here. + +- **Glulx games** — different VM, out of scope. only z-machine V3/V5/V8. + + +open questions +============== + +things to resolve during implementation: + +1. **stackmanager snapshot format** — do we deep-copy ZRoutine objects, or + serialize to bytes? deep copy is simpler; bytes are more compact for + persistence. probably start with deep copy, optimize later. + +2. **initial player setup** — when a new player joins mid-game, where do + they start? MojoZork puts everyone at West of House. we could let the + config specify a start room, or always use the game's default start. + +3. **player object properties** — what name and description does a player + object get? MojoZork uses "Adventurer" plus a number. we could use the + MUD player's name, but that requires writing a z-machine string into + the property table. + +4. **item conflict** — if player A is carrying an item and player B tries + to take it during their turn (shared object tree), what happens? the + game will say "you can't see that here" because the item's parent is + player A's object, not the room. this is probably fine — the z-machine + handles it naturally. + +5. **scoring** — most games track score in a global. is score per-player + or shared? MojoZork doesn't address this. probably per-player (add to + player_globals config).