mud/docs/how/if-journey.rst

if journey — from arcade terminal to moldable worlds
=====================================================

This doc tracks the IF (interactive fiction) integration effort for the MUD engine. It's a living document — updated as research progresses and decisions are made. For detailed technical analysis, see the two interpreter audits (``viola-embedding-audit.rst`` and ``zvm-embedding-audit.rst``) and the original integration notes (``if-integration.txt``).

the vision — five levels of integration
----------------------------------------

Five levels emerged from design discussion. They represent a spectrum of how deeply IF worlds integrate with the MUD world, from simplest to most ambitious.

Level 1 — terminal mode
~~~~~~~~~~~~~~~~~~~~~~~~

Subprocess (dfrotz), text in/out, spectators see text scrolling on a screen. Player sits at an arcade terminal, plays Zork. Others in the room see the output. The IF world is opaque — a black box.

Implemented. See ``docs/how/if-terminal.txt`` for how the system works.

Level 2 — inspectable world
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Embedded interpreter. MUD can READ z-machine state. Know what room the player is in, describe it to spectators, track progress. Spectators don't just see text — they know "jared is in the Trophy Room." Read-only bridge between MUD and IF world.

Level 3 — moldable world
~~~~~~~~~~~~~~~~~~~~~~~~~

Embedded interpreter. MUD can READ AND WRITE z-machine state. Inject items into IF game objects. Put a note in the Zork mailbox. The z-machine object tree (parent/child/sibling with attributes and properties) becomes accessible from MUD code. Two-way bridge. Game world is modifiable from outside.

Level 4 — shared world
~~~~~~~~~~~~~~~~~~~~~~~

Multiple MUD players mapped into the same z-machine instance. Each has their own player object. Independent inventory and position. MojoZork's MultiZork proved this works for V3 games. The IF world is a zone in the MUD that multiple players inhabit simultaneously.

Level 5 — transcendent world
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Z-machine object tree and MUD entity model are unified. An item in Zork IS a MUD item. Pick it up in the IF world, carry it back to the MUD. The mailbox exists at coordinates in both worlds simultaneously. Full bidirectional entity bridge.

Note: Level 1 uses subprocess. Levels 2-5 require being inside the interpreter. Once you're at level 2, the jump to 3 is small (reading memory vs writing it). Level 4 is the MojoZork leap. Level 5 is the dream.

what we know — audit findings
------------------------------

Two Python z-machine interpreters were audited in detail. Don't repeat everything here — see the audit docs for details. Focus on decision-relevant facts.

viola (DFillmore/viola)
~~~~~~~~~~~~~~~~~~~~~~~

Can run games today. All V1-V5 opcodes implemented, partial V6-V8. But global state is deeply tangled (13/18 modules with mutable globals). Multiple instances in one process: not feasible without major refactor.

pygame dependency is cleanly separable (adapter pattern, unidirectional). ``sys.exit()`` in error paths (8+ locations) — needs patching for server use. Memory leaks (5+ unbounded growth patterns) — fixable with cleanup hooks.

Object tree accessors exist and are wired through all opcodes. Full quetzal save/restore working.

See: ``docs/how/viola-embedding-audit.rst``

zvm (sussman/zvm)
~~~~~~~~~~~~~~~~~

Cannot run any game. ~46/108 opcodes implemented, including ZERO input opcodes. But instance-based state (mostly clean, two minor global leaks fixable in ~20 lines). Multiple instances in one process: structurally possible.

IO abstraction is excellent — purpose-built for embedding (abstract ZUI with stubs). Proper exception hierarchy, zero ``sys.exit()`` calls. No memory leaks, clean bounded state.

Object tree parser has complete read API, mostly complete write API. BUT: many object opcodes not wired up at CPU level. Save parser works, save writer is stubbed.

See: ``docs/how/zvm-embedding-audit.rst``

The verdict from the audits: "zvm has the architecture you'd want. viola has the implementation you'd need."

MojoZork (C, Ryan C. Gordon)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

V3 only but has working multiplayer telnet server (MultiZork). Proves multiplayer z-machine works (up to 4 players, separate inventories). ``runInstruction()`` for single-step execution — the async pattern we want. Excellent reference architecture, not directly usable (C only).

the hybrid path — option D
---------------------------

This emerged from comparing the audits side by side. Use zvm's architecture (clean IO abstraction, instance-based state, exception model) as the skeleton. Port viola's working opcode implementations into it.

Why this is attractive:

- zvm's opcodes take ``self`` (ZCpu) and access state through instance attributes
- viola's opcodes use module-level globals (``zcode.game.PC``, ``zcode.memory.data``, etc)
- porting means translating global lookups to instance attribute lookups
- that's mechanical, not creative — could port 5-10 opcodes per hour once the pattern is established
- gets the clean design with the working code

Why this could be harder than it sounds:

- viola and zvm may represent z-machine internals differently
- memory layout assumptions, stack frame format, string encoding details
- porting opcodes may require porting the data structures they operate on
- need to verify each ported opcode against the z-machine spec, not just translate

Estimated effort: medium. Less than finishing zvm from scratch, less than refactoring viola's globals. But not trivial.

the object tree — key to moldable worlds
-----------------------------------------

This is what makes levels 3-5 possible. The z-machine has an object tree — every game entity is a node with parent/child/sibling pointers, attributes (boolean flags), and properties (variable-length data).

What the object tree gives us:

- Read what room the player is in (player object's parent)
- Read container contents (children of the container object)
- Inject items (create objects, parent them to containers)
- Modify game state (set/clear attributes, change properties)
- Query the dictionary (what words the parser recognizes)

Both interpreters have object tree parsers:

- viola: complete read + write, all opcodes wired, working
- zvm: complete read, mostly complete write (missing ``set_attr``/``clear_attr``), many opcodes unwired at CPU level, bug in ``insert_object``

The dictionary problem (level 3+)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Injecting an object into the mailbox works for "look in mailbox" — the game iterates children and prints short names. But "take [new item]" fails unless the word exists in the game's dictionary. The dictionary is baked into the story file.

Options:

- Add words to dictionary at runtime (memory surgery — relocating/expanding the dictionary)
- Intercept input before the parser and handle custom items at the MUD layer
- Use existing dictionary words for injected items ("note", "scroll", "key" are common)
- Hybrid: intercept unrecognized words, check if they match MUD-injected items, handle outside z-machine

This is a level 3-5 problem. Not a blocker for levels 1-2.

games we care about
-------------------

The games that motivated this work:

The Wizard Sniffer (Buster Hudson, 2017)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You play a pig who sniffs out wizards. IFComp winner, XYZZY winner. Screwball comedy. The pig/wizard game that started this. Z-machine format. Would need V5 support.

Lost Pig (Admiral Jota, 2007)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Grunk the orc chases an escaped pig. IFComp winner, 4 XYZZY awards. Famous for its responsive parser and comedy writing. Z-machine format. V5.

Zork I, II, III
~~~~~~~~~~~~~~~

The classics. Everyone should play them. V3.

Hitchhiker's Guide to the Galaxy
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Funny, frustrating, great spectator game. V3.

Also: Anchorhead, Photopia, Spider and Web, Shade, Colossal Cave.

V3 covers the Infocom catalog. V5 covers most modern IF including the pig games. V8 covers big modern Inform 7 games but is lower priority.

Note: no Python Glulx interpreter exists. Games that target Glulx (some modern Inform 7) are out of scope unless we subprocess to a C interpreter.

architecture fit
----------------

How this fits the existing MUD architecture. The codebase is ready:

Mode stack
~~~~~~~~~~

Push "if" mode. All input routes to IF handler, bypassing command dispatch. Same pattern as editor mode. Already proven.

Input routing
~~~~~~~~~~~~~

``server.py`` shell loop checks ``player.mode``. Add elif for "if" mode, route to ``if_game.handle_input()``. Same as ``editor.handle_input()``.

Room-local broadcasting
~~~~~~~~~~~~~~~~~~~~~~~~

Implemented for IF. ``broadcast_to_spectators()`` in ``if_session.py`` sends to all players at the same (x,y) location. Pattern will be reused for ambient messages, weather, room events.

State storage
~~~~~~~~~~~~~

Quetzal saves stored as files in ``data/if_saves/{player}/{game}.qzl``. Filesystem approach (simpler than SQLite blobs for dfrotz, which already writes to disk).

Terminal game object
~~~~~~~~~~~~~~~~~~~~

A room object (or coordinate-anchored object) that hosts IF sessions. Players "use" it to enter IF mode. Pattern extends to other interactive objects.

open questions
--------------

Things we haven't figured out yet. Update this as questions get answered.

1. V3 opcode footprint
~~~~~~~~~~~~~~~~~~~~~~

How many of the ~62 missing zvm opcodes are actually exercised by V3 games? V3 uses a smaller subset. If we target V3 first, the hybrid might need 30 ported, not 62.

UPDATE: Opcode tracing (via ``scripts/trace_zmachine.py``) found Zork 1 uses 69 opcodes. zvm had 36 implemented. 33 were ported from viola. All 69 are now implemented in the hybrid interpreter (``src/mudlib/zmachine/``).

Remaining gaps: save/restore (QuetzalWriter needs completion) and sread tokenization.

2. zvm/viola memory layout compatibility
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Do they represent z-machine memory the same way? Both use bytearrays, but header parsing, object table offsets, string encoding — are these compatible enough that porting opcodes is translation, or is it a deeper rewrite?

3. Async model
~~~~~~~~~~~~~~

Both interpreters have blocking run loops. Options:

- ``run_in_executor`` (thread pool) — standard pattern, adds latency
- extract ``step()`` and call from async loop — zvm audit says this is ~5 lines
- run in separate thread with queue-based IO — more complex but natural

Which is best for the MUD's tick-based game loop?

4. Multiplayer z-machine
~~~~~~~~~~~~~~~~~~~~~~~~

MojoZork does this for V3. What would it take for V5? The V5 object model is larger (65535 objects vs 255). Do V5 games assume single-player in ways that break multiplayer?

5. Game file licensing
~~~~~~~~~~~~~~~~~~~~~~

Infocom games are abandonware but not legally free. Modern IF games (Lost Pig, Wizard Sniffer) are freely distributable. Need to figure out what we can bundle vs what players bring.

6. Dictionary injection feasibility
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

How hard is it to add words to a z-machine dictionary at runtime? The dictionary is in static memory. Adding words means expanding it, which means relocating it if there's no space. Is this practical?

what to do next
---------------

Concrete next steps, roughly ordered. Update as items get done.

- [x] trace V3 opcode usage: run zork through viola with opcode logging, get the actual set of opcodes a real game uses. this tells us how much porting work the hybrid path actually requires. (done — found 69 opcodes, see ``scripts/trace_zmachine.py``)

- [ ] compare memory layouts: look at how viola and zvm represent z-machine memory, object tables, string tables. determine if opcode porting is mechanical translation or deeper adaptation.

- [x] prototype the hybrid: pick 5-10 common opcodes, port them from viola to zvm's architecture. see how the pattern feels. if it's smooth, the hybrid is viable. if every opcode is a battle, reconsider. (done — all 69 Zork 1 opcodes ported, hybrid interpreter lives in ``src/mudlib/zmachine/``)

- [x] build level 1 prototype: regardless of interpreter choice, implement the terminal object, IF mode, and subprocess dfrotz path. this proves the MUD-side architecture (mode stack, spectators, save/restore) independently of the interpreter question. (done — see ``docs/how/if-terminal.txt``)

- [ ] study MojoZork's multiplayer model: read the MultiZork source for how it handles multiple players in one z-machine. document the pattern for our eventual level 4.

- [x] find the game files: locate freely distributable z-machine story files for the games we care about. Wizard Sniffer, Lost Pig, Zork (if legally available). (zork1.z3 bundled in content/stories/)

milestone — Zork 1 playable in hybrid interpreter
--------------------------------------------------

The hybrid interpreter (zvm architecture + ported viola opcodes) can now run Zork 1. This is the first working implementation targeting levels 2-5 — inspectable, moldable, and shared worlds. Level 1 (terminal mode) uses subprocess dfrotz; this is the embedded path.

What works:

- 69 V3 opcodes ported, all Zork 1-required opcodes implemented
- key implementations: ``op_test`` (conditional logic), ``op_verify`` (story file checksums), ``sread`` with ZLexer tokenization (parsing player input)
- ``step()`` method for async MUD integration — single instruction at a time, no blocking loop
- instruction trace deque (last 20 instructions) for debugging state errors
- smoke test: ``scripts/run_zork1.py`` runs the game headless, exercises core opcode paths

What this enables:

- Read z-machine state (object tree, variables) from MUD code (level 2)
- Write z-machine state, inject items, modify world (level 3)
- Multiplayer instances (level 4, following MojoZork patterns)
- Entity bridge (level 5, further out)

The step-based execution model means IF sessions can run in the async MUD game loop without blocking. Each player command advances their z-machine instance by N instructions (until output or a stopping condition). The trace deque captures the last 20 instructions for debugging unexpected state.

related documents
-----------------

``docs/how/if-terminal.txt`` — how the level 1 IF system works (implementation reference)

``docs/how/if-integration.txt`` — original research and integration plan (predates audits)

``docs/how/viola-embedding-audit.rst`` — detailed viola architecture audit

``docs/how/zvm-embedding-audit.rst`` — detailed zvm architecture audit with comparison

``docs/how/architecture-plan.txt`` — MUD engine architecture plan

``DREAMBOOK.md`` — project vision and philosophy