Add research on IF possibilities
This commit is contained in:
parent
f36085c921
commit
e0910151f4
5 changed files with 1845 additions and 0 deletions
676
docs/how/if-integration.txt
Normal file
676
docs/how/if-integration.txt
Normal file
|
|
@ -0,0 +1,676 @@
|
||||||
|
interactive fiction integration — hosting z-machine games in a MUD
|
||||||
|
====================================================================
|
||||||
|
|
||||||
|
where this fits. the dreambook calls for IF games as in-world content — a room
|
||||||
|
in the MUD becomes a portal to a z-machine adventure. players enter IF mode,
|
||||||
|
play through a self-contained story, and other players see "jared is playing
|
||||||
|
zork in the arcade." fully isolated from world noise, but visible to
|
||||||
|
spectators in the same room.
|
||||||
|
|
||||||
|
the architecture is ready. we have the mode stack, input routing, editor mode
|
||||||
|
pattern. the missing pieces:
|
||||||
|
- z-machine interpreter (either embedded or subprocess)
|
||||||
|
- room-local spectator broadcasting (others see your game output)
|
||||||
|
- "terminal" game object concept (a room object that hosts IF sessions)
|
||||||
|
- save/restore integration (quetzal snapshots to sqlite)
|
||||||
|
|
||||||
|
this doc synthesizes research on three z-machine implementations and proposes
|
||||||
|
integration paths.
|
||||||
|
|
||||||
|
|
||||||
|
the z-machine landscape
|
||||||
|
=======================
|
||||||
|
|
||||||
|
three implementations evaluated:
|
||||||
|
|
||||||
|
sussman/zvm (python, pure stdlib):
|
||||||
|
- versions 1-5 claimed, but only ~40% of opcodes implemented
|
||||||
|
- interpreter HALTS on any unimplemented opcode — can't gracefully degrade
|
||||||
|
- excellent pluggable I/O: abstract ZUI, ZScreen, ZInputStream, ZOutputStream
|
||||||
|
- quetzal save/restore infrastructure exists but opcodes are stubbed
|
||||||
|
- pure python, zero dependencies — perfect for embedding
|
||||||
|
- blocking run loop, not async-compatible
|
||||||
|
- clean API: ZMachine(story_bytes, ui=custom_ui).run()
|
||||||
|
- python 3 port was a "blind fix" — functional but fragile
|
||||||
|
- verdict: TOO INCOMPLETE. would need 2-4 weeks of opcode implementation
|
||||||
|
before any real game runs. the I/O architecture is excellent reference
|
||||||
|
material but the CPU is not production-ready.
|
||||||
|
|
||||||
|
viola (python, pygame dependency):
|
||||||
|
- versions 1-8, comprehensive modern standard 1.1 compliance
|
||||||
|
- 120+ opcodes implemented — ALL documented z-spec instructions have
|
||||||
|
implementations
|
||||||
|
- clean stream architecture: 5 output streams, each individually addressable
|
||||||
|
- full quetzal 1.4 save/restore support with undo
|
||||||
|
- active maintenance (2024 copyright, author engaged)
|
||||||
|
- dependencies: pygame (for rendering/input), numpy (for image processing)
|
||||||
|
- bundled ififf submodule for blorb/quetzal/IFF handling
|
||||||
|
- blocking main loop (execloop), module-level global state
|
||||||
|
- no tests whatsoever
|
||||||
|
- clean module separation: zcode/ (VM core) vs vio/ (I/O layer)
|
||||||
|
- ~8,500 lines in zcode/ + ~2,000 in vio/
|
||||||
|
- verdict: MOST COMPLETE python z-machine available. pygame dependency is
|
||||||
|
only in vio/ — replace that module with a text-callback I/O backend and
|
||||||
|
the core VM is usable. strongest candidate for embedding.
|
||||||
|
|
||||||
|
mojozork (C, educational implementation):
|
||||||
|
- version 3 only (rejects other versions with error)
|
||||||
|
- ~45 opcodes, complete for v3 (covers ~90% of infocom catalog)
|
||||||
|
- excellent pluggable I/O via function pointer callbacks
|
||||||
|
- has runInstruction() for single-step execution — async-friendly by design
|
||||||
|
- multiple frontends: stdio, libretro, SDL3, and a TELNET MULTIPLAYER SERVER
|
||||||
|
- MultiZork uses sqlite for multiplayer state persistence
|
||||||
|
- zlib license (very permissive)
|
||||||
|
- author: ryan c. gordon (well-known open source dev)
|
||||||
|
- custom save format (not quetzal)
|
||||||
|
- C only — would need ctypes/CFFI wrapper or subprocess
|
||||||
|
- verdict: EXCELLENT REFERENCE for architecture patterns. MultiZork shows
|
||||||
|
exactly how to embed a z-machine in a multiplayer telnet server. v3-only
|
||||||
|
is limiting but covers classic infocom games. best studied, potentially
|
||||||
|
wrapped via C extension.
|
||||||
|
|
||||||
|
|
||||||
|
integration options
|
||||||
|
===================
|
||||||
|
|
||||||
|
three paths forward:
|
||||||
|
|
||||||
|
option A: embed viola (python native)
|
||||||
|
- fork or vendor viola's zcode/ module
|
||||||
|
- replace vio/ with our own I/O backend: MudZUI class
|
||||||
|
- wrap the blocking execloop with async bridge (run_in_executor)
|
||||||
|
- state serialization via viola's quetzal support
|
||||||
|
- pros: pure python, full z-machine 1-8 support, comprehensive opcodes,
|
||||||
|
native access to VM state
|
||||||
|
- cons: no tests in upstream, blocking loop requires thread pool,
|
||||||
|
global state needs refactoring for multiple concurrent games
|
||||||
|
|
||||||
|
option B: subprocess dfrozt (quick prototype)
|
||||||
|
- spawn dfrotz in a subprocess, pipe stdin/stdout
|
||||||
|
- asyncio subprocess API handles I/O
|
||||||
|
- save/restore via dfrotz's built-in quetzal support
|
||||||
|
- pros: FASTEST PATH TO WORKING. dfrozt is battle-tested, z-machine 1-8,
|
||||||
|
comprehensive game compatibility
|
||||||
|
- cons: less control over VM state, harder to implement spectators (need to
|
||||||
|
tee output), subprocess management overhead, can't introspect game state
|
||||||
|
for GMCP/rich features
|
||||||
|
|
||||||
|
option C: write our own (long-term ideal)
|
||||||
|
- study mojozork and sussman/zvm architectures
|
||||||
|
- implement enough opcodes for target games (start with v3)
|
||||||
|
- async-native from the start
|
||||||
|
- pros: perfect fit for dreambook philosophy, full control, async-native,
|
||||||
|
clean integration with MUD internals
|
||||||
|
- cons: SIGNIFICANT DETOUR. weeks/months before first game runs. opcode
|
||||||
|
implementation is tedious and error-prone. spec compliance is hard.
|
||||||
|
|
||||||
|
recommendation: START WITH B (dfrozt subprocess), PLAN FOR A (viola embedding).
|
||||||
|
|
||||||
|
rationale: dfrozt gets IF working in hours. players can play zork today. we
|
||||||
|
learn what the integration needs (spectator broadcasting, save management,
|
||||||
|
input routing) with real usage. once the architecture is proven, we can
|
||||||
|
replace the subprocess with embedded viola for better control and state
|
||||||
|
access. option C remains on the table if viola doesn't fit, but we don't
|
||||||
|
invest in a VM until we know what the MUD needs from it.
|
||||||
|
|
||||||
|
|
||||||
|
the "terminal" game object
|
||||||
|
==========================
|
||||||
|
|
||||||
|
a terminal is a room object that hosts an IF session. mechanically:
|
||||||
|
|
||||||
|
- a terminal has a z-machine story file (path or blob reference)
|
||||||
|
- when a player enters the terminal, it pushes IF mode onto their stack
|
||||||
|
- input from the player routes to the z-machine interpreter
|
||||||
|
- output from the z-machine routes to the player (and spectators)
|
||||||
|
- the terminal tracks active sessions (who's playing, current state)
|
||||||
|
|
||||||
|
example from a player's perspective:
|
||||||
|
|
||||||
|
> look
|
||||||
|
you stand in a neon-lit arcade. rows of terminals hum quietly.
|
||||||
|
a large screen on the north wall shows "ZORK - The Great Underground
|
||||||
|
Empire" in green phosphor text.
|
||||||
|
|
||||||
|
jared is standing here, absorbed in a terminal.
|
||||||
|
sarah is watching jared's game on a nearby screen.
|
||||||
|
|
||||||
|
> use terminal
|
||||||
|
you sit down at a terminal. the screen flickers to life.
|
||||||
|
|
||||||
|
[IF mode engaged. type 'quit' to exit the game.]
|
||||||
|
|
||||||
|
ZORK I: The Great Underground Empire
|
||||||
|
Copyright (c) 1981, 1982, 1983 Infocom, Inc. All rights reserved.
|
||||||
|
ZORK is a registered trademark of Infocom, Inc.
|
||||||
|
Revision 88 / Serial number 840726
|
||||||
|
|
||||||
|
West of House
|
||||||
|
You are standing in an open field west of a white house, with a boarded
|
||||||
|
front door.
|
||||||
|
There is a small mailbox here.
|
||||||
|
|
||||||
|
>
|
||||||
|
|
||||||
|
at this point the player's input goes to the z-machine. the terminal object
|
||||||
|
manages the interpreter subprocess/instance. other players in the room see
|
||||||
|
"jared is playing zork" in their ambient messages.
|
||||||
|
|
||||||
|
|
||||||
|
spectator broadcasting
|
||||||
|
======================
|
||||||
|
|
||||||
|
key feature from mojozork's MultiZork: others can watch your game.
|
||||||
|
|
||||||
|
in the MUD: if sarah is in the same room as jared, she can see his game
|
||||||
|
output in near-realtime. not keystroke-by-keystroke, but command and response.
|
||||||
|
|
||||||
|
implementation:
|
||||||
|
|
||||||
|
- terminal object has a set of active_spectators (player refs)
|
||||||
|
- when the z-machine sends output to the active player, terminal mirrors
|
||||||
|
that output to all spectators in the same room
|
||||||
|
- spectators see: "[jared's game] > n\nYou are in a maze of twisty little
|
||||||
|
passages, all alike."
|
||||||
|
- spectators can 'watch jared' to start spectating, 'stop watching' to stop
|
||||||
|
- spectators are in normal mode, not IF mode. they see their own world
|
||||||
|
events too.
|
||||||
|
|
||||||
|
this creates communal play. someone stuck on a puzzle can have friends watch
|
||||||
|
and suggest commands. speedruns become spectator events. IF becomes social.
|
||||||
|
|
||||||
|
technical: output from the z-machine goes to a broadcast function:
|
||||||
|
|
||||||
|
def send_to_player_and_spectators(player, output):
|
||||||
|
player.send(output)
|
||||||
|
terminal = player.current_terminal
|
||||||
|
for spectator in terminal.spectators:
|
||||||
|
if spectator.location == terminal.location:
|
||||||
|
spectator.send(f"[{player.name}'s game] {output}")
|
||||||
|
|
||||||
|
spectators don't share game state, just output. they can't send commands to
|
||||||
|
someone else's game.
|
||||||
|
|
||||||
|
|
||||||
|
viola embedding details
|
||||||
|
========================
|
||||||
|
|
||||||
|
if we go with option A, here's how viola integration would work:
|
||||||
|
|
||||||
|
replace vio/ with MudZUI:
|
||||||
|
|
||||||
|
class MudZUI:
|
||||||
|
"""viola I/O backend that routes to MUD player sessions"""
|
||||||
|
|
||||||
|
def __init__(self, player, terminal):
|
||||||
|
self.player = player
|
||||||
|
self.terminal = terminal
|
||||||
|
self.output_buffer = []
|
||||||
|
|
||||||
|
def write(self, text):
|
||||||
|
"""called by VM to send output"""
|
||||||
|
self.output_buffer.append(text)
|
||||||
|
|
||||||
|
def flush(self):
|
||||||
|
"""send buffered output to player and spectators"""
|
||||||
|
output = ''.join(self.output_buffer)
|
||||||
|
self.terminal.broadcast_output(self.player, output)
|
||||||
|
self.output_buffer.clear()
|
||||||
|
|
||||||
|
def read_line(self):
|
||||||
|
"""called by VM to get player input"""
|
||||||
|
# return input from player's IF mode input queue
|
||||||
|
return self.player.if_input_queue.get()
|
||||||
|
|
||||||
|
def read_char(self):
|
||||||
|
"""single-key input for timed events"""
|
||||||
|
# viola supports this, we'd need to too
|
||||||
|
return self.player.if_input_queue.get_char()
|
||||||
|
|
||||||
|
wrap the blocking VM loop:
|
||||||
|
|
||||||
|
async def run_if_game(player, terminal, story_path):
|
||||||
|
"""run z-machine in thread pool, bridge to async"""
|
||||||
|
loop = asyncio.get_event_loop()
|
||||||
|
|
||||||
|
# load story file
|
||||||
|
with open(story_path, 'rb') as f:
|
||||||
|
story_data = f.read()
|
||||||
|
|
||||||
|
# create VM with our I/O backend
|
||||||
|
zui = MudZUI(player, terminal)
|
||||||
|
zmachine = ZMachine(story_data, zui)
|
||||||
|
|
||||||
|
# run in executor (thread pool) since VM loop is blocking
|
||||||
|
def run_vm():
|
||||||
|
try:
|
||||||
|
zmachine.run()
|
||||||
|
except GameOver:
|
||||||
|
pass # normal exit
|
||||||
|
|
||||||
|
await loop.run_in_executor(None, run_vm)
|
||||||
|
|
||||||
|
state management:
|
||||||
|
|
||||||
|
- viola has full quetzal save/restore
|
||||||
|
- when player types 'save', VM writes quetzal IFF to bytes
|
||||||
|
- we store that blob in sqlite: game_saves table (player_id, terminal_id,
|
||||||
|
save_slot, quetzal_data, timestamp)
|
||||||
|
- when player types 'restore', we fetch the blob and pass to VM
|
||||||
|
- viola handles all the z-machine state serialization
|
||||||
|
|
||||||
|
multiple concurrent games:
|
||||||
|
|
||||||
|
- viola uses module-level global state in some places
|
||||||
|
- need to audit zcode/ and refactor globals into instance state
|
||||||
|
- each terminal object owns a ZMachine instance
|
||||||
|
- multiple players can play the same game in different terminals
|
||||||
|
- each gets their own VM instance with independent state
|
||||||
|
|
||||||
|
this is work, but manageable. viola's clean separation of zcode/ (VM) from
|
||||||
|
vio/ (I/O) makes the I/O replacement straightforward. the async bridge is a
|
||||||
|
standard pattern. the global state refactor is the biggest risk.
|
||||||
|
|
||||||
|
|
||||||
|
dfrozt subprocess details
|
||||||
|
==========================
|
||||||
|
|
||||||
|
if we go with option B for prototyping, here's the implementation:
|
||||||
|
|
||||||
|
spawn dfrozt:
|
||||||
|
|
||||||
|
async def run_if_game_subprocess(player, terminal, story_path):
|
||||||
|
"""run dfrotz as subprocess, pipe I/O"""
|
||||||
|
process = await asyncio.create_subprocess_exec(
|
||||||
|
'dfrotz',
|
||||||
|
'-p', # plain output, no formatting
|
||||||
|
'-m', # disable MORE prompts
|
||||||
|
story_path,
|
||||||
|
stdin=asyncio.subprocess.PIPE,
|
||||||
|
stdout=asyncio.subprocess.PIPE,
|
||||||
|
stderr=asyncio.subprocess.PIPE,
|
||||||
|
)
|
||||||
|
|
||||||
|
# bridge player input to subprocess stdin
|
||||||
|
async def input_loop():
|
||||||
|
while process.returncode is None:
|
||||||
|
command = await player.if_input_queue.get()
|
||||||
|
process.stdin.write(f"{command}\n".encode())
|
||||||
|
await process.stdin.drain()
|
||||||
|
|
||||||
|
# bridge subprocess stdout to player and spectators
|
||||||
|
async def output_loop():
|
||||||
|
while process.returncode is None:
|
||||||
|
line = await process.stdout.readline()
|
||||||
|
if not line:
|
||||||
|
break
|
||||||
|
text = line.decode('utf-8', errors='replace')
|
||||||
|
terminal.broadcast_output(player, text)
|
||||||
|
|
||||||
|
# run both loops concurrently
|
||||||
|
await asyncio.gather(input_loop(), output_loop())
|
||||||
|
await process.wait()
|
||||||
|
|
||||||
|
save/restore:
|
||||||
|
|
||||||
|
- dfrotz handles saves via its own commands (SAVE, RESTORE)
|
||||||
|
- save files go to filesystem
|
||||||
|
- we could intercept SAVE/RESTORE commands, manage files in a
|
||||||
|
player-specific directory, copy to sqlite for persistence
|
||||||
|
- or let dfrozt write to /tmp/ and just copy the files on exit
|
||||||
|
|
||||||
|
spectator output:
|
||||||
|
|
||||||
|
- subprocess output goes to all (player + spectators) via broadcast
|
||||||
|
- tee is automatic — we control the output loop
|
||||||
|
|
||||||
|
exit/cleanup:
|
||||||
|
|
||||||
|
- player types 'quit' or EOF — we send QUIT to process, wait for exit
|
||||||
|
- process crash — we catch it, log error, tell player "game crashed"
|
||||||
|
- terminal cleanup — kill subprocess if still running
|
||||||
|
|
||||||
|
pros: simple, robust, dfrozt is battle-tested.
|
||||||
|
cons: less control. can't introspect VM state for GMCP data. can't implement
|
||||||
|
custom opcodes. subprocess overhead (but negligible for text games).
|
||||||
|
|
||||||
|
|
||||||
|
mojozork patterns to study
|
||||||
|
===========================
|
||||||
|
|
||||||
|
MultiZork is mojozork's multiplayer telnet server. it shows:
|
||||||
|
|
||||||
|
1. session management: each player gets a separate VM instance, but the game
|
||||||
|
world is shared. players see each other's actions. this is DIFFERENT from
|
||||||
|
our model (each terminal is isolated) but the architecture is instructive.
|
||||||
|
|
||||||
|
2. sqlite persistence: MultiZork stores game state in sqlite. saves are just
|
||||||
|
snapshots of the VM memory. we'd do the same with quetzal blobs.
|
||||||
|
|
||||||
|
3. command broadcasting: when one player types something, MultiZork echoes it
|
||||||
|
to others in the same location. same as our spectator model.
|
||||||
|
|
||||||
|
4. pluggable I/O: mojozork's I/O is function pointers:
|
||||||
|
|
||||||
|
typedef struct {
|
||||||
|
void (*print)(const char *text);
|
||||||
|
char *(*read_line)(void);
|
||||||
|
void (*save_state)(const uint8_t *data, size_t len);
|
||||||
|
uint8_t *(*restore_state)(size_t *len);
|
||||||
|
} IOCallbacks;
|
||||||
|
|
||||||
|
clean, simple. viola's ZUI is the python equivalent.
|
||||||
|
|
||||||
|
5. tick-based execution: MultiZork runs the VM in a loop, checking for input
|
||||||
|
each tick. same as our game loop. z-machine opcodes are fast — dozens
|
||||||
|
execute per tick.
|
||||||
|
|
||||||
|
MultiZork source is excellent reference for "how to embed IF in a multiplayer
|
||||||
|
server." if we write our own VM (option C), MultiZork is the template.
|
||||||
|
|
||||||
|
|
||||||
|
save/restore and persistence
|
||||||
|
=============================
|
||||||
|
|
||||||
|
z-machine has built-in save/restore opcodes. quetzal is the standard format.
|
||||||
|
|
||||||
|
in our MUD:
|
||||||
|
|
||||||
|
- player types SAVE in the IF game
|
||||||
|
- VM serializes its state to quetzal IFF (binary format)
|
||||||
|
- we capture that blob and write to sqlite:
|
||||||
|
|
||||||
|
CREATE TABLE if_saves (
|
||||||
|
id INTEGER PRIMARY KEY,
|
||||||
|
player_id INTEGER NOT NULL,
|
||||||
|
terminal_id INTEGER NOT NULL,
|
||||||
|
slot INTEGER DEFAULT 0,
|
||||||
|
quetzal_data BLOB NOT NULL,
|
||||||
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
UNIQUE(player_id, terminal_id, slot)
|
||||||
|
);
|
||||||
|
|
||||||
|
- player types RESTORE, we fetch the blob and pass back to VM
|
||||||
|
|
||||||
|
autosave:
|
||||||
|
|
||||||
|
- when player exits IF mode (QUIT command or connection drop), we could
|
||||||
|
autosave their state
|
||||||
|
- on reconnect/re-enter, offer to resume from last autosave
|
||||||
|
|
||||||
|
multiple save slots:
|
||||||
|
|
||||||
|
- z-machine games usually support named saves ("SAVE mysave")
|
||||||
|
- we could parse the filename and use it as the slot identifier
|
||||||
|
- or just offer slots 1-10 and map them to quetzal blobs
|
||||||
|
|
||||||
|
cross-session saves:
|
||||||
|
|
||||||
|
- player saves in session A, restores in session B (different terminal)
|
||||||
|
- works fine if both terminals host the same story file
|
||||||
|
- could enable "speedrun mode" — reset to a specific save and time from
|
||||||
|
there
|
||||||
|
|
||||||
|
undo:
|
||||||
|
|
||||||
|
- z-machine has UNDO opcode
|
||||||
|
- viola implements it via a stack of quetzal snapshots
|
||||||
|
- we could surface undo as a MUD command outside the game (safety net if
|
||||||
|
player makes a mistake)
|
||||||
|
|
||||||
|
|
||||||
|
if mode on the mode stack
|
||||||
|
==========================
|
||||||
|
|
||||||
|
when player enters a terminal, push IF mode:
|
||||||
|
|
||||||
|
player.mode_stack.push(Mode.IF)
|
||||||
|
|
||||||
|
IF mode behavior:
|
||||||
|
|
||||||
|
- all input goes to if_input_queue, not command dispatcher
|
||||||
|
- world events (chat, movement, ambient) are buffered, not displayed
|
||||||
|
- combat can't start (you're isolated)
|
||||||
|
- on exit, pop mode, show summary of buffered events
|
||||||
|
|
||||||
|
commands available in IF mode:
|
||||||
|
|
||||||
|
- anything the z-machine game accepts (depends on the game)
|
||||||
|
- special MUD commands prefixed with / or !:
|
||||||
|
- /quit — exit the game, pop IF mode
|
||||||
|
- /save [slot] — trigger VM save
|
||||||
|
- /restore [slot] — trigger VM restore
|
||||||
|
- /undo — trigger VM undo
|
||||||
|
- /help — show IF mode help
|
||||||
|
|
||||||
|
input routing:
|
||||||
|
|
||||||
|
before IF mode: player types "north" → command dispatcher → cmd_move
|
||||||
|
during IF mode: player types "north" → if_input_queue → z-machine
|
||||||
|
|
||||||
|
the mode stack handles this cleanly. IF mode is just another mode, like combat
|
||||||
|
or editor.
|
||||||
|
|
||||||
|
|
||||||
|
terminal object implementation sketch
|
||||||
|
======================================
|
||||||
|
|
||||||
|
rough structure:
|
||||||
|
|
||||||
|
class IFTerminal:
|
||||||
|
"""a room object that hosts z-machine games"""
|
||||||
|
|
||||||
|
def __init__(self, story_path, name, description):
|
||||||
|
self.story_path = story_path # path to .z3/.z5/.z8 file
|
||||||
|
self.name = name
|
||||||
|
self.description = description
|
||||||
|
self.active_sessions = {} # player -> IFSession
|
||||||
|
self.spectators = set() # players watching
|
||||||
|
|
||||||
|
async def enter(self, player):
|
||||||
|
"""player sits at terminal, start IF session"""
|
||||||
|
if player in self.active_sessions:
|
||||||
|
player.send("You're already playing this game.")
|
||||||
|
return
|
||||||
|
|
||||||
|
# push IF mode
|
||||||
|
player.mode_stack.push(Mode.IF)
|
||||||
|
|
||||||
|
# create session (VM instance or subprocess)
|
||||||
|
session = await self.create_session(player)
|
||||||
|
self.active_sessions[player] = session
|
||||||
|
|
||||||
|
# announce to room
|
||||||
|
self.broadcast_except(player, f"{player.name} sits down at {self.name}.")
|
||||||
|
|
||||||
|
async def create_session(self, player):
|
||||||
|
"""spawn VM or subprocess for this player"""
|
||||||
|
# option A: return ZMachineSession(player, self)
|
||||||
|
# option B: return DfrotzSession(player, self)
|
||||||
|
pass
|
||||||
|
|
||||||
|
def broadcast_output(self, player, output):
|
||||||
|
"""send game output to player and spectators"""
|
||||||
|
player.send(output)
|
||||||
|
for spectator in self.spectators:
|
||||||
|
if spectator.location == self.location:
|
||||||
|
spectator.send(f"[{player.name}'s game]\n{output}")
|
||||||
|
|
||||||
|
async def exit(self, player):
|
||||||
|
"""player leaves terminal, cleanup session"""
|
||||||
|
session = self.active_sessions.pop(player, None)
|
||||||
|
if session:
|
||||||
|
await session.cleanup()
|
||||||
|
player.mode_stack.pop() # exit IF mode
|
||||||
|
self.broadcast_except(player, f"{player.name} stands up from {self.name}.")
|
||||||
|
|
||||||
|
terminals are room objects. they could be defined in world data:
|
||||||
|
|
||||||
|
- rooms:
|
||||||
|
- id: arcade
|
||||||
|
description: "a dimly lit arcade. terminals line the walls."
|
||||||
|
objects:
|
||||||
|
- type: if_terminal
|
||||||
|
name: "zork terminal"
|
||||||
|
story: "games/zork1.z3"
|
||||||
|
description: "an old CRT displaying 'ZORK I' in green text"
|
||||||
|
|
||||||
|
or created by players (builder commands):
|
||||||
|
|
||||||
|
> create terminal "hitchhiker terminal" games/hhgg.z5
|
||||||
|
You create a new IF terminal.
|
||||||
|
|
||||||
|
> describe terminal "A sleek modern terminal. The screen reads:
|
||||||
|
'The Hitchhiker's Guide to the Galaxy - Don't Panic'"
|
||||||
|
|
||||||
|
|
||||||
|
phased implementation plan
|
||||||
|
===========================
|
||||||
|
|
||||||
|
phase 1: dfrozt subprocess prototype
|
||||||
|
- define IFTerminal class
|
||||||
|
- implement IF mode on mode stack
|
||||||
|
- spawn dfrozt subprocess for a hardcoded game (zork1.z3)
|
||||||
|
- route player input to subprocess stdin
|
||||||
|
- route subprocess stdout to player
|
||||||
|
- /quit command to exit
|
||||||
|
goal: player can sit at a terminal and play zork. no saves, no spectators.
|
||||||
|
estimate: 1-2 days.
|
||||||
|
|
||||||
|
phase 2: spectator broadcasting
|
||||||
|
- terminal tracks spectators (players in same room)
|
||||||
|
- tee game output to spectators with prefix
|
||||||
|
- 'watch [player]' and 'stop watching' commands
|
||||||
|
goal: others can watch your game in realtime.
|
||||||
|
estimate: half day.
|
||||||
|
|
||||||
|
phase 3: save/restore via filesystem
|
||||||
|
- intercept SAVE/RESTORE commands
|
||||||
|
- manage save files in player-specific directory
|
||||||
|
- copy to sqlite on exit for persistence
|
||||||
|
goal: players can save progress and resume later.
|
||||||
|
estimate: 1 day.
|
||||||
|
|
||||||
|
phase 4: terminal as room object
|
||||||
|
- define terminals in world data (YAML)
|
||||||
|
- load at startup, spawn on player enter
|
||||||
|
- multiple terminals, different games
|
||||||
|
goal: arcade room with multiple games, each on its own terminal.
|
||||||
|
estimate: 1 day.
|
||||||
|
|
||||||
|
phase 5: evaluate viola embedding
|
||||||
|
- fork/vendor viola's zcode/
|
||||||
|
- implement MudZUI (I/O backend)
|
||||||
|
- wrap VM loop with run_in_executor
|
||||||
|
- test with zork1, compare to dfrozt
|
||||||
|
goal: decide if viola is better than subprocess.
|
||||||
|
estimate: 2-3 days.
|
||||||
|
|
||||||
|
phase 6: switch to viola if proven
|
||||||
|
- replace dfrozt sessions with viola sessions
|
||||||
|
- refactor terminal to use embedded VM
|
||||||
|
- quetzal saves to sqlite (direct, no filesystem)
|
||||||
|
- expose VM state for GMCP (current room, inventory, score)
|
||||||
|
goal: full control over VM, native python integration.
|
||||||
|
estimate: 2-3 days (assuming viola works).
|
||||||
|
|
||||||
|
phase 7: builder tools
|
||||||
|
- 'create terminal' command for admins/builders
|
||||||
|
- upload .z3/.z5/.z8 files to server
|
||||||
|
- terminal object editor (change description, story file)
|
||||||
|
goal: builders can add new IF games without code changes.
|
||||||
|
estimate: 2 days.
|
||||||
|
|
||||||
|
total estimate: 10-15 days for full IF integration with all features.
|
||||||
|
|
||||||
|
if we stop after phase 4, we have working IF-in-MUD with spectators and saves.
|
||||||
|
that's enough to prove the concept. phases 5-7 are refinement and tooling.
|
||||||
|
|
||||||
|
|
||||||
|
what we learn from this
|
||||||
|
========================
|
||||||
|
|
||||||
|
IF integration exercises several MUD systems:
|
||||||
|
|
||||||
|
- mode stack: IF mode is the third mode (after normal and editor). proves
|
||||||
|
the stack abstraction works for diverse isolation needs.
|
||||||
|
|
||||||
|
- session management: each terminal session is a persistent object (VM or
|
||||||
|
subprocess) tied to a player. different from stateless command dispatch.
|
||||||
|
|
||||||
|
- room-local broadcasting: spectator output is the first use of "broadcast
|
||||||
|
to players in this room." will be useful for ambient messages, weather,
|
||||||
|
room events later.
|
||||||
|
|
||||||
|
- binary blob persistence: quetzal saves are blobs in sqlite. same pattern
|
||||||
|
will apply to uploaded files, cached terrain chunks, whatever.
|
||||||
|
|
||||||
|
- game objects with behavior: terminals are room objects that do things
|
||||||
|
(spawn VMs, route I/O). pattern extends to doors, NPCs, triggers.
|
||||||
|
|
||||||
|
getting IF right means the architecture can handle editor mode, crafting
|
||||||
|
mini-games, puzzles, dialogue trees, anything with isolated state and
|
||||||
|
custom input handling.
|
||||||
|
|
||||||
|
|
||||||
|
risks and unknowns
|
||||||
|
==================
|
||||||
|
|
||||||
|
viola's lack of tests:
|
||||||
|
we'd be trusting ~10k lines of untested code. bugs in opcode implementation
|
||||||
|
would surface as "game doesn't work" with no clear cause. mitigation: test
|
||||||
|
with a suite of known games (zork, hhgg, curses, anchorhead) before
|
||||||
|
committing. if bugs emerge, consider dfrozt as the long-term solution.
|
||||||
|
|
||||||
|
async bridge overhead:
|
||||||
|
viola's blocking loop in a thread pool adds latency. run_in_executor is
|
||||||
|
designed for this but it's not zero-cost. mitigation: measure latency in
|
||||||
|
prototype. if it's perceptible (>100ms), consider writing an async-native
|
||||||
|
VM or sticking with subprocess.
|
||||||
|
|
||||||
|
global state in viola:
|
||||||
|
haven't audited zcode/ for globals yet. if it's pervasive, the refactor to
|
||||||
|
instance state could be large. mitigation: audit before committing to viola.
|
||||||
|
if globals are deeply baked in, subprocess is safer.
|
||||||
|
|
||||||
|
z-machine version support:
|
||||||
|
viola claims 1-8, mojozork does 3 only. most infocom classics are v3 or v5.
|
||||||
|
modern IF (inform 7) compiles to v8. if we only care about classics, v3 is
|
||||||
|
enough. if we want modern games, need v8. mitigation: decide target game
|
||||||
|
set before choosing VM.
|
||||||
|
|
||||||
|
spectator bandwidth:
|
||||||
|
if 10 players are watching one game, every line gets sent 11 times (player
|
||||||
|
+ 10 spectators). that's fine for text but worth monitoring. mitigation:
|
||||||
|
spectator output could be rate-limited or summarized (show every 5 lines,
|
||||||
|
not every line).
|
||||||
|
|
||||||
|
save file size:
|
||||||
|
quetzal saves are typically 20-100KB. if player saves frequently, sqlite
|
||||||
|
grows. mitigation: limit saves per player per game (10 slots?), auto-prune
|
||||||
|
old saves, or compress quetzal blobs (they're IFF, should compress well).
|
||||||
|
|
||||||
|
|
||||||
|
closing
|
||||||
|
=======
|
||||||
|
|
||||||
|
IF-in-MUD is not a detour. it's a proof that the architecture supports
|
||||||
|
isolated, stateful, player-driven content. the mode stack, room objects,
|
||||||
|
save/restore, multiplayer isolation — all tested in a concrete use case.
|
||||||
|
|
||||||
|
start with dfrozt subprocess (phase 1-4). prove the concept. if it works,
|
||||||
|
consider viola for tighter integration. if viola is risky, subprocess is
|
||||||
|
fine long-term. mojozork shows that C via subprocess is viable.
|
||||||
|
|
||||||
|
the dream: player-created IF games, built in the editor, tested in-world,
|
||||||
|
published for others. that requires a DSL and builder tools. but the
|
||||||
|
foundation is terminals + mode stack + save/restore. get that right and the
|
||||||
|
dream is achievable.
|
||||||
|
|
||||||
|
|
||||||
|
code
|
||||||
|
----
|
||||||
|
|
||||||
|
this document docs/how/if-integration.txt
|
||||||
|
dreambook DREAMBOOK.md
|
||||||
|
architecture docs/how/architecture-plan.txt
|
||||||
258
docs/how/if-journey.rst
Normal file
258
docs/how/if-journey.rst
Normal file
|
|
@ -0,0 +1,258 @@
|
||||||
|
if journey — from arcade terminal to moldable worlds
|
||||||
|
=====================================================
|
||||||
|
|
||||||
|
This doc tracks the IF (interactive fiction) integration effort for the MUD engine. It's a living document — updated as research progresses and decisions are made. For detailed technical analysis, see the two interpreter audits (``viola-embedding-audit.rst`` and ``zvm-embedding-audit.rst``) and the original integration notes (``if-integration.txt``).
|
||||||
|
|
||||||
|
the vision — five levels of integration
|
||||||
|
----------------------------------------
|
||||||
|
|
||||||
|
Five levels emerged from design discussion. They represent a spectrum of how deeply IF worlds integrate with the MUD world, from simplest to most ambitious.
|
||||||
|
|
||||||
|
Level 1 — terminal mode
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Subprocess (dfrotz), text in/out, spectators see text scrolling on a screen. Player sits at an arcade terminal, plays Zork. Others in the room see the output. The IF world is opaque — a black box.
|
||||||
|
|
||||||
|
Works today with dfrotz. Proven technology.
|
||||||
|
|
||||||
|
Level 2 — inspectable world
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Embedded interpreter. MUD can READ z-machine state. Know what room the player is in, describe it to spectators, track progress. Spectators don't just see text — they know "jared is in the Trophy Room." Read-only bridge between MUD and IF world.
|
||||||
|
|
||||||
|
Level 3 — moldable world
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Embedded interpreter. MUD can READ AND WRITE z-machine state. Inject items into IF game objects. Put a note in the Zork mailbox. The z-machine object tree (parent/child/sibling with attributes and properties) becomes accessible from MUD code. Two-way bridge. Game world is modifiable from outside.
|
||||||
|
|
||||||
|
Level 4 — shared world
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Multiple MUD players mapped into the same z-machine instance. Each has their own player object. Independent inventory and position. MojoZork's MultiZork proved this works for V3 games. The IF world is a zone in the MUD that multiple players inhabit simultaneously.
|
||||||
|
|
||||||
|
Level 5 — transcendent world
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Z-machine object tree and MUD entity model are unified. An item in Zork IS a MUD item. Pick it up in the IF world, carry it back to the MUD. The mailbox exists at coordinates in both worlds simultaneously. Full bidirectional entity bridge.
|
||||||
|
|
||||||
|
Note: Level 1 uses subprocess. Levels 2-5 require being inside the interpreter. Once you're at level 2, the jump to 3 is small (reading memory vs writing it). Level 4 is the MojoZork leap. Level 5 is the dream.
|
||||||
|
|
||||||
|
what we know — audit findings
|
||||||
|
------------------------------
|
||||||
|
|
||||||
|
Two Python z-machine interpreters were audited in detail. Don't repeat everything here — see the audit docs for details. Focus on decision-relevant facts.
|
||||||
|
|
||||||
|
viola (DFillmore/viola)
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Can run games today. All V1-V5 opcodes implemented, partial V6-V8. But global state is deeply tangled (13/18 modules with mutable globals). Multiple instances in one process: not feasible without major refactor.
|
||||||
|
|
||||||
|
pygame dependency is cleanly separable (adapter pattern, unidirectional). ``sys.exit()`` in error paths (8+ locations) — needs patching for server use. Memory leaks (5+ unbounded growth patterns) — fixable with cleanup hooks.
|
||||||
|
|
||||||
|
Object tree accessors exist and are wired through all opcodes. Full quetzal save/restore working.
|
||||||
|
|
||||||
|
See: ``docs/how/viola-embedding-audit.rst``
|
||||||
|
|
||||||
|
zvm (sussman/zvm)
|
||||||
|
~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Cannot run any game. ~46/108 opcodes implemented, including ZERO input opcodes. But instance-based state (mostly clean, two minor global leaks fixable in ~20 lines). Multiple instances in one process: structurally possible.
|
||||||
|
|
||||||
|
IO abstraction is excellent — purpose-built for embedding (abstract ZUI with stubs). Proper exception hierarchy, zero ``sys.exit()`` calls. No memory leaks, clean bounded state.
|
||||||
|
|
||||||
|
Object tree parser has complete read API, mostly complete write API. BUT: many object opcodes not wired up at CPU level. Save parser works, save writer is stubbed.
|
||||||
|
|
||||||
|
See: ``docs/how/zvm-embedding-audit.rst``
|
||||||
|
|
||||||
|
The verdict from the audits: "zvm has the architecture you'd want. viola has the implementation you'd need."
|
||||||
|
|
||||||
|
MojoZork (C, Ryan C. Gordon)
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
V3 only but has working multiplayer telnet server (MultiZork). Proves multiplayer z-machine works (up to 4 players, separate inventories). ``runInstruction()`` for single-step execution — the async pattern we want. Excellent reference architecture, not directly usable (C only).
|
||||||
|
|
||||||
|
the hybrid path — option D
|
||||||
|
---------------------------
|
||||||
|
|
||||||
|
This emerged from comparing the audits side by side. Use zvm's architecture (clean IO abstraction, instance-based state, exception model) as the skeleton. Port viola's working opcode implementations into it.
|
||||||
|
|
||||||
|
Why this is attractive:
|
||||||
|
|
||||||
|
- zvm's opcodes take ``self`` (ZCpu) and access state through instance attributes
|
||||||
|
- viola's opcodes use module-level globals (``zcode.game.PC``, ``zcode.memory.data``, etc)
|
||||||
|
- porting means translating global lookups to instance attribute lookups
|
||||||
|
- that's mechanical, not creative — could port 5-10 opcodes per hour once the pattern is established
|
||||||
|
- gets the clean design with the working code
|
||||||
|
|
||||||
|
Why this could be harder than it sounds:
|
||||||
|
|
||||||
|
- viola and zvm may represent z-machine internals differently
|
||||||
|
- memory layout assumptions, stack frame format, string encoding details
|
||||||
|
- porting opcodes may require porting the data structures they operate on
|
||||||
|
- need to verify each ported opcode against the z-machine spec, not just translate
|
||||||
|
|
||||||
|
Estimated effort: medium. Less than finishing zvm from scratch, less than refactoring viola's globals. But not trivial.
|
||||||
|
|
||||||
|
the object tree — key to moldable worlds
|
||||||
|
-----------------------------------------
|
||||||
|
|
||||||
|
This is what makes levels 3-5 possible. The z-machine has an object tree — every game entity is a node with parent/child/sibling pointers, attributes (boolean flags), and properties (variable-length data).
|
||||||
|
|
||||||
|
What the object tree gives us:
|
||||||
|
|
||||||
|
- Read what room the player is in (player object's parent)
|
||||||
|
- Read container contents (children of the container object)
|
||||||
|
- Inject items (create objects, parent them to containers)
|
||||||
|
- Modify game state (set/clear attributes, change properties)
|
||||||
|
- Query the dictionary (what words the parser recognizes)
|
||||||
|
|
||||||
|
Both interpreters have object tree parsers:
|
||||||
|
|
||||||
|
- viola: complete read + write, all opcodes wired, working
|
||||||
|
- zvm: complete read, mostly complete write (missing ``set_attr``/``clear_attr``), many opcodes unwired at CPU level, bug in ``insert_object``
|
||||||
|
|
||||||
|
The dictionary problem (level 3+)
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Injecting an object into the mailbox works for "look in mailbox" — the game iterates children and prints short names. But "take [new item]" fails unless the word exists in the game's dictionary. The dictionary is baked into the story file.
|
||||||
|
|
||||||
|
Options:
|
||||||
|
|
||||||
|
- Add words to dictionary at runtime (memory surgery — relocating/expanding the dictionary)
|
||||||
|
- Intercept input before the parser and handle custom items at the MUD layer
|
||||||
|
- Use existing dictionary words for injected items ("note", "scroll", "key" are common)
|
||||||
|
- Hybrid: intercept unrecognized words, check if they match MUD-injected items, handle outside z-machine
|
||||||
|
|
||||||
|
This is a level 3-5 problem. Not a blocker for levels 1-2.
|
||||||
|
|
||||||
|
games we care about
|
||||||
|
-------------------
|
||||||
|
|
||||||
|
The games that motivated this work:
|
||||||
|
|
||||||
|
The Wizard Sniffer (Buster Hudson, 2017)
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
You play a pig who sniffs out wizards. IFComp winner, XYZZY winner. Screwball comedy. The pig/wizard game that started this. Z-machine format. Would need V5 support.
|
||||||
|
|
||||||
|
Lost Pig (Admiral Jota, 2007)
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Grunk the orc chases an escaped pig. IFComp winner, 4 XYZZY awards. Famous for its responsive parser and comedy writing. Z-machine format. V5.
|
||||||
|
|
||||||
|
Zork I, II, III
|
||||||
|
~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
The classics. Everyone should play them. V3.
|
||||||
|
|
||||||
|
Hitchhiker's Guide to the Galaxy
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Funny, frustrating, great spectator game. V3.
|
||||||
|
|
||||||
|
Also: Anchorhead, Photopia, Spider and Web, Shade, Colossal Cave.
|
||||||
|
|
||||||
|
V3 covers the Infocom catalog. V5 covers most modern IF including the pig games. V8 covers big modern Inform 7 games but is lower priority.
|
||||||
|
|
||||||
|
Note: no Python Glulx interpreter exists. Games that target Glulx (some modern Inform 7) are out of scope unless we subprocess to a C interpreter.
|
||||||
|
|
||||||
|
architecture fit
|
||||||
|
----------------
|
||||||
|
|
||||||
|
How this fits the existing MUD architecture. The codebase is ready:
|
||||||
|
|
||||||
|
Mode stack
|
||||||
|
~~~~~~~~~~
|
||||||
|
|
||||||
|
Push "if" mode. All input routes to IF handler, bypassing command dispatch. Same pattern as editor mode. Already proven.
|
||||||
|
|
||||||
|
Input routing
|
||||||
|
~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
``server.py`` shell loop checks ``player.mode``. Add elif for "if" mode, route to ``if_game.handle_input()``. Same as ``editor.handle_input()``.
|
||||||
|
|
||||||
|
Room-local broadcasting
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Not implemented yet. IF integration is the first use case. Terminal object maintains spectator list, broadcasts output to all. Pattern will be reused for ambient messages, weather, room events.
|
||||||
|
|
||||||
|
State storage
|
||||||
|
~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Quetzal saves as blobs in SQLite. Same pattern as other binary persistence.
|
||||||
|
|
||||||
|
Terminal game object
|
||||||
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
A room object (or coordinate-anchored object) that hosts IF sessions. Players "use" it to enter IF mode. Pattern extends to other interactive objects.
|
||||||
|
|
||||||
|
open questions
|
||||||
|
--------------
|
||||||
|
|
||||||
|
Things we haven't figured out yet. Update this as questions get answered.
|
||||||
|
|
||||||
|
1. V3 opcode footprint
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
How many of the ~62 missing zvm opcodes are actually exercised by V3 games? V3 uses a smaller subset. If we target V3 first, the hybrid might need 30 ported, not 62. Research: run a V3 game through viola with opcode tracing, collect the set.
|
||||||
|
|
||||||
|
2. zvm/viola memory layout compatibility
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Do they represent z-machine memory the same way? Both use bytearrays, but header parsing, object table offsets, string encoding — are these compatible enough that porting opcodes is translation, or is it a deeper rewrite?
|
||||||
|
|
||||||
|
3. Async model
|
||||||
|
~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Both interpreters have blocking run loops. Options:
|
||||||
|
|
||||||
|
- ``run_in_executor`` (thread pool) — standard pattern, adds latency
|
||||||
|
- extract ``step()`` and call from async loop — zvm audit says this is ~5 lines
|
||||||
|
- run in separate thread with queue-based IO — more complex but natural
|
||||||
|
|
||||||
|
Which is best for the MUD's tick-based game loop?
|
||||||
|
|
||||||
|
4. Multiplayer z-machine
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
MojoZork does this for V3. What would it take for V5? The V5 object model is larger (65535 objects vs 255). Do V5 games assume single-player in ways that break multiplayer?
|
||||||
|
|
||||||
|
5. Game file licensing
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Infocom games are abandonware but not legally free. Modern IF games (Lost Pig, Wizard Sniffer) are freely distributable. Need to figure out what we can bundle vs what players bring.
|
||||||
|
|
||||||
|
6. Dictionary injection feasibility
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
How hard is it to add words to a z-machine dictionary at runtime? The dictionary is in static memory. Adding words means expanding it, which means relocating it if there's no space. Is this practical?
|
||||||
|
|
||||||
|
what to do next
|
||||||
|
---------------
|
||||||
|
|
||||||
|
Concrete next steps, roughly ordered. Update as items get done.
|
||||||
|
|
||||||
|
- [ ] trace V3 opcode usage: run zork through viola with opcode logging, get the actual set of opcodes a real game uses. this tells us how much porting work the hybrid path actually requires.
|
||||||
|
|
||||||
|
- [ ] compare memory layouts: look at how viola and zvm represent z-machine memory, object tables, string tables. determine if opcode porting is mechanical translation or deeper adaptation.
|
||||||
|
|
||||||
|
- [ ] prototype the hybrid: pick 5-10 common opcodes, port them from viola to zvm's architecture. see how the pattern feels. if it's smooth, the hybrid is viable. if every opcode is a battle, reconsider.
|
||||||
|
|
||||||
|
- [ ] build level 1 prototype: regardless of interpreter choice, implement the terminal object, IF mode, and subprocess dfrotz path. this proves the MUD-side architecture (mode stack, spectators, save/restore) independently of the interpreter question.
|
||||||
|
|
||||||
|
- [ ] study MojoZork's multiplayer model: read the MultiZork source for how it handles multiple players in one z-machine. document the pattern for our eventual level 4.
|
||||||
|
|
||||||
|
- [ ] find the game files: locate freely distributable z-machine story files for the games we care about. Wizard Sniffer, Lost Pig, Zork (if legally available).
|
||||||
|
|
||||||
|
related documents
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
``docs/how/if-integration.txt`` — original research and integration plan (predates audits)
|
||||||
|
|
||||||
|
``docs/how/viola-embedding-audit.rst`` — detailed viola architecture audit
|
||||||
|
|
||||||
|
``docs/how/zvm-embedding-audit.rst`` — detailed zvm architecture audit with comparison
|
||||||
|
|
||||||
|
``docs/how/architecture-plan.txt`` — MUD engine architecture plan
|
||||||
|
|
||||||
|
``DREAMBOOK.md`` — project vision and philosophy
|
||||||
378
docs/how/mojozork-audit.rst
Normal file
378
docs/how/mojozork-audit.rst
Normal file
|
|
@ -0,0 +1,378 @@
|
||||||
|
mojozork audit
|
||||||
|
===============
|
||||||
|
|
||||||
|
technical audit of icculus/mojozork, a C z-machine interpreter with multiplayer
|
||||||
|
telnet server. this sits alongside the viola and zvm audits as reference for our
|
||||||
|
level 4 (shared world) integration plans.
|
||||||
|
|
||||||
|
project facts
|
||||||
|
=============
|
||||||
|
|
||||||
|
author: Ryan C. Gordon (icculus)
|
||||||
|
|
||||||
|
language: C, single-file per target
|
||||||
|
|
||||||
|
files::
|
||||||
|
|
||||||
|
mojozork.c (1,800 lines) — standalone single-player V3 z-machine
|
||||||
|
multizorkd.c (2,784 lines) — multiplayer telnet server
|
||||||
|
mojozork-libretro.c — RetroArch core frontend
|
||||||
|
mojozork-sdl3.c — SDL3 graphical frontend
|
||||||
|
CMakeLists.txt — build system, four optional targets
|
||||||
|
|
||||||
|
version support: V3 only. covers most of the infocom catalog. V4/V5/V6+ opcodes
|
||||||
|
are stubbed but unimplemented.
|
||||||
|
|
||||||
|
includes zork1.dat (92KB story file) — Activision released Zork I-III for free.
|
||||||
|
|
||||||
|
only external dependency: sqlite3 (multizork server only).
|
||||||
|
|
||||||
|
what matters for us
|
||||||
|
===================
|
||||||
|
|
||||||
|
this is NOT a candidate for direct use (it's C, we're Python). what matters:
|
||||||
|
|
||||||
|
1. the multiplayer architecture — MultiZork proves you can have multiple
|
||||||
|
players in one z-machine instance. this is our Level 4 (shared world) from
|
||||||
|
if-journey.rst.
|
||||||
|
|
||||||
|
2. the step execution model — runInstruction() executes one opcode at a time.
|
||||||
|
step_completed flag breaks out of the loop at READ/QUIT. this is the async
|
||||||
|
pattern we want in our interpreter.
|
||||||
|
|
||||||
|
3. the memory virtualization trick — two function pointers
|
||||||
|
(get_virtualized_mem_ptr and remap_objectid) redirect memory access for
|
||||||
|
multiplayer objects. clean separation of concern.
|
||||||
|
|
||||||
|
4. the persistence model — SQLite schema for saving complete z-machine state,
|
||||||
|
player state, transcripts. same pattern we'd use.
|
||||||
|
|
||||||
|
5. the game-specific problem — MultiZork works for Zork 1 only. the multiplayer
|
||||||
|
code is deeply hardcoded to Zork 1's specific object IDs, global variable
|
||||||
|
indices, and memory layout. Ryan himself says it won't even work with other
|
||||||
|
builds of Zork 1.
|
||||||
|
|
||||||
|
core interpreter
|
||||||
|
================
|
||||||
|
|
||||||
|
ZMachineState struct
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
mojozork.c:88-131::
|
||||||
|
|
||||||
|
story: uint8* — game file loaded into memory
|
||||||
|
header: ZHeader — parsed 40-byte z-machine header
|
||||||
|
logical_pc / pc — program counter (offset and pointer)
|
||||||
|
sp / bp — stack and base pointer into uint16 stack[2048]
|
||||||
|
operands[8] + operand_count — current instruction arguments
|
||||||
|
opcodes[256] — function pointer dispatch table
|
||||||
|
step_completed — flag set by READ/QUIT to break execution
|
||||||
|
quit — flag for game exit
|
||||||
|
writestr callback — function pointer for output
|
||||||
|
die callback — function pointer for fatal errors
|
||||||
|
|
||||||
|
execution model
|
||||||
|
---------------
|
||||||
|
|
||||||
|
mojozork.c main loop::
|
||||||
|
|
||||||
|
while (!GState->quit) {
|
||||||
|
runInstruction();
|
||||||
|
}
|
||||||
|
|
||||||
|
runInstruction() (line ~1611-1676):
|
||||||
|
|
||||||
|
1. fetch opcode byte from PC
|
||||||
|
2. parse operand types from encoding bits
|
||||||
|
3. read operand values (large constant, small constant, variable, omitted)
|
||||||
|
4. dispatch: opcodes[opcode_number].fn()
|
||||||
|
5. handler modifies state (PC, stack, memory)
|
||||||
|
|
||||||
|
the step_completed pattern (used by MultiZork)::
|
||||||
|
|
||||||
|
GState->step_completed = 0;
|
||||||
|
while (!GState->step_completed) {
|
||||||
|
runInstruction();
|
||||||
|
}
|
||||||
|
// READ or QUIT set step_completed = 1, breaking out
|
||||||
|
|
||||||
|
this is the async pattern we want. execute until natural yield point (input
|
||||||
|
needed), then return control to the server.
|
||||||
|
|
||||||
|
object tree (V3)
|
||||||
|
================
|
||||||
|
|
||||||
|
object entry structure
|
||||||
|
----------------------
|
||||||
|
|
||||||
|
V3 object entry is 9 bytes::
|
||||||
|
|
||||||
|
bytes 0-3: attributes (32 boolean flags)
|
||||||
|
byte 4: parent object ID (0=none)
|
||||||
|
byte 5: sibling object ID (0=none)
|
||||||
|
byte 6: first child object ID (0=none)
|
||||||
|
bytes 7-8: address of property table
|
||||||
|
|
||||||
|
V3 allows 255 objects (8-bit IDs). Zork 1 uses 250. the 5 unused slots are
|
||||||
|
where MultiZork puts player objects.
|
||||||
|
|
||||||
|
object tree functions
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
- getObjectPtr(objid) — returns pointer to 9-byte entry in object table
|
||||||
|
- getObjectProperty(objid, propid, *size) — walks property list, returns data
|
||||||
|
- unparentObject(objid) — removes from parent's child list (linear sibling walk)
|
||||||
|
- opcode_insert_obj — unparent, then insert as first child of destination
|
||||||
|
- opcode_remove_obj — unparent, clear parent/sibling fields
|
||||||
|
- opcode_get_parent/get_sibling/get_child — tree traversal
|
||||||
|
- opcode_set_attr/clear_attr/test_attr — bit manipulation on attribute flags
|
||||||
|
|
||||||
|
property tables: variable-length entries in descending ID order. each entry has
|
||||||
|
a size/ID byte (3 bits size, 5 bits propid) followed by data. 31 default
|
||||||
|
property values stored at start of object table.
|
||||||
|
|
||||||
|
multiplayer architecture
|
||||||
|
========================
|
||||||
|
|
||||||
|
the interesting part.
|
||||||
|
|
||||||
|
player state
|
||||||
|
------------
|
||||||
|
|
||||||
|
Player struct (multizorkd.c:96-127) stores per-player snapshots::
|
||||||
|
|
||||||
|
PC, SP, BP + full stack copy (uint16[2048])
|
||||||
|
object_table_data[9] — the player's object (parent/child/sibling/attrs)
|
||||||
|
property_table_data[32] — player name in ZSCII + properties
|
||||||
|
touchbits[32] — per-player room-visited flags (256 bits for 250 objects)
|
||||||
|
|
||||||
|
10 player-specific global variables:
|
||||||
|
location, deaths, dead, lit, verbose, superbrief,
|
||||||
|
alwayslit, lucky, loadallowed, coffin_held
|
||||||
|
|
||||||
|
connection pointer, username, rejoin hash
|
||||||
|
againbuf (last command for "again"/"g")
|
||||||
|
|
||||||
|
Instance struct wraps ZMachineState + up to 4 Players + database IDs + jmpbuf
|
||||||
|
for error recovery.
|
||||||
|
|
||||||
|
the 5-slot trick
|
||||||
|
----------------
|
||||||
|
|
||||||
|
Zork 1 uses objects 1-250 of 255 possible. objects 251-254 become player
|
||||||
|
objects. BUT: no free space in the object table — Infocom's tools pack it
|
||||||
|
tight, with property data immediately after object 250.
|
||||||
|
|
||||||
|
solution: virtual memory. player object data lives outside z-machine address
|
||||||
|
space. when the VM requests object 251-254:
|
||||||
|
|
||||||
|
- getObjectPtr() returns a pointer to player.object_table_data (external buffer)
|
||||||
|
- property table addresses map to 0xFE00-0xFFFF (high memory, unreachable by
|
||||||
|
game code)
|
||||||
|
- get_virtualized_mem_ptr() intercepts these fake addresses and redirects to
|
||||||
|
player.property_table_data
|
||||||
|
|
||||||
|
this works because Zork 1 never directly accesses the object table by address —
|
||||||
|
it always uses the z-machine object opcodes. the override hooks intercept at
|
||||||
|
the opcode level.
|
||||||
|
|
||||||
|
the player swap
|
||||||
|
---------------
|
||||||
|
|
||||||
|
step_instance() (multizorkd.c:1308-1507) — the core multiplayer function.
|
||||||
|
|
||||||
|
before running:
|
||||||
|
|
||||||
|
1. restore player's PC/SP/BP/stack from snapshot
|
||||||
|
2. swap in player-specific globals (globals[0]=location, globals[60]=lucky, etc)
|
||||||
|
3. patch 6 hardcoded "object #4" references in Zork's bytecode to point at this
|
||||||
|
player's object
|
||||||
|
4. set INVISIBLE and NDESCBIT on the player object (so they don't see themselves)
|
||||||
|
5. if input provided, write it into z-machine's input buffer and tokenize
|
||||||
|
|
||||||
|
execute:
|
||||||
|
|
||||||
|
6. while (!step_completed) { runInstruction(); }
|
||||||
|
|
||||||
|
after running:
|
||||||
|
|
||||||
|
7. save PC/SP/BP/stack to player snapshot
|
||||||
|
8. save player-specific globals back to player struct
|
||||||
|
9. save touchbits (room visited flags) for all rooms
|
||||||
|
10. check for win condition (globals[140] != 0), reset touchbits for endgame room
|
||||||
|
11. clear INVISIBLE/NDESCBIT on player object
|
||||||
|
12. if QUIT opcode hit, flag player as game_over, drop connection
|
||||||
|
|
||||||
|
error recovery: setjmp/longjmp around the execution loop. if die() is called,
|
||||||
|
longjmp back, broadcast error, free instance.
|
||||||
|
|
||||||
|
chat system
|
||||||
|
-----------
|
||||||
|
|
||||||
|
- "!message" — say to players in same room (room = same parent object)
|
||||||
|
- "!!message" — broadcast to all players in instance
|
||||||
|
- regular text — z-machine command for current player
|
||||||
|
|
||||||
|
the game-specific problem
|
||||||
|
--------------------------
|
||||||
|
|
||||||
|
the multiplayer code is riddled with "ZORK 1 SPECIFIC MAGIC" comments.
|
||||||
|
|
||||||
|
hardcoded::
|
||||||
|
|
||||||
|
object 4 = player
|
||||||
|
object 82 = rooms container
|
||||||
|
object 180 = West of House (start room)
|
||||||
|
6 bytecode locations where Zork compares against literal 4 instead of using PLAYER global
|
||||||
|
10 specific global variable indices for player state
|
||||||
|
attribute 3 = TOUCHBIT, attribute 7 = INVISIBLE, attribute 14 = NDESCBIT
|
||||||
|
|
||||||
|
Ryan's own words: "there's an enormous amount of hardcoded tapdancing in
|
||||||
|
multizork to work with this build of Zork 1. It won't even work with other
|
||||||
|
versions of Zork 1, as the global variables will be in a different order at the
|
||||||
|
least."
|
||||||
|
|
||||||
|
for other games, you'd need to:
|
||||||
|
|
||||||
|
1. identify which globals are player-specific (game analysis required)
|
||||||
|
2. find hardcoded player object references (disassembly required)
|
||||||
|
3. determine room structure (game-specific object tree analysis)
|
||||||
|
4. identify visibility/description attributes (game-specific)
|
||||||
|
|
||||||
|
this is fundamentally a per-game effort, not a generic system.
|
||||||
|
|
||||||
|
persistence
|
||||||
|
===========
|
||||||
|
|
||||||
|
SQLite schema in multizorkd.c (lines 170-250)::
|
||||||
|
|
||||||
|
instances:
|
||||||
|
hash, num_players, start/save times, dynamic_memory (BLOB), story filename
|
||||||
|
|
||||||
|
players:
|
||||||
|
hash, username, PC/SP/BP, stack (BLOB), object/property/touchbits (BLOBs),
|
||||||
|
all player-specific globals, game_over flag
|
||||||
|
|
||||||
|
transcripts:
|
||||||
|
player FK, texttype (output/input/system), content text
|
||||||
|
|
||||||
|
crashes:
|
||||||
|
instance FK, timestamp, current_player, PC, error string
|
||||||
|
|
||||||
|
blocked:
|
||||||
|
IP address blocking for failed logins
|
||||||
|
|
||||||
|
used_hashes:
|
||||||
|
tracks issued rejoin codes
|
||||||
|
|
||||||
|
auto-save every 30 moves. full dynamic memory snapshot + all player state.
|
||||||
|
|
||||||
|
reconnection: players get a 6-char hash code at game start. typing the code at
|
||||||
|
login restores their session, even if all players disconnected and the game was
|
||||||
|
archived.
|
||||||
|
|
||||||
|
transcripts saved for every session — accessible on the web
|
||||||
|
(multizork-transcripts.php).
|
||||||
|
|
||||||
|
telnet server
|
||||||
|
=============
|
||||||
|
|
||||||
|
simple custom telnet implementation (not a library):
|
||||||
|
|
||||||
|
- select()-based I/O loop (no threads)
|
||||||
|
- newline conversion (\n -> \r\n)
|
||||||
|
- dynamic output buffers (realloc on growth)
|
||||||
|
- connection state machine: READY -> DRAINING -> CLOSING
|
||||||
|
- input function pointers per connection (login/lobby/ingame) — state machine
|
||||||
|
pattern
|
||||||
|
- IP-based rate limiting (blocked table, 24h timeout)
|
||||||
|
|
||||||
|
no telnet option negotiation (no NAWS, no GMCP, no MSDP). raw text in/out.
|
||||||
|
|
||||||
|
what we learn from this
|
||||||
|
========================
|
||||||
|
|
||||||
|
step execution works
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
runInstruction() + step_completed flag = clean pause/resume at input prompts.
|
||||||
|
this is the model we want. whether we use viola, zvm hybrid, or write our own,
|
||||||
|
this pattern is proven.
|
||||||
|
|
||||||
|
memory virtualization is elegant
|
||||||
|
---------------------------------
|
||||||
|
|
||||||
|
two function pointer overrides (get_virtualized_mem_ptr, remap_objectid) let
|
||||||
|
one codebase serve both single-player and multiplayer. no opcode changes
|
||||||
|
needed. the principle: intercept at the memory access level, not the
|
||||||
|
instruction level.
|
||||||
|
|
||||||
|
multiplayer z-machine is possible but game-specific
|
||||||
|
----------------------------------------------------
|
||||||
|
|
||||||
|
MultiZork proves the concept works. multiple players, independent inventory,
|
||||||
|
shared world. but the implementation requires deep knowledge of the specific
|
||||||
|
game's internals. a generic solution would need:
|
||||||
|
|
||||||
|
- static analysis tools to identify player-specific globals
|
||||||
|
- disassembly tools to find hardcoded player references
|
||||||
|
- per-game configuration files mapping globals, objects, attributes
|
||||||
|
|
||||||
|
this is not impossible but it's a significant tooling investment.
|
||||||
|
|
||||||
|
the object table is more hackable than expected
|
||||||
|
------------------------------------------------
|
||||||
|
|
||||||
|
adding objects to a running game works IF the game uses opcodes (not direct
|
||||||
|
memory access) for object operations. the virtual memory trick is clever — fake
|
||||||
|
addresses that the VM can't distinguish from real ones.
|
||||||
|
|
||||||
|
persistence patterns transfer directly
|
||||||
|
---------------------------------------
|
||||||
|
|
||||||
|
the SQLite schema maps cleanly to our needs: quetzal blobs (they use raw memory
|
||||||
|
snapshots), player state, transcripts. the reconnection hash is a good UX
|
||||||
|
pattern.
|
||||||
|
|
||||||
|
V3 vs V5 matters for multiplayer
|
||||||
|
---------------------------------
|
||||||
|
|
||||||
|
V3: 255 objects, 8-bit IDs, 9-byte entries. tight but hackable.
|
||||||
|
|
||||||
|
V5: 65535 objects, 16-bit IDs, 14-byte entries. much more room for player
|
||||||
|
objects but also much more complex object tree. V5 games may use objects more
|
||||||
|
aggressively (modern IF is larger). the "5 spare slots" trick is V3-specific.
|
||||||
|
|
||||||
|
comparison with viola and zvm
|
||||||
|
==============================
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
aspect | mojozork | viola | zvm
|
||||||
|
----------------|--------------------|----------------------|--------------------
|
||||||
|
language | C | Python | Python
|
||||||
|
version support | V3 only | V1-V5 (V6-V8 partial)| V1-V5 (structural)
|
||||||
|
can run games | yes | yes | no (~46/108 opcodes)
|
||||||
|
global state | GState pointer | 13 modules w/globals | instance-based (clean)
|
||||||
|
IO abstraction | function pointers | adapter pattern | abstract ZUI
|
||||||
|
step execution | runInstruction() | feasible (5 lines) | feasible (5 lines)
|
||||||
|
multiplayer | working (Zork 1) | not supported | not supported
|
||||||
|
save/restore | custom format | quetzal | stubbed
|
||||||
|
error handling | die() + longjmp | sys.exit() (8+ locs) | exceptions (clean)
|
||||||
|
object tree | complete V3 | complete V3-V5 | read complete, write partial
|
||||||
|
test suite | script-based | none | minimal
|
||||||
|
usable by us | reference only (C) | embeddable with work | skeleton needs finishing
|
||||||
|
|
||||||
|
the verdict from if-journey.rst still holds: "zvm has the architecture you'd
|
||||||
|
want. viola has the implementation you'd need." MojoZork adds: "and mojozork
|
||||||
|
has the multiplayer proof you need to believe level 4 is achievable."
|
||||||
|
|
||||||
|
related documents
|
||||||
|
=================
|
||||||
|
|
||||||
|
- ``docs/how/if-journey.rst`` — integration vision and roadmap
|
||||||
|
- ``docs/how/viola-embedding-audit.rst`` — viola architecture audit
|
||||||
|
- ``docs/how/zvm-embedding-audit.rst`` — zvm architecture audit
|
||||||
|
- ``docs/how/if-integration.txt`` — original integration plan
|
||||||
|
- Ryan's Patreon post (Aug 17, 2021) — motivation and technical narrative
|
||||||
|
behind MultiZork
|
||||||
156
docs/how/viola-embedding-audit.rst
Normal file
156
docs/how/viola-embedding-audit.rst
Normal file
|
|
@ -0,0 +1,156 @@
|
||||||
|
=============================
|
||||||
|
viola embedding audit — Z-machine interpreter feasibility
|
||||||
|
=============================
|
||||||
|
|
||||||
|
viola is a Python Z-machine interpreter being evaluated for embedding in mudlib
|
||||||
|
to run interactive fiction games over telnet. this audit covers architecture,
|
||||||
|
isolation requirements, and modification paths.
|
||||||
|
|
||||||
|
|
||||||
|
1. global state — fork-level entangled
|
||||||
|
=======================================
|
||||||
|
|
||||||
|
13 of 18 modules in zcode/ have mutable module-level globals.
|
||||||
|
|
||||||
|
critical three:
|
||||||
|
|
||||||
|
- game.py — PC, callstack, currentframe, undolist, interruptstack. modified by
|
||||||
|
20+ functions across multiple modules via global declarations
|
||||||
|
- memory.py — data (entire Z-machine memory as a bytearray). read/written by
|
||||||
|
nearly every module
|
||||||
|
- screen.py — currentWindow, zwindow list, color spectrum dict. all mutated at
|
||||||
|
runtime
|
||||||
|
|
||||||
|
every module does ``import zcode`` then accesses ``zcode.game.PC``,
|
||||||
|
``zcode.memory.data``, etc. hardcoded module-level lookups, not
|
||||||
|
dependency-injected. state mutation touches 1000+ function calls.
|
||||||
|
|
||||||
|
wrapping in a class would require threading a state parameter through the
|
||||||
|
entire codebase. process-level isolation (subprocess per game) is the realistic
|
||||||
|
path for concurrent games.
|
||||||
|
|
||||||
|
|
||||||
|
2. pygame boundary — clean
|
||||||
|
===========================
|
||||||
|
|
||||||
|
zero pygame imports in zcode/. architecture uses adapter pattern:
|
||||||
|
|
||||||
|
- zcode/ imports only vio.zcode as io (aliased)
|
||||||
|
- vio/zcode.py contains ALL pygame calls (1044 lines)
|
||||||
|
- dependency is unidirectional — vio.zcode never imports from zcode/
|
||||||
|
- zcode/ extends io classes via inheritance (window, font, sound channels)
|
||||||
|
|
||||||
|
can swap vio/zcode.py for a telnet IO adapter without touching VM core.
|
||||||
|
|
||||||
|
|
||||||
|
3. error paths — server-hostile
|
||||||
|
================================
|
||||||
|
|
||||||
|
8+ locations call sys.exit() directly:
|
||||||
|
|
||||||
|
- memory out-of-bounds (5 locations in memory.py)
|
||||||
|
- invalid opcode (opcodes.py)
|
||||||
|
- division by zero (numbers.py)
|
||||||
|
- corrupt story file (viola.py)
|
||||||
|
|
||||||
|
fatal path: ``zcode.error.fatal()`` -> ``sys.exit()``. not catchable exceptions.
|
||||||
|
|
||||||
|
fix: patch error.fatal() to raise a custom exception instead.
|
||||||
|
|
||||||
|
|
||||||
|
4. version coverage — V1-V5 solid, V6 partial, V7-V8 theoretical
|
||||||
|
=================================================================
|
||||||
|
|
||||||
|
- V1-V5: all opcodes implemented. most IF games are V3 or V5.
|
||||||
|
- V6: graphics framework exists but 3 opcodes marked "unfinished"
|
||||||
|
- V7-V8: supported at memory/opcode level but likely never tested
|
||||||
|
- no test suite in the repo
|
||||||
|
|
||||||
|
for a MUD, V3 and V5 cover Zork, Curses, Anchorhead, etc.
|
||||||
|
|
||||||
|
|
||||||
|
5. input model — moderate-to-hard adapter needed
|
||||||
|
=================================================
|
||||||
|
|
||||||
|
line input (READ):
|
||||||
|
easy — maps naturally to telnet line mode
|
||||||
|
|
||||||
|
single char (READ_CHAR):
|
||||||
|
hard — needs raw telnet mode or buffering
|
||||||
|
|
||||||
|
timed input:
|
||||||
|
hard — needs async server-side timer
|
||||||
|
|
||||||
|
mouse input:
|
||||||
|
impossible — stub it
|
||||||
|
|
||||||
|
arrow/function keys:
|
||||||
|
moderate — parse ANSI escape sequences
|
||||||
|
|
||||||
|
pygame is hardwired with no abstraction layer. need to create a TelnetInput
|
||||||
|
class implementing the same interface as vio.zcode.input. interface surface is
|
||||||
|
small: getinput, starttimer, stoptimer.
|
||||||
|
|
||||||
|
|
||||||
|
6. step execution mode — moderate refactor, feasible
|
||||||
|
=====================================================
|
||||||
|
|
||||||
|
main loop in routines.py:60 is clean::
|
||||||
|
|
||||||
|
while not quit and not restart:
|
||||||
|
check interrupts -> decode(PC) -> runops(oldpc)
|
||||||
|
|
||||||
|
one instruction = decode + execute. only blocking call is pygame.event.wait()
|
||||||
|
at bottom of input stack. natural yield points are input requests.
|
||||||
|
|
||||||
|
execute_one() API achievable by:
|
||||||
|
|
||||||
|
1. making IO non-blocking
|
||||||
|
2. converting z_read()'s inner while loop to a state machine
|
||||||
|
3. unwrapping recursive interrupt_call() -> execloop() pattern
|
||||||
|
|
||||||
|
|
||||||
|
7. memory/cleanup — will leak, fixable
|
||||||
|
=======================================
|
||||||
|
|
||||||
|
unbounded growth:
|
||||||
|
|
||||||
|
- undo stack (game.py:23) — each undo saves full memory copy (MBs). never
|
||||||
|
cleared.
|
||||||
|
- command history (input.py:22) — every input appended forever
|
||||||
|
- static routine cache (routines.py:30) — never cleared
|
||||||
|
- static word cache (memory.py:108) — never cleared
|
||||||
|
- object caches (objects.py:18-26) — 4 dicts, never purged
|
||||||
|
|
||||||
|
estimate: 10-100+ MB/hour depending on undo usage. fixable by adding cleanup
|
||||||
|
calls to restart/setup path.
|
||||||
|
|
||||||
|
|
||||||
|
8. IFIFF submodule — quetzal is independent
|
||||||
|
============================================
|
||||||
|
|
||||||
|
- quetzal (saves) and blorb (multimedia) are architecturally separate
|
||||||
|
- zcode/game.py imports only quetzal — no blorb references in save/restore
|
||||||
|
- quetzal API: qdata(), save(), restore()
|
||||||
|
- pure stdlib, no external deps
|
||||||
|
- can cherry-pick quetzal.py without pulling in blorb
|
||||||
|
|
||||||
|
|
||||||
|
priority for upstream PRs
|
||||||
|
==========================
|
||||||
|
|
||||||
|
1. patch error.fatal() to raise instead of sys.exit() — small effort, unblocks
|
||||||
|
server use
|
||||||
|
2. add cleanup hooks for caches/undo on restart — small effort, fixes memory
|
||||||
|
leaks
|
||||||
|
3. create IO abstraction interface — medium effort, enables telnet adapter
|
||||||
|
4. step execution mode (execute_one()) — medium effort, enables async embedding
|
||||||
|
5. extract quetzal as standalone — small effort, clean dependency
|
||||||
|
|
||||||
|
|
||||||
|
pragmatic path
|
||||||
|
==============
|
||||||
|
|
||||||
|
the global state issue is bypassed with subprocess isolation (one Python
|
||||||
|
process per game). fine for a MUD with less than 100 concurrent players.
|
||||||
|
process-per-game sidesteps the entire global state refactor.
|
||||||
377
docs/how/zvm-embedding-audit.rst
Normal file
377
docs/how/zvm-embedding-audit.rst
Normal file
|
|
@ -0,0 +1,377 @@
|
||||||
|
=============================
|
||||||
|
zvm embedding audit — Z-machine interpreter feasibility
|
||||||
|
=============================
|
||||||
|
|
||||||
|
zvm is a Python Z-machine interpreter (by Ben Collins-Sussman) being evaluated
|
||||||
|
for embedding in mudlib to run interactive fiction games over telnet. this audit
|
||||||
|
covers architecture, isolation requirements, and modification paths. compared
|
||||||
|
against the viola audit for apples-to-apples decision-making.
|
||||||
|
|
||||||
|
|
||||||
|
1. global state — mostly clean, two leaks
|
||||||
|
==========================================
|
||||||
|
|
||||||
|
state is instance-based. ``ZMachine.__init__`` wires everything together::
|
||||||
|
|
||||||
|
ZMemory(story) -- memory
|
||||||
|
ZStringFactory(mem) -- string decoding
|
||||||
|
ZObjectParser(mem) -- object tree
|
||||||
|
ZStackManager(mem) -- call/data stacks
|
||||||
|
ZOpDecoder(mem, stack) -- instruction decoder
|
||||||
|
ZStreamManager(mem, ui) -- I/O streams
|
||||||
|
ZCpu(mem, opdecoder, stack, objects, string, streams, ui) -- CPU
|
||||||
|
|
||||||
|
dependency graph flows one way: ZCpu depends on everything else, everything
|
||||||
|
else depends on ZMemory. no circular dependencies. clean layering.
|
||||||
|
|
||||||
|
two global leaks:
|
||||||
|
|
||||||
|
- zlogging.py — executes at import time. opens debug.log and disasm.log in cwd,
|
||||||
|
sets root logger to DEBUG. all instances share the same loggers
|
||||||
|
- zcpu.py uses ``random.seed()`` / ``random.randint()`` — global PRNG state.
|
||||||
|
multiple instances would interfere with each other's randomness
|
||||||
|
|
||||||
|
both fixable with ~20 lines. the logging can be made instance-scoped, the PRNG
|
||||||
|
replaced with ``random.Random()`` instances.
|
||||||
|
|
||||||
|
compared to viola's 13/18 modules with mutable globals, this is dramatically
|
||||||
|
better. multiple ZMachine instances in one process is structurally possible.
|
||||||
|
|
||||||
|
|
||||||
|
2. IO boundary — excellent, purpose-built for embedding
|
||||||
|
========================================================
|
||||||
|
|
||||||
|
this is zvm's strongest feature. the README explicitly states the design goal:
|
||||||
|
"no user interface. meant to be used as the backend in other programs."
|
||||||
|
|
||||||
|
IO is fully abstracted via a four-component ``ZUI`` object (zui.py)::
|
||||||
|
|
||||||
|
ZUI(audio, screen, keyboard_input, filesystem)
|
||||||
|
|
||||||
|
each component is an abstract base class with NotImplementedError stubs:
|
||||||
|
|
||||||
|
- ZScreen (zscreen.py) — write(), split_window(), select_window(),
|
||||||
|
set_cursor_position(), erase_window(), set_text_style(), set_text_color()
|
||||||
|
- ZInputStream (zstream.py) — read_line(), read_char() with full signatures
|
||||||
|
for timed input, max length, terminating characters
|
||||||
|
- ZFilesystem (zfilesystem.py) — save_game(), restore_game(),
|
||||||
|
open_transcript_file_for_writing/reading()
|
||||||
|
- ZAudio (zaudio.py) — play_bleep(), play_sound_effect()
|
||||||
|
|
||||||
|
trivialzui.py is the reference stdio implementation showing how to subclass.
|
||||||
|
|
||||||
|
for MUD embedding: implement ZScreen.write() to push to telnet session,
|
||||||
|
ZInputStream.read_line() to receive from telnet reader, ZFilesystem to store
|
||||||
|
saves in SQLite. natural fit.
|
||||||
|
|
||||||
|
compared to viola where pygame is hardwired and you'd need to create a
|
||||||
|
TelnetInput class from scratch, zvm hands you the interface contract.
|
||||||
|
|
||||||
|
|
||||||
|
3. error paths — server-friendly
|
||||||
|
=================================
|
||||||
|
|
||||||
|
zero sys.exit() calls in the zvm/ package. only sys.exit() is in the CLI
|
||||||
|
runner run_story.py, which is appropriate.
|
||||||
|
|
||||||
|
clean exception hierarchy:
|
||||||
|
|
||||||
|
- ZMachineError (zmachine.py)
|
||||||
|
- ZCpuError -> ZCpuIllegalInstruction, ZCpuDivideByZero, ZCpuNotImplemented
|
||||||
|
- ZMemoryError -> ZMemoryIllegalWrite, ZMemoryOutOfBounds, ZMemoryBadMemoryLayout
|
||||||
|
- ZObjectError -> ZObjectIllegalObjectNumber, ZObjectIllegalAttributeNumber
|
||||||
|
- ZStackError -> ZStackNoRoutine, ZStackNoSuchVariable, ZStackPopError
|
||||||
|
- QuetzalError -> QuetzalMalformedChunk, QuetzalMismatchedFile
|
||||||
|
|
||||||
|
CPU run loop does not catch exceptions — errors propagate to the caller.
|
||||||
|
correct behavior for embedding.
|
||||||
|
|
||||||
|
one weakness: some opcodes have bare ``assert`` statements that would be
|
||||||
|
stripped with ``python -O``.
|
||||||
|
|
||||||
|
compared to viola's 8+ sys.exit() calls via error.fatal(), this is exactly
|
||||||
|
what you want for a server.
|
||||||
|
|
||||||
|
|
||||||
|
4. version coverage — V1-V5 declared, V3-V5 structural
|
||||||
|
========================================================
|
||||||
|
|
||||||
|
zmemory.py handles v1-v5 with version-switched code throughout. v6-v8 are
|
||||||
|
not supported (raises ZMemoryUnsupportedVersion).
|
||||||
|
|
||||||
|
opcode table (zcpu.py) has version annotations like ``(op_call_2s, 4)``
|
||||||
|
meaning "available from v4 onward". systematic and well-done.
|
||||||
|
|
||||||
|
v1-v3 object model uses 1-byte pointers (max 255 objects), v4-v5 uses
|
||||||
|
2-byte (max 65535). properly handled in zobjectparser.py.
|
||||||
|
|
||||||
|
test suite only uses curses.z5 — v1-v3 support exists structurally but has
|
||||||
|
untested code paths.
|
||||||
|
|
||||||
|
compared to viola's solid V1-V5 plus partial V6 and theoretical V7-V8,
|
||||||
|
zvm is narrower but the architecture is cleaner where it does exist.
|
||||||
|
|
||||||
|
|
||||||
|
5. input model — abstracted but unimplemented
|
||||||
|
===============================================
|
||||||
|
|
||||||
|
line input (read_line):
|
||||||
|
interface complete — full signature with original_text, max_length,
|
||||||
|
terminating_characters, timed_input_routine, timed_input_interval.
|
||||||
|
trivial implementation handles basic line editing, ignores timed input.
|
||||||
|
|
||||||
|
single char (read_char):
|
||||||
|
interface complete — signature includes timed_input_routine and
|
||||||
|
timed_input_interval. CPU explicitly raises ZCpuNotImplemented if
|
||||||
|
time/routine are nonzero.
|
||||||
|
|
||||||
|
timed input:
|
||||||
|
feature flag system exists (``features["has_timed_input"] = False``).
|
||||||
|
completely unimplemented at CPU level.
|
||||||
|
|
||||||
|
**critical problem**: op_sread (v1-v3) has an empty body — silently does
|
||||||
|
nothing. op_sread_v4 and op_aread (v5) both raise ZCpuNotImplemented.
|
||||||
|
the interpreter cannot accept text input. it cannot reach the first prompt
|
||||||
|
of any game.
|
||||||
|
|
||||||
|
compared to viola where input works and needs an adapter, zvm has the
|
||||||
|
better interface design but no working implementation behind it.
|
||||||
|
|
||||||
|
|
||||||
|
6. step execution mode — not available, easy refactor
|
||||||
|
======================================================
|
||||||
|
|
||||||
|
run loop in zcpu.py::
|
||||||
|
|
||||||
|
def run(self):
|
||||||
|
while True:
|
||||||
|
(opcode_class, opcode_number, operands) = self._opdecoder.get_next_instruction()
|
||||||
|
implemented, func = self._get_handler(opcode_class, opcode_number)
|
||||||
|
if not implemented:
|
||||||
|
break
|
||||||
|
func(self, *operands)
|
||||||
|
|
||||||
|
tight while-True with no yielding. no step() method, no async support, no
|
||||||
|
callbacks between instructions, no instruction count limit.
|
||||||
|
|
||||||
|
the loop body is clean and self-contained though. extracting step() is a ~5
|
||||||
|
line change — pull the body into a method, call it from run().
|
||||||
|
|
||||||
|
for async MUD embedding, options are:
|
||||||
|
|
||||||
|
1. extract step() and call from an async loop with awaits at IO points
|
||||||
|
2. run each ZMachine in a thread (viable since state is instance-based)
|
||||||
|
|
||||||
|
compared to viola's similar tight loop, the refactor here is actually easier
|
||||||
|
because the body is simpler (no interrupt handling, no recursive execloop).
|
||||||
|
|
||||||
|
|
||||||
|
7. memory/cleanup — clean, no leak risk
|
||||||
|
=========================================
|
||||||
|
|
||||||
|
memory is a bytearray, fixed size, bounded by story file.
|
||||||
|
ZMachine.__init__ creates two copies (pristine + working).
|
||||||
|
|
||||||
|
ZStackManager uses a Python list as call stack. frames are popped in
|
||||||
|
finish_routine(). bounded by Z-machine stack design.
|
||||||
|
|
||||||
|
ZStringFactory, ZCharTranslator, ZLexer all pre-load from story file at init.
|
||||||
|
bounded by file content.
|
||||||
|
|
||||||
|
ZObjectParser does not cache anything — every access reads directly from
|
||||||
|
ZMemory.
|
||||||
|
|
||||||
|
one minor leak: quetzal.py QuetzalParser stores self._file and closes it
|
||||||
|
manually but does not use a context manager. exception during parsing would
|
||||||
|
leak the file handle.
|
||||||
|
|
||||||
|
compared to viola's 5+ unbounded growth patterns (undo stack, command history,
|
||||||
|
routine cache, word cache, object caches), zvm is dramatically cleaner.
|
||||||
|
|
||||||
|
|
||||||
|
8. object tree API — complete read, mostly complete write
|
||||||
|
==========================================================
|
||||||
|
|
||||||
|
zobjectparser.py public API:
|
||||||
|
|
||||||
|
read:
|
||||||
|
- get_attribute(objectnum, attrnum) — single attribute (0/1)
|
||||||
|
- get_all_attributes(objectnum) — list of all set attribute numbers
|
||||||
|
- get_parent(objectnum) — parent object number
|
||||||
|
- get_child(objectnum) — first child
|
||||||
|
- get_sibling(objectnum) — next sibling
|
||||||
|
- get_shortname(objectnum) — object's short name as string
|
||||||
|
- get_prop(objectnum, propnum) — property value
|
||||||
|
- get_prop_addr_len(objectnum, propnum) — property address and length
|
||||||
|
- get_all_properties(objectnum) — dict of all properties
|
||||||
|
- describe_object(objectnum) — debug pretty-printer
|
||||||
|
|
||||||
|
write:
|
||||||
|
- set_parent(objectnum, new_parent_num)
|
||||||
|
- set_child(objectnum, new_child_num)
|
||||||
|
- set_sibling(objectnum, new_sibling_num)
|
||||||
|
- insert_object(parent_object, new_child) — handles unlinking
|
||||||
|
- set_property(objectnum, propnum, value)
|
||||||
|
|
||||||
|
missing: set_attribute() and clear_attribute(). the parser can read attributes
|
||||||
|
but cannot set them. would need to be added.
|
||||||
|
|
||||||
|
bug in insert_object(): sibling walk loop at line 273 never advances current
|
||||||
|
or prev — would infinite-loop. needs fix.
|
||||||
|
|
||||||
|
**critical for MUD embedding**: many CPU opcodes that USE the parser are
|
||||||
|
unimplemented at the CPU level:
|
||||||
|
|
||||||
|
- op_test_attr, op_set_attr, op_clear_attr — not wired up
|
||||||
|
- op_get_sibling — not wired up (parser method exists)
|
||||||
|
- op_jin (test parent) — not wired up
|
||||||
|
- op_remove_obj, op_print_obj — not wired up
|
||||||
|
- op_get_prop_addr, op_get_next_prop, op_get_prop_len — not wired up
|
||||||
|
|
||||||
|
the parser infrastructure is there and correct. the CPU just doesn't call it.
|
||||||
|
|
||||||
|
compared to viola's zcode/objects.py which has working accessors wired through
|
||||||
|
all opcodes, zvm has a better parser design but the plumbing is incomplete.
|
||||||
|
|
||||||
|
|
||||||
|
9. save/restore — parse works, write is stubbed
|
||||||
|
=================================================
|
||||||
|
|
||||||
|
quetzal.py QuetzalParser can parse Quetzal save files:
|
||||||
|
- IFhd chunks (metadata) — working
|
||||||
|
- CMem chunks (compressed memory) — working
|
||||||
|
- UMem chunks (uncompressed memory) — has a bug (wrong attribute name)
|
||||||
|
- Stks chunks (stack frames) — working
|
||||||
|
|
||||||
|
QuetzalWriter is almost entirely stubbed. all three data methods return "0".
|
||||||
|
file writing logic exists but writes nonsense.
|
||||||
|
|
||||||
|
all save/restore CPU opcodes (op_save, op_restore, op_save_v4, op_restore_v4,
|
||||||
|
op_save_v5, op_restore_v5, op_save_undo, op_restore_undo) raise
|
||||||
|
ZCpuNotImplemented.
|
||||||
|
|
||||||
|
alternative for MUD: since ZMemory is a bytearray, could snapshot/restore
|
||||||
|
raw memory + stack state directly without Quetzal format.
|
||||||
|
|
||||||
|
compared to viola's working quetzal implementation, zvm's save system needs
|
||||||
|
significant work.
|
||||||
|
|
||||||
|
|
||||||
|
10. test suite — minimal
|
||||||
|
=========================
|
||||||
|
|
||||||
|
5 registered test modules:
|
||||||
|
|
||||||
|
- bitfield_tests.py — 6 tests, BitField bit manipulation
|
||||||
|
- zscii_tests.py — 4 tests, string encoding/decoding
|
||||||
|
- lexer_tests.py — 3 tests, dictionary parsing
|
||||||
|
- quetzal_tests.py — 2 tests, save file parsing
|
||||||
|
- glk_tests.py — 7 tests, requires compiled CheapGlk .so
|
||||||
|
|
||||||
|
not tested at all: ZCpu (no opcode tests), ZMemory, ZObjectParser,
|
||||||
|
ZStackManager, ZOpDecoder, ZStreamManager.
|
||||||
|
|
||||||
|
all tests that need a story file use stories/curses.z5 with hardcoded paths.
|
||||||
|
|
||||||
|
compared to viola which has no tests at all, zvm has some but they don't
|
||||||
|
cover the critical subsystems.
|
||||||
|
|
||||||
|
|
||||||
|
11. dependencies — zero
|
||||||
|
========================
|
||||||
|
|
||||||
|
pure stdlib. setup.py declares no dependencies. python >= 3.6.
|
||||||
|
|
||||||
|
uses: logging, random, time, itertools, re, chunk, os, sys.
|
||||||
|
ctypes only for optional Glk native integration.
|
||||||
|
|
||||||
|
both viola and zvm are pure stdlib. no advantage either way.
|
||||||
|
|
||||||
|
|
||||||
|
12. completeness — the dealbreaker
|
||||||
|
====================================
|
||||||
|
|
||||||
|
of ~108 Z-machine opcodes, zvm implements approximately 46. the remaining ~62
|
||||||
|
are stubbed with ZCpuNotImplemented or have empty bodies.
|
||||||
|
|
||||||
|
unimplemented opcodes include fundamentals:
|
||||||
|
|
||||||
|
- ALL input opcodes (op_sread, op_aread) — cannot accept player input
|
||||||
|
- ALL save/restore opcodes — cannot save or load games
|
||||||
|
- critical object opcodes (test_attr, set_attr, clear_attr, get_sibling,
|
||||||
|
remove_obj, print_obj at CPU level)
|
||||||
|
- many branch/comparison ops
|
||||||
|
- string printing variants
|
||||||
|
|
||||||
|
the interpreter cannot execute any real interactive fiction game to
|
||||||
|
completion. it cannot reach the first prompt of Zork.
|
||||||
|
|
||||||
|
|
||||||
|
verdict — comparison with viola
|
||||||
|
================================
|
||||||
|
|
||||||
|
+---------------------+--------------+----------------+
|
||||||
|
| criterion | zvm | viola |
|
||||||
|
+---------------------+--------------+----------------+
|
||||||
|
| IO abstraction | excellent | needs adapter |
|
||||||
|
| global state | mostly clean | deeply tangled |
|
||||||
|
| multi-instance | structurally | process-only |
|
||||||
|
| error handling | exceptions | sys.exit() |
|
||||||
|
| memory leaks | none | 5+ patterns |
|
||||||
|
| object tree parser | complete | complete |
|
||||||
|
| object tree opcodes | ~half wired | all wired |
|
||||||
|
| opcode coverage | ~46/108 | all V1-V5 |
|
||||||
|
| can run a game | NO | YES |
|
||||||
|
| input handling | abstracted | working |
|
||||||
|
| save/restore | parse only | working |
|
||||||
|
| dependencies | zero | zero |
|
||||||
|
| tests | minimal | none |
|
||||||
|
| maintenance | abandoned | abandoned |
|
||||||
|
+---------------------+--------------+----------------+
|
||||||
|
|
||||||
|
zvm has the architecture you'd want. viola has the implementation you'd need.
|
||||||
|
|
||||||
|
zvm is a well-designed skeleton — clean IO abstraction, instance-based state,
|
||||||
|
proper exceptions, no memory leaks. but it's roughly half-built. finishing the
|
||||||
|
62 missing opcodes is weeks of work equivalent to writing a new interpreter,
|
||||||
|
except you're also debugging someone else's partial implementation.
|
||||||
|
|
||||||
|
viola is a working interpreter with terrible architecture for embedding —
|
||||||
|
global state everywhere, sys.exit() in error paths, pygame hardwired. but it
|
||||||
|
can run Zork right now. the refactoring targets are known and bounded.
|
||||||
|
|
||||||
|
|
||||||
|
pragmatic path
|
||||||
|
===============
|
||||||
|
|
||||||
|
the "moldable world" vision (levels 3-5 from the design discussion) requires
|
||||||
|
being inside the interpreter with access to the object tree. both interpreters
|
||||||
|
have the parser infrastructure for this.
|
||||||
|
|
||||||
|
option A — fix viola's embedding problems:
|
||||||
|
1. patch error.fatal() to raise (small)
|
||||||
|
2. swap pygame IO for telnet adapter (medium)
|
||||||
|
3. add cleanup hooks for caches (small)
|
||||||
|
4. subprocess isolation handles global state (free)
|
||||||
|
total: working IF in a MUD, with known limitations on multi-instance
|
||||||
|
|
||||||
|
option B — finish zvm's implementation:
|
||||||
|
1. implement ~62 missing opcodes (large)
|
||||||
|
2. fix insert_object bug (small)
|
||||||
|
3. add set_attribute/clear_attribute to parser (small)
|
||||||
|
4. complete save writer (medium)
|
||||||
|
total: clean embeddable interpreter, but weeks of opcode work first
|
||||||
|
|
||||||
|
option C — write our own interpreter:
|
||||||
|
designed for embedding from day one. state is a first-class object. object
|
||||||
|
tree is an API. multiple games in one process. but it's the longest path and
|
||||||
|
testing against real games is the hard part.
|
||||||
|
|
||||||
|
option D — hybrid:
|
||||||
|
use zvm's architecture (ZUI interface, exception model, instance-based state)
|
||||||
|
as the skeleton. port viola's working opcode implementations into it. gets
|
||||||
|
the clean design with the working code. medium effort, high reward.
|
||||||
|
|
||||||
|
the hybrid path is probably the most interesting. zvm got the hard design
|
||||||
|
decisions right. viola got the hard implementation work done. merging the two
|
||||||
|
is less work than either finishing zvm or refactoring viola.
|
||||||
Loading…
Reference in a new issue