Compare commits
10 commits
2cf303dd67
...
8097bbcf55
| Author | SHA1 | Date | |
|---|---|---|---|
| 8097bbcf55 | |||
| ec4e53b2d4 | |||
| 1b08c36b85 | |||
| bbd70e8577 | |||
| 6e62ca203b | |||
| fa31f39a65 | |||
| f8f5ac7ad0 | |||
| 127ca5f56e | |||
| 5ffbe660fb | |||
| 2ce82e7d87 |
9 changed files with 711 additions and 38 deletions
|
|
@ -198,7 +198,7 @@ How many of the ~62 missing zvm opcodes are actually exercised by V3 games? V3 u
|
|||
|
||||
UPDATE: Opcode tracing (via ``scripts/trace_zmachine.py``) found Zork 1 uses 69 opcodes. zvm had 36 implemented. 33 were ported from viola. All 69 are now implemented in the hybrid interpreter (``src/mudlib/zmachine/``).
|
||||
|
||||
Remaining gaps: save/restore (QuetzalWriter needs completion) and sread tokenization.
|
||||
All V3 gaps have been resolved. sread tokenization works correctly. save/restore is not yet functional (see question 7).
|
||||
|
||||
2. zvm/viola memory layout compatibility
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
|
@ -231,6 +231,16 @@ Infocom games are abandonware but not legally free. Modern IF games (Lost Pig, W
|
|||
|
||||
How hard is it to add words to a z-machine dictionary at runtime? The dictionary is in static memory. Adding words means expanding it, which means relocating it if there's no space. Is this practical?
|
||||
|
||||
7. Save/restore in the hybrid interpreter
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
``op_save`` is a stub that always branches false (game prints "Failed."). The infrastructure is mostly there — ``TrivialFilesystem.save_game()`` prompts for a filename and writes to disk, ``QuetzalParser`` can read save files — but two pieces are missing:
|
||||
|
||||
- ``QuetzalWriter`` chunk generators (``ifhd``, ``cmem``, ``stks``) are all stubs returning ``"0"``
|
||||
- ``op_save`` doesn't collect game state or call the filesystem
|
||||
|
||||
To make save work: implement ``QuetzalWriter`` (XOR-compress dynamic memory against original story, serialize stack frames into Quetzal format), then wire ``op_save`` to generate the bytes and call ``self._ui.filesystem.save_game(data)``. Restore should be simpler since ``QuetzalParser`` already works — just need to wire ``op_restore`` to call ``filesystem.restore_game()`` and apply the parsed state.
|
||||
|
||||
what to do next
|
||||
---------------
|
||||
|
||||
|
|
@ -244,6 +254,8 @@ Concrete next steps, roughly ordered. Update as items get done.
|
|||
|
||||
- [x] build level 1 prototype: regardless of interpreter choice, implement the terminal object, IF mode, and subprocess dfrotz path. this proves the MUD-side architecture (mode stack, spectators, save/restore) independently of the interpreter question. (done — see ``docs/how/if-terminal.txt``)
|
||||
|
||||
- [ ] implement save/restore: finish ``QuetzalWriter`` chunk generators and wire ``op_save``/``op_restore`` to the filesystem layer. restore should be easier since ``QuetzalParser`` already works.
|
||||
|
||||
- [ ] study MojoZork's multiplayer model: read the MultiZork source for how it handles multiple players in one z-machine. document the pattern for our eventual level 4.
|
||||
|
||||
- [x] find the game files: locate freely distributable z-machine story files for the games we care about. Wizard Sniffer, Lost Pig, Zork (if legally available). (zork1.z3 bundled in content/stories/)
|
||||
|
|
@ -260,6 +272,8 @@ What works:
|
|||
- ``step()`` method for async MUD integration — single instruction at a time, no blocking loop
|
||||
- instruction trace deque (last 20 instructions) for debugging state errors
|
||||
- smoke test: ``scripts/run_zork1.py`` runs the game headless, exercises core opcode paths
|
||||
- parser and lexer: all Zork 1 commands work (look, open mailbox, read leaflet, inventory, take, drop, navigation)
|
||||
- the interpreter is fully playable for Zork 1 (save/restore not yet wired — see open question 7)
|
||||
|
||||
What this enables:
|
||||
|
||||
|
|
|
|||
344
docs/how/zmachine-garbled-output-investigation.rst
Normal file
344
docs/how/zmachine-garbled-output-investigation.rst
Normal file
|
|
@ -0,0 +1,344 @@
|
|||
Z-Machine Garbled Output Investigation
|
||||
========================================
|
||||
|
||||
Context
|
||||
-------
|
||||
|
||||
Branch: ``zmachine-zork1-integration``
|
||||
|
||||
The hybrid z-machine interpreter runs Zork 1 but produces garbled output:
|
||||
|
||||
- Room name "West of House" prints correctly
|
||||
- Room description shows "er " instead of full text
|
||||
- After "look" command, output starts correctly then becomes garbage with ? characters
|
||||
- "There is a small mailbox here." prints correctly
|
||||
|
||||
Investigation
|
||||
-------------
|
||||
|
||||
The z-string decoder (ZStringTranslator, ZCharTranslator, ZsciiTranslator in
|
||||
zstring.py) is correct. The abbreviation table loads correctly. All 96
|
||||
abbreviations decode to the right text.
|
||||
|
||||
The diagnostic script (``scripts/debug_zstrings.py``) revealed the problem:
|
||||
|
||||
- ``op_print_paddr paddr=0x0051 (byte=0x00a2)`` decodes to "er "
|
||||
- Address 0x00a2 is in the abbreviation table area, not where the room
|
||||
description should be
|
||||
- The z-string at 0x00a2 really is "er " - the decoder is right, the address
|
||||
is wrong
|
||||
|
||||
Root cause: bugs in ``zobjectparser.py`` property parsing.
|
||||
|
||||
Bugs Found
|
||||
----------
|
||||
|
||||
All bugs are in ``zobjectparser.py``.
|
||||
|
||||
Bug 1: Off-by-one in shortname skip (ROOT CAUSE)
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
In ``get_prop_addr_len()`` line 392::
|
||||
|
||||
addr += 2 * self._memory[addr]
|
||||
|
||||
This skips the shortname data but not the length byte. Compare with
|
||||
``get_all_properties()`` which does it correctly::
|
||||
|
||||
shortname_length = self._memory[addr]
|
||||
addr += 1 # skip length byte
|
||||
addr += 2 * shortname_length # skip shortname data
|
||||
|
||||
The off-by-one means the property table scan starts 1 byte too early, reading
|
||||
the last byte of the shortname z-string as if it were a property header. This
|
||||
corrupts every subsequent property lookup.
|
||||
|
||||
Bug 2: BitField slice wrong width for V3 property number
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
In ``get_prop_addr_len()`` line 399 and ``get_all_properties()`` line 446::
|
||||
|
||||
pnum = bf[4:0]
|
||||
|
||||
BitField slice ``[4:0]`` has start > stop, so it swaps to ``[0:4]`` = 4 bits.
|
||||
But V3 property numbers are 5 bits (bits 0-4). Should be ``bf[0:5]``.
|
||||
|
||||
This means property numbers 16-31 are read as 0-15 (bit 4 is lost).
|
||||
|
||||
Bug 3: BitField slice wrong width for V3 property size
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
In ``get_prop_addr_len()`` line 400, ``get_all_properties()`` line 447, and
|
||||
``get_property_length()`` line 555::
|
||||
|
||||
size = bf[7:5] + 1
|
||||
|
||||
BitField slice ``[7:5]`` swaps to ``[5:7]`` = 2 bits. But V3 property sizes are
|
||||
3 bits (bits 5-7). Should be ``bf[5:8]``.
|
||||
|
||||
This means properties with size > 4 bytes are read with the wrong size (e.g.,
|
||||
size 5 reads as size 1), causing the property iterator to skip too few bytes.
|
||||
|
||||
Bug 4: Same BitField slice issue in V5 path
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
In ``get_prop_addr_len()`` lines 409, 413::
|
||||
|
||||
pnum = bf[5:0] # swaps to [0:5] = 5 bits, should be [0:6] = 6 bits
|
||||
size = bf2[5:0] # swaps to [0:5] = 5 bits, should be [0:6] = 6 bits
|
||||
|
||||
Compare with ``get_all_properties()`` V5 path which correctly uses ``bf[0:6]``.
|
||||
|
||||
Also in ``get_property_length()`` line 563: same ``bf2[5:0]`` issue.
|
||||
|
||||
How to Fix
|
||||
----------
|
||||
|
||||
All fixes are in ``zobjectparser.py``:
|
||||
|
||||
1. Line 392: change ``addr += 2 * self._memory[addr]`` to::
|
||||
|
||||
addr += 1 + 2 * self._memory[addr]
|
||||
|
||||
2. Lines 399, 446: change ``bf[4:0]`` to ``bf[0:5]``
|
||||
|
||||
3. Lines 400, 447, 555: change ``bf[7:5]`` to ``bf[5:8]``
|
||||
|
||||
4. Lines 409, 413, 563: change ``bf[5:0]`` to ``bf[0:6]``
|
||||
|
||||
The pattern: BitField uses Python slice semantics ``[start:stop)`` where the
|
||||
number of bits extracted is ``stop - start``. The buggy code used ``[high:low]``
|
||||
notation thinking it would extract bits high through low inclusive, but the swap
|
||||
logic makes it extract fewer bits than intended.
|
||||
|
||||
This is the same class of bug as the write_global fix in commit e5329d6
|
||||
(BitField slice extracting 7 bits not 8).
|
||||
|
||||
Diagnostic Tools
|
||||
----------------
|
||||
|
||||
``scripts/debug_zstrings.py`` traces all z-string decoding with caller info.
|
||||
|
||||
Run with::
|
||||
|
||||
echo -e "look\nquit\ny" | python3 scripts/debug_zstrings.py 2>/tmp/trace.log
|
||||
|
||||
Trace goes to stderr, game output to stdout.
|
||||
|
||||
Session 2: Additional Bugs Found and Fixed
|
||||
-------------------------------------------
|
||||
|
||||
Bug 5: Same off-by-one in get_next_property
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
``get_next_property()`` had the same shortname skip bug as Bug 1::
|
||||
|
||||
addr += 2 * self._memory[addr]
|
||||
|
||||
Same fix: ``addr += 1 + 2 * self._memory[addr]``
|
||||
|
||||
Also had the same BitField slice bugs (Bugs 2, 4) for the V3/V5
|
||||
property number return values.
|
||||
|
||||
Bug 6: get_property_length reads wrong byte for V5 two-byte headers
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
In the V5 two-byte property header path, the code read the size from
|
||||
the first header byte (which contains the property number) instead of
|
||||
the second header byte (which contains the size)::
|
||||
|
||||
bf2 = BitField(self._memory[data_address - 2]) # WRONG: first byte
|
||||
size = bf2[0:6]
|
||||
|
||||
The second byte was already read into ``bf`` at ``data_address - 1``.
|
||||
Fix: use ``bf[0:6]`` directly instead of creating ``bf2``.
|
||||
|
||||
Bug 7: op_dec_chk and op_inc_chk use unsigned comparison
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
In ``zcpu.py``, both ``op_dec_chk`` and ``op_inc_chk`` compared 16-bit
|
||||
values as unsigned integers instead of signed. The Z-machine spec
|
||||
requires signed comparison.
|
||||
|
||||
Effect: when a counter decremented from 0 to 65535 (wrapping), the check
|
||||
``65535 < 0`` evaluated false (unsigned), causing infinite loops in Z-code
|
||||
that counted down past zero. This made Zork 1's word-printing loop run
|
||||
forever, dumping raw memory bytes to the screen as characters.
|
||||
|
||||
Fix: wrap both operands with ``_make_signed()`` before comparing.
|
||||
|
||||
What's Fixed Now
|
||||
-----------------
|
||||
|
||||
After all fixes:
|
||||
|
||||
- Room descriptions display correctly on startup
|
||||
- The infinite garbage output loop is fixed (dec_chk now works with signed)
|
||||
- ``just check`` passes (516 tests)
|
||||
|
||||
What's Still Broken
|
||||
--------------------
|
||||
|
||||
Commands like "look", "open mailbox", "quit" all produce::
|
||||
|
||||
You used the word "look" in a way that I don't understand.
|
||||
|
||||
The parser finds words in the dictionary (dict_addr is correct, e.g.
|
||||
"look" at 0x44f2) and stores them in the parse buffer correctly. But the
|
||||
game's Z-code parser can't match them to valid verb syntax.
|
||||
|
||||
Instruction tracing after ``op_sread`` shows the game's parser:
|
||||
|
||||
1. Reads parse buffer correctly (loadw(0x2551, 1) -> dict_addr 17650)
|
||||
2. Checks dict_addr against specific word addresses (je comparisons) -
|
||||
these are special-word checks, none match
|
||||
3. Calls a function with the dict_addr that reads ``loadb(17650, 4)`` to
|
||||
get the word's flags byte (0x41) from the dictionary entry
|
||||
4. Tests individual flag bits (16, 64, 8, 32, 128, 4) against 0x41
|
||||
5. Bit 6 (64) matches - the function extracts data from byte 5 (0xc2)
|
||||
6. Despite finding the word type, the parser still rejects the command
|
||||
|
||||
The bug is likely in how the parser uses the extracted word-type data to
|
||||
look up verb syntax. Possible areas:
|
||||
|
||||
- The ``and`` / ``je`` sequence that extracts a sub-field from the flags
|
||||
byte may produce the wrong value
|
||||
- The verb number extracted from byte 5 (0xc2 = 194) may need signed
|
||||
interpretation or bit masking
|
||||
- Another opcode (``op_jl``, ``op_jg``, ``op_sub``) may have a similar
|
||||
unsigned-vs-signed bug
|
||||
- The ``call`` opcode may have incorrect return value handling
|
||||
|
||||
Debugging approach for next session:
|
||||
|
||||
1. Extend the instruction trace past 100 steps to see what happens after
|
||||
the word-type function returns
|
||||
2. Compare the execution trace against dfrotz instruction trace (add
|
||||
``-t`` flag to dfrotz for tracing)
|
||||
3. Check ``op_jl`` and ``op_jg`` for the same unsigned comparison bug
|
||||
4. Check ``op_sub`` and ``op_mul`` for signed arithmetic issues
|
||||
5. Look at how the ``and`` result (step 21 in trace) flows through to
|
||||
the verb syntax lookup
|
||||
|
||||
Diagnostic Tools
|
||||
~~~~~~~~~~~~~~~~
|
||||
|
||||
``scripts/debug_zstrings.py`` traces all z-string decoding with caller info.
|
||||
|
||||
Run with::
|
||||
|
||||
echo -e "look\nquit\ny" | python3 scripts/debug_zstrings.py 2>/tmp/trace.log
|
||||
|
||||
Trace goes to stderr, game output to stdout.
|
||||
|
||||
To add instruction tracing after sread, add to ``zcpu.py``::
|
||||
|
||||
# In step(), after decoding:
|
||||
print(f"PC={pc:#06x} {handler_name}({', '.join(str(a) for a in args)})",
|
||||
file=sys.stderr)
|
||||
|
||||
Session 3: Parser Fixed - Interpreter Now Playable
|
||||
---------------------------------------------------
|
||||
|
||||
Bug 8: op_jl uses unsigned comparison
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
In ``zcpu.py``, ``op_jl`` (jump if less-than) compared raw 16-bit values
|
||||
without ``_make_signed()``, the same class of bug as Bug 7 (op_dec_chk
|
||||
and op_inc_chk).
|
||||
|
||||
Additionally, ``op_jl`` had a non-standard signature::
|
||||
|
||||
def op_jl(self, a, *others):
|
||||
for b in others:
|
||||
if a < b:
|
||||
...
|
||||
|
||||
The Z-machine spec says JL takes exactly 2 operands. Compare with
|
||||
``op_jg`` which correctly uses ``def op_jg(self, a, b)``.
|
||||
|
||||
Fix: add ``_make_signed()`` calls to both operands, normalize signature
|
||||
to ``(self, a, b)`` matching ``op_jg``.
|
||||
|
||||
This alone did not fix the parser - needed in combination with Bug 9.
|
||||
|
||||
Bug 9: finish_routine return value storage - THE PARSER BUG
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Two sub-bugs in ``finish_routine()`` in ``zstackmanager.py``:
|
||||
|
||||
**Range check used decimal instead of hex**::
|
||||
|
||||
if result_variable < 10: # WRONG: should be 0x10 (16)
|
||||
self.set_local_variable(result_variable, result)
|
||||
|
||||
Z-machine variable numbering: 0 = stack push, 1-15 = locals, 16+ = globals.
|
||||
The check ``< 10`` meant local variables 10-15 were written as globals.
|
||||
|
||||
**Missing index adjustment for 1-based variable numbering**::
|
||||
|
||||
self.set_local_variable(result_variable, result) # WRONG
|
||||
|
||||
Z-machine variables are 1-based (variable 1 = first local), but
|
||||
``set_local_variable()`` uses 0-based indexing (index 0 = first local).
|
||||
Should be ``set_local_variable(result_variable - 1, result)``.
|
||||
|
||||
Both ``_read_variable()`` and ``_write_result()`` in ``zcpu.py``
|
||||
correctly used ``addr - 1`` and ``< 0x10``, but ``finish_routine()``
|
||||
in ``zstackmanager.py`` did not.
|
||||
|
||||
Effect: when a function returned a value to a local variable, it went
|
||||
to the wrong slot. The caller read the correct slot and got stale data
|
||||
(0 or the initialization value from the function call).
|
||||
|
||||
This is why every verb was recognized by the dictionary but rejected by
|
||||
the parser. The word-type checker function returned the correct type,
|
||||
but the return value landed in the wrong local variable. The parser saw
|
||||
0 and could not match any syntax pattern.
|
||||
|
||||
Room descriptions worked because those code paths used stack returns
|
||||
(variable 0) or global returns, not local variable returns.
|
||||
|
||||
Fix: change ``< 10`` to ``< 0x10``, add ``- 1`` to the
|
||||
``set_local_variable`` call.
|
||||
|
||||
Bug 10: zlexer does not truncate words for dictionary lookup
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
In ``zlexer.py``, V3 dictionary entries store words truncated to 6
|
||||
characters (4 bytes of Z-string = 2 Z-words). V4+ uses 9 characters.
|
||||
|
||||
``parse_input()`` looked up full-length input words, so "mailbox" (7
|
||||
characters) did not match "mailbo" (6 characters) in the dictionary.
|
||||
|
||||
Fix: truncate lookup key to ``6 if version <= 3 else 9`` before
|
||||
``dict.get()``. The original word is preserved in the return list for
|
||||
correct parse buffer positions.
|
||||
|
||||
What's Fixed Now
|
||||
~~~~~~~~~~~~~~~~
|
||||
|
||||
After all fixes:
|
||||
|
||||
- All Zork 1 commands work: look, open mailbox, read leaflet, go north,
|
||||
inventory, quit, take, drop
|
||||
- Navigation, object manipulation, multi-word commands, game logic all
|
||||
functional
|
||||
- The interpreter is playable
|
||||
|
||||
Diagnostic Tools
|
||||
----------------
|
||||
|
||||
``scripts/debug_zstrings.py`` traces all z-string decoding with caller info.
|
||||
|
||||
Run with::
|
||||
|
||||
echo -e "look\nquit\ny" | python3 scripts/debug_zstrings.py 2>/tmp/trace.log
|
||||
|
||||
Trace goes to stderr, game output to stdout.
|
||||
|
||||
To add instruction tracing after sread, add to ``zcpu.py``::
|
||||
|
||||
# In step(), after decoding:
|
||||
print(f"PC={pc:#06x} {handler_name}({', '.join(str(a) for a in args)})",
|
||||
file=sys.stderr)
|
||||
189
scripts/debug_zstrings.py
Executable file
189
scripts/debug_zstrings.py
Executable file
|
|
@ -0,0 +1,189 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Diagnostic script to trace z-string decoding in Zork 1.
|
||||
|
||||
This script loads Zork 1 and traces the z-string decoding pipeline:
|
||||
1. Dumps all abbreviations from the abbreviation table
|
||||
2. Monkey-patches ZStringFactory.get() to log every string decode
|
||||
3. Runs the game with piped input to capture string decoding during gameplay
|
||||
"""
|
||||
|
||||
import inspect
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def main():
|
||||
# Add src to path so we can import mudlib
|
||||
project_root = Path(__file__).parent.parent
|
||||
sys.path.insert(0, str(project_root / "src"))
|
||||
|
||||
from mudlib.zmachine import ZMachine
|
||||
from mudlib.zmachine.trivialzui import create_zui
|
||||
from mudlib.zmachine.zstring import ZStringFactory
|
||||
|
||||
print("=" * 80)
|
||||
print("Z-STRING DECODING DIAGNOSTIC FOR ZORK 1")
|
||||
print("=" * 80)
|
||||
print()
|
||||
|
||||
story_path = project_root / "content" / "stories" / "zork1.z3"
|
||||
|
||||
if not story_path.exists():
|
||||
print(f"ERROR: Story file not found at {story_path}")
|
||||
sys.exit(1)
|
||||
|
||||
with open(story_path, "rb") as f:
|
||||
story_bytes = f.read()
|
||||
|
||||
print(f"Loaded {len(story_bytes)} bytes from {story_path.name}")
|
||||
print()
|
||||
|
||||
# Create ZUI and ZMachine
|
||||
ui = create_zui()
|
||||
zmachine = ZMachine(story_bytes, ui, debugmode=False)
|
||||
|
||||
# Access the string factory
|
||||
string_factory = zmachine._stringfactory
|
||||
|
||||
# PART 1: Dump all abbreviations
|
||||
print("=" * 80)
|
||||
print("ABBREVIATION TABLE DUMP")
|
||||
print("=" * 80)
|
||||
print()
|
||||
|
||||
abbrevs = string_factory.zchr._abbrevs
|
||||
if not abbrevs:
|
||||
print("No abbreviations found (version 1 game or failed to load)")
|
||||
else:
|
||||
print(f"Total abbreviations: {len(abbrevs)}")
|
||||
print()
|
||||
|
||||
for table in range(3):
|
||||
print(f"--- Table {table} ---")
|
||||
for index in range(32):
|
||||
key = (table, index)
|
||||
if key in abbrevs:
|
||||
zscii_codes = abbrevs[key]
|
||||
text = string_factory.zscii.get(zscii_codes)
|
||||
# Show ZSCII codes (truncate if too long)
|
||||
if len(zscii_codes) > 20:
|
||||
zscii_display = str(zscii_codes[:20]) + "..."
|
||||
else:
|
||||
zscii_display = str(zscii_codes)
|
||||
# Show text (truncate if too long)
|
||||
if len(text) > 60:
|
||||
text_display = repr(text[:60]) + "..."
|
||||
else:
|
||||
text_display = repr(text)
|
||||
print(f" [{table},{index:2d}] ZSCII={zscii_display}")
|
||||
print(f" Text={text_display}")
|
||||
print()
|
||||
|
||||
# PART 2: Monkey-patch ZStringFactory.get() to trace all string decodes
|
||||
print("=" * 80)
|
||||
print("STRING DECODING TRACE")
|
||||
print("=" * 80)
|
||||
print()
|
||||
print("Format: [opcode args] -> addr")
|
||||
print(" zchars -> zscii -> text")
|
||||
print()
|
||||
|
||||
def traced_get(self, addr):
|
||||
# Get caller information using inspect
|
||||
stack = inspect.stack()
|
||||
caller_name = "unknown"
|
||||
caller_info = ""
|
||||
|
||||
# Walk up the stack to find the opcode function
|
||||
for frame_info in stack[1:]: # Skip our own frame
|
||||
func_name = frame_info.function
|
||||
if func_name.startswith("op_"):
|
||||
caller_name = func_name
|
||||
# Try to extract arguments from the frame
|
||||
frame_locals = frame_info.frame.f_locals
|
||||
# For op_print_addr, show the addr argument
|
||||
if func_name == "op_print_addr" and "addr" in frame_locals:
|
||||
caller_info = f" addr={frame_locals['addr']:#06x}"
|
||||
# For op_print_paddr, show packed address and conversion
|
||||
elif func_name == "op_print_paddr" and "string_paddr" in frame_locals:
|
||||
paddr = frame_locals["string_paddr"]
|
||||
# Show both the packed address and what it converts to
|
||||
# For z3, byte_addr = paddr * 2
|
||||
byte_addr = paddr * 2
|
||||
caller_info = f" paddr={paddr:#06x} (byte={byte_addr:#06x})"
|
||||
# For op_print_obj, show the object number
|
||||
elif func_name == "op_print_obj" and "obj" in frame_locals:
|
||||
caller_info = f" obj={frame_locals['obj']}"
|
||||
# For op_print, note it's inline (z-string follows opcode)
|
||||
elif func_name == "op_print":
|
||||
caller_info = " (inline z-string)"
|
||||
break
|
||||
|
||||
# Get z-chars from ZStringTranslator
|
||||
zchars = self.zstr.get(addr)
|
||||
|
||||
# Convert z-chars to ZSCII codes
|
||||
zscii_codes = self.zchr.get(zchars)
|
||||
|
||||
# Convert ZSCII to Unicode text
|
||||
text = self.zscii.get(zscii_codes)
|
||||
|
||||
# Log to stderr so it doesn't interfere with game output
|
||||
# Truncate long lists/strings for readability
|
||||
if len(zchars) > 20:
|
||||
zchars_display = str(zchars[:20]) + f"... (len={len(zchars)})"
|
||||
else:
|
||||
zchars_display = str(zchars)
|
||||
|
||||
if len(zscii_codes) > 20:
|
||||
zscii_display = str(zscii_codes[:20]) + f"... (len={len(zscii_codes)})"
|
||||
else:
|
||||
zscii_display = str(zscii_codes)
|
||||
|
||||
text_display = repr(text[:80]) + "..." if len(text) > 80 else repr(text)
|
||||
|
||||
print(f"[{caller_name}{caller_info}] -> {addr:#06x}", file=sys.stderr)
|
||||
print(f" zchars={zchars_display}", file=sys.stderr)
|
||||
print(f" zscii={zscii_display}", file=sys.stderr)
|
||||
print(f" text={text_display}", file=sys.stderr)
|
||||
print(file=sys.stderr)
|
||||
|
||||
return text
|
||||
|
||||
# Apply the monkey patch
|
||||
ZStringFactory.get = traced_get # type: ignore[assignment]
|
||||
|
||||
# PART 3: Run the game with piped input
|
||||
print("=" * 80)
|
||||
print("GAME OUTPUT (running interpreter with tracing enabled)")
|
||||
print("=" * 80)
|
||||
print()
|
||||
print(
|
||||
"Note: String decode traces are written to stderr and "
|
||||
"interleaved with game output."
|
||||
)
|
||||
print("To separate them, run: python3 scripts/debug_zstrings.py 2>trace.log")
|
||||
print()
|
||||
print(
|
||||
"To see just the trace: "
|
||||
"python3 scripts/debug_zstrings.py 2>&1 >/dev/null | grep zchars"
|
||||
)
|
||||
print()
|
||||
print("Press Ctrl+C to stop the game.")
|
||||
print()
|
||||
|
||||
try:
|
||||
zmachine.run()
|
||||
except KeyboardInterrupt:
|
||||
print("\n[Interrupted by user]")
|
||||
except Exception as e:
|
||||
print(f"\n[Game ended: {type(e).__name__}: {e}]")
|
||||
|
||||
print()
|
||||
print("=" * 80)
|
||||
print("DIAGNOSTIC COMPLETE")
|
||||
print("=" * 80)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
@ -368,12 +368,13 @@ def _read_line(original_text=None, terminating_characters=None):
|
|||
|
||||
if char == "\r":
|
||||
char_to_print = "\n"
|
||||
elif char == _BACKSPACE_CHAR:
|
||||
elif char in (_BACKSPACE_CHAR, _DELETE_CHAR):
|
||||
char_to_print = f"{_BACKSPACE_CHAR} {_BACKSPACE_CHAR}"
|
||||
else:
|
||||
char_to_print = char
|
||||
|
||||
sys.stdout.write(char_to_print)
|
||||
sys.stdout.flush()
|
||||
return string
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -251,17 +251,14 @@ class ZCpu:
|
|||
# Fallthrough: No args were equal to a.
|
||||
self._branch(False)
|
||||
|
||||
def op_jl(self, a, *others):
|
||||
"""Branch if the first argument is less than any subsequent
|
||||
argument. Note that the second operand may be absent, in
|
||||
which case there is no jump."""
|
||||
for b in others:
|
||||
if a < b:
|
||||
self._branch(True)
|
||||
return
|
||||
|
||||
# Fallthrough: No args were greater than a.
|
||||
self._branch(False)
|
||||
def op_jl(self, a, b):
|
||||
"""Branch if the first argument is less than the second."""
|
||||
a = self._make_signed(a)
|
||||
b = self._make_signed(b)
|
||||
if a < b:
|
||||
self._branch(True)
|
||||
else:
|
||||
self._branch(False)
|
||||
|
||||
def op_jg(self, a, b):
|
||||
"""Branch if the first argument is greater than the second."""
|
||||
|
|
@ -278,7 +275,7 @@ class ZCpu:
|
|||
val = self._read_variable(variable)
|
||||
val = (val - 1) % 65536
|
||||
self._write_result(val, store_addr=variable)
|
||||
self._branch(val < test_value)
|
||||
self._branch(self._make_signed(val) < self._make_signed(test_value))
|
||||
|
||||
def op_inc_chk(self, variable, test_value):
|
||||
"""Increment the variable, and branch if the value becomes
|
||||
|
|
@ -286,7 +283,7 @@ class ZCpu:
|
|||
val = self._read_variable(variable)
|
||||
val = (val + 1) % 65536
|
||||
self._write_result(val, store_addr=variable)
|
||||
self._branch(val > test_value)
|
||||
self._branch(self._make_signed(val) > self._make_signed(test_value))
|
||||
|
||||
def op_jin(self, obj1, obj2):
|
||||
"""Branch if obj1's parent equals obj2."""
|
||||
|
|
@ -499,7 +496,8 @@ class ZCpu:
|
|||
def op_print_paddr(self, string_paddr):
|
||||
"""Print the string at the given packed address."""
|
||||
zstr_address = self._memory.packed_address(string_paddr)
|
||||
self._ui.screen.write(self._string.get(zstr_address))
|
||||
text = self._string.get(zstr_address)
|
||||
self._ui.screen.write(text)
|
||||
|
||||
def op_load(self, variable):
|
||||
"""Load the value of the given variable and store it."""
|
||||
|
|
@ -530,7 +528,8 @@ class ZCpu:
|
|||
def op_print(self):
|
||||
"""Print the embedded ZString."""
|
||||
zstr_address = self._opdecoder.get_zstring()
|
||||
self._ui.screen.write(self._string.get(zstr_address))
|
||||
text = self._string.get(zstr_address)
|
||||
self._ui.screen.write(text)
|
||||
|
||||
def op_print_ret(self):
|
||||
"""TODO: Write docstring here."""
|
||||
|
|
@ -662,6 +661,7 @@ class ZCpu:
|
|||
max_words = self._memory[parse_buffer_addr]
|
||||
tokens = self._lexer.parse_input(text)
|
||||
num_words = min(len(tokens), max_words)
|
||||
|
||||
self._memory[parse_buffer_addr + 1] = num_words
|
||||
offset = 0
|
||||
for i in range(num_words):
|
||||
|
|
|
|||
|
|
@ -124,9 +124,13 @@ class ZLexer:
|
|||
|
||||
token_list = self._tokenise_string(string, separators)
|
||||
|
||||
# Truncate to dictionary resolution (6 chars V1-3, 9 chars V4+)
|
||||
max_word_len = 6 if self._memory.version <= 3 else 9
|
||||
|
||||
final_list = []
|
||||
for word in token_list:
|
||||
byte_addr = dict.get(word, 0)
|
||||
lookup_key = word[:max_word_len]
|
||||
byte_addr = dict.get(lookup_key, 0)
|
||||
final_list.append([word, byte_addr])
|
||||
|
||||
return final_list
|
||||
|
|
|
|||
|
|
@ -389,15 +389,15 @@ class ZObjectParser:
|
|||
# start at the beginning of the object's proptable
|
||||
addr = self._get_proptable_addr(objectnum)
|
||||
# skip past the shortname of the object
|
||||
addr += 2 * self._memory[addr]
|
||||
addr += 1 + 2 * self._memory[addr]
|
||||
pnum = 0
|
||||
|
||||
if 1 <= self._memory.version <= 3:
|
||||
while self._memory[addr] != 0:
|
||||
bf = BitField(self._memory[addr])
|
||||
addr += 1
|
||||
pnum = bf[4:0]
|
||||
size = bf[7:5] + 1
|
||||
pnum = bf[0:5]
|
||||
size = bf[5:8] + 1
|
||||
if pnum == propnum:
|
||||
return (addr, size)
|
||||
addr += size
|
||||
|
|
@ -406,11 +406,11 @@ class ZObjectParser:
|
|||
while self._memory[addr] != 0:
|
||||
bf = BitField(self._memory[addr])
|
||||
addr += 1
|
||||
pnum = bf[5:0]
|
||||
pnum = bf[0:6]
|
||||
if bf[7]:
|
||||
bf2 = BitField(self._memory[addr])
|
||||
addr += 1
|
||||
size = bf2[5:0]
|
||||
size = bf2[0:6]
|
||||
else:
|
||||
size = 2 if bf[6] else 1
|
||||
if pnum == propnum:
|
||||
|
|
@ -443,8 +443,8 @@ class ZObjectParser:
|
|||
while self._memory[addr] != 0:
|
||||
bf = BitField(self._memory[addr])
|
||||
addr += 1
|
||||
pnum = bf[4:0]
|
||||
size = bf[7:5] + 1
|
||||
pnum = bf[0:5]
|
||||
size = bf[5:8] + 1
|
||||
proplist[pnum] = (addr, size)
|
||||
addr += size
|
||||
|
||||
|
|
@ -508,17 +508,17 @@ class ZObjectParser:
|
|||
# Return first property number
|
||||
addr = self._get_proptable_addr(objectnum)
|
||||
# Skip past the shortname
|
||||
addr += 2 * self._memory[addr]
|
||||
addr += 1 + 2 * self._memory[addr]
|
||||
# Read first property number
|
||||
if self._memory[addr] == 0:
|
||||
return 0
|
||||
|
||||
if 1 <= self._memory.version <= 3:
|
||||
bf = BitField(self._memory[addr])
|
||||
return bf[4:0]
|
||||
return bf[0:5]
|
||||
elif 4 <= self._memory.version <= 5:
|
||||
bf = BitField(self._memory[addr])
|
||||
return bf[5:0]
|
||||
return bf[0:6]
|
||||
else:
|
||||
raise ZObjectIllegalVersion
|
||||
|
||||
|
|
@ -552,15 +552,14 @@ class ZObjectParser:
|
|||
|
||||
if 1 <= self._memory.version <= 3:
|
||||
bf = BitField(self._memory[size_addr])
|
||||
size = bf[7:5] + 1
|
||||
size = bf[5:8] + 1
|
||||
return size
|
||||
|
||||
elif 4 <= self._memory.version <= 5:
|
||||
bf = BitField(self._memory[size_addr])
|
||||
if bf[7]:
|
||||
# Two size bytes, look at second byte
|
||||
bf2 = BitField(self._memory[data_address - 2])
|
||||
size = bf2[5:0]
|
||||
# Two-byte header: size is in bits 0-5 of this byte
|
||||
size = bf[0:6]
|
||||
if size == 0:
|
||||
size = 64
|
||||
return size
|
||||
|
|
|
|||
|
|
@ -100,6 +100,7 @@ class ZStackBottom:
|
|||
stack can treat all frames uniformly without special-case checks for
|
||||
the bottom sentinel.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self.program_counter = 0 # used as a cache only
|
||||
self.stack = []
|
||||
|
|
@ -125,7 +126,7 @@ class ZStackManager:
|
|||
|
||||
current_routine = self._call_stack[-1]
|
||||
|
||||
return current_routine.local_vars[varnum] # type: ignore[possibly-missing-attribute]
|
||||
return current_routine.local_vars[varnum]
|
||||
|
||||
def set_local_variable(self, varnum, value):
|
||||
"""Set value of local variable VARNUM to VALUE in
|
||||
|
|
@ -140,19 +141,19 @@ class ZStackManager:
|
|||
|
||||
current_routine = self._call_stack[-1]
|
||||
|
||||
current_routine.local_vars[varnum] = value # type: ignore[possibly-missing-attribute]
|
||||
current_routine.local_vars[varnum] = value
|
||||
|
||||
def push_stack(self, value):
|
||||
"Push VALUE onto the top of the current routine's data stack."
|
||||
|
||||
current_routine = self._call_stack[-1]
|
||||
current_routine.stack.append(value) # type: ignore[possibly-missing-attribute]
|
||||
current_routine.stack.append(value)
|
||||
|
||||
def pop_stack(self):
|
||||
"Remove and return value from the top of the data stack."
|
||||
|
||||
current_routine = self._call_stack[-1]
|
||||
return current_routine.stack.pop() # type: ignore[possibly-missing-attribute]
|
||||
return current_routine.stack.pop()
|
||||
|
||||
def get_stack_frame_index(self):
|
||||
"Return current stack frame number. For use by 'catch' opcode."
|
||||
|
|
@ -198,9 +199,9 @@ class ZStackManager:
|
|||
if exiting_routine.return_addr == 0: # type: ignore[possibly-missing-attribute]
|
||||
# Push to stack
|
||||
self.push_stack(return_value)
|
||||
elif 0 < exiting_routine.return_addr < 10: # type: ignore[possibly-missing-attribute]
|
||||
elif 0 < exiting_routine.return_addr < 0x10: # type: ignore[possibly-missing-attribute]
|
||||
# Store in local var
|
||||
self.set_local_variable(exiting_routine.return_addr, return_value) # type: ignore[possibly-missing-attribute]
|
||||
self.set_local_variable(exiting_routine.return_addr - 1, return_value) # type: ignore[possibly-missing-attribute]
|
||||
else:
|
||||
# Store in global var
|
||||
self._memory.write_global(exiting_routine.return_addr, return_value) # type: ignore[possibly-missing-attribute]
|
||||
|
|
|
|||
|
|
@ -38,6 +38,11 @@ class MockMemory:
|
|||
def write_global(self, varnum, value):
|
||||
self.globals[varnum] = value
|
||||
|
||||
def generate_checksum(self):
|
||||
"""Generate checksum from 0x40 onwards, modulo 0x10000."""
|
||||
total = sum(self.data[0x40:])
|
||||
return total % 0x10000
|
||||
|
||||
|
||||
class MockStackManager:
|
||||
"""Mock stack manager for testing."""
|
||||
|
|
@ -291,6 +296,90 @@ class ZMachineOpcodeTests(TestCase):
|
|||
# Should just not raise an exception
|
||||
self.cpu.op_show_status()
|
||||
|
||||
def test_op_test_all_flags_set(self):
|
||||
"""Test op_test branches when all flags are set in bitmap."""
|
||||
self.decoder.branch_condition = True
|
||||
self.decoder.branch_offset = 10
|
||||
old_pc = self.cpu._opdecoder.program_counter
|
||||
|
||||
# bitmap 0b11010110, flags 0b10010100 - all flags present
|
||||
self.cpu.op_test(0b11010110, 0b10010100)
|
||||
|
||||
# Should branch (offset - 2)
|
||||
self.assertEqual(self.cpu._opdecoder.program_counter, old_pc + 8)
|
||||
|
||||
def test_op_test_some_flags_missing(self):
|
||||
"""Test op_test doesn't branch when some flags are missing."""
|
||||
self.decoder.branch_condition = True
|
||||
old_pc = self.cpu._opdecoder.program_counter
|
||||
|
||||
# bitmap 0b11010110, flags 0b10011100 - bit 3 missing
|
||||
self.cpu.op_test(0b11010110, 0b10011100)
|
||||
|
||||
# Should not branch
|
||||
self.assertEqual(self.cpu._opdecoder.program_counter, old_pc)
|
||||
|
||||
def test_op_test_zero_flags(self):
|
||||
"""Test op_test with zero flags always branches (all 0 flags set)."""
|
||||
self.decoder.branch_condition = True
|
||||
self.decoder.branch_offset = 15
|
||||
old_pc = self.cpu._opdecoder.program_counter
|
||||
|
||||
# Any bitmap with flags=0 should pass (0 & 0 == 0)
|
||||
self.cpu.op_test(0b11111111, 0)
|
||||
|
||||
# Should branch
|
||||
self.assertEqual(self.cpu._opdecoder.program_counter, old_pc + 13)
|
||||
|
||||
def test_op_test_identical(self):
|
||||
"""Test op_test branches when bitmap and flags are identical."""
|
||||
self.decoder.branch_condition = True
|
||||
self.decoder.branch_offset = 20
|
||||
old_pc = self.cpu._opdecoder.program_counter
|
||||
|
||||
# Identical bitmap and flags
|
||||
self.cpu.op_test(0b10101010, 0b10101010)
|
||||
|
||||
# Should branch
|
||||
self.assertEqual(self.cpu._opdecoder.program_counter, old_pc + 18)
|
||||
|
||||
def test_op_verify_matching_checksum(self):
|
||||
"""Test op_verify branches when checksum matches."""
|
||||
# Set expected checksum at 0x1C
|
||||
self.memory.write_word(0x1C, 0x1234)
|
||||
# Set data that produces matching checksum
|
||||
# checksum = sum(data[0x40:]) % 0x10000
|
||||
# For simplicity, set one byte to produce desired checksum
|
||||
self.memory[0x40] = 0x34
|
||||
self.memory[0x41] = 0x12
|
||||
# Sum = 0x34 + 0x12 = 0x46, need 0x1234
|
||||
# Set more bytes: 0x1234 - 0x46 = 0x11EE
|
||||
for i in range(0x42, 0x42 + 0x11EE):
|
||||
self.memory[i] = 1 if i < 0x42 + 0x11EE else 0
|
||||
|
||||
self.decoder.branch_condition = True
|
||||
self.decoder.branch_offset = 25
|
||||
old_pc = self.cpu._opdecoder.program_counter
|
||||
|
||||
self.cpu.op_verify()
|
||||
|
||||
# Should branch
|
||||
self.assertEqual(self.cpu._opdecoder.program_counter, old_pc + 23)
|
||||
|
||||
def test_op_verify_mismatched_checksum(self):
|
||||
"""Test op_verify doesn't branch when checksum doesn't match."""
|
||||
# Set expected checksum at 0x1C
|
||||
self.memory.write_word(0x1C, 0x5678)
|
||||
# Memory data will produce different checksum (mostly zeros)
|
||||
|
||||
self.decoder.branch_condition = True
|
||||
old_pc = self.cpu._opdecoder.program_counter
|
||||
|
||||
self.cpu.op_verify()
|
||||
|
||||
# Should not branch (checksums don't match)
|
||||
self.assertEqual(self.cpu._opdecoder.program_counter, old_pc)
|
||||
|
||||
|
||||
class MockObjectParser:
|
||||
"""Mock object parser for testing CPU opcodes."""
|
||||
|
|
@ -319,6 +408,9 @@ class MockObjectParser:
|
|||
def get_sibling(self, objnum):
|
||||
return self.siblings.get(objnum, 0)
|
||||
|
||||
def get_child(self, objnum):
|
||||
return self.children.get(objnum, 0)
|
||||
|
||||
def remove_object(self, objnum):
|
||||
# Simple implementation - just clear parent
|
||||
self.parents[objnum] = 0
|
||||
|
|
@ -519,6 +611,35 @@ class ZMachineObjectOpcodeTests(TestCase):
|
|||
# Should store 0
|
||||
self.assertEqual(self.stack.pop_stack(), 0)
|
||||
|
||||
def test_op_get_child_with_child(self):
|
||||
"""Test get_child stores child and branches if nonzero."""
|
||||
self.objects.children[10] = 5
|
||||
self.decoder.store_address = 0
|
||||
self.decoder.branch_condition = True
|
||||
self.decoder.branch_offset = 12
|
||||
old_pc = self.cpu._opdecoder.program_counter
|
||||
|
||||
self.cpu.op_get_child(10)
|
||||
|
||||
# Should store 5
|
||||
self.assertEqual(self.stack.pop_stack(), 5)
|
||||
# Should branch (offset - 2)
|
||||
self.assertEqual(self.cpu._opdecoder.program_counter, old_pc + 10)
|
||||
|
||||
def test_op_get_child_no_child(self):
|
||||
"""Test get_child with no child doesn't branch."""
|
||||
self.objects.children[10] = 0
|
||||
self.decoder.store_address = 0
|
||||
self.decoder.branch_condition = True
|
||||
old_pc = self.cpu._opdecoder.program_counter
|
||||
|
||||
self.cpu.op_get_child(10)
|
||||
|
||||
# Should store 0
|
||||
self.assertEqual(self.stack.pop_stack(), 0)
|
||||
# Should not branch
|
||||
self.assertEqual(self.cpu._opdecoder.program_counter, old_pc)
|
||||
|
||||
|
||||
class ZMachineComplexOpcodeTests(TestCase):
|
||||
"""Test suite for complex Z-machine opcodes (input, save/restore, restart)."""
|
||||
|
|
|
|||
Loading…
Reference in a new issue