Add docs
This commit is contained in:
parent
cb73ceb2d3
commit
3cf16586aa
GPG key ID:
22B5C6D66A38B06C
5 changed files with 680 additions and 0 deletions
57
docs/cola.txt
Normal file
57
docs/cola.txt
Normal file
|
|
@ -0,0 +1,57 @@
|
||||||
|
Cola - Text CRDT for Real-Time Collaborative Editing
|
||||||
|
=====================================================
|
||||||
|
|
||||||
|
https://github.com/nomad/cola
|
||||||
|
|
||||||
|
What is it?
|
||||||
|
A Rust library implementing a Conflict-free Replicated Data Type (CRDT)
|
||||||
|
specifically designed for collaborative text editing. Allows multiple peers
|
||||||
|
to edit the same document concurrently without a central server.
|
||||||
|
|
||||||
|
|
||||||
|
Why it's interesting
|
||||||
|
--------------------
|
||||||
|
- Peer-to-peer: no server needed, peers sync directly
|
||||||
|
- Convergence guaranteed: all replicas eventually reach same state
|
||||||
|
- Designed for text: not a generic CRDT, optimized for editing operations
|
||||||
|
- Rust: fast, safe, could compile to WASM for browser or FFI for other langs
|
||||||
|
|
||||||
|
|
||||||
|
How CRDTs work (simplified)
|
||||||
|
---------------------------
|
||||||
|
Instead of "insert char at position 5", operations are like "insert char
|
||||||
|
after unique-id-xyz". Each character gets a unique ID based on who inserted
|
||||||
|
it and when. This means concurrent edits never conflict - they just get
|
||||||
|
ordered deterministically.
|
||||||
|
|
||||||
|
|
||||||
|
Potential uses
|
||||||
|
--------------
|
||||||
|
- Build an editor-agnostic collab layer
|
||||||
|
- Terminal multiplexer with shared buffers
|
||||||
|
- Plugin backend for vim/emacs/helix
|
||||||
|
- Pair with a simple transport (WebRTC, TCP, WebSocket)
|
||||||
|
|
||||||
|
|
||||||
|
To explore
|
||||||
|
----------
|
||||||
|
1. Clone the repo, run the examples
|
||||||
|
2. Look at the Replica and Insertion types
|
||||||
|
3. See how edits are encoded and merged
|
||||||
|
4. Think about what transport layer you'd use
|
||||||
|
5. Consider: could this power a "collab daemon" that editors connect to?
|
||||||
|
|
||||||
|
|
||||||
|
Related projects
|
||||||
|
----------------
|
||||||
|
- Automerge: more general CRDT, bigger community
|
||||||
|
- Yjs: JavaScript CRDT, powers many web editors
|
||||||
|
- diamond-types: another Rust text CRDT, by the Automerge folks
|
||||||
|
|
||||||
|
|
||||||
|
Links
|
||||||
|
-----
|
||||||
|
Repo: https://github.com/nomad/cola
|
||||||
|
CRDTs: https://crdt.tech
|
||||||
|
Automerge: https://automerge.org
|
||||||
|
Yjs: https://yjs.dev
|
||||||
125
docs/notes.txt
Normal file
125
docs/notes.txt
Normal file
|
|
@ -0,0 +1,125 @@
|
||||||
|
CLI Collaborative Editing Research
|
||||||
|
===================================
|
||||||
|
|
||||||
|
The problem: Zed/VSCode have great collab features. What about terminal folks
|
||||||
|
who want to use vim/emacs/whatever but still pair/mob program in real-time?
|
||||||
|
|
||||||
|
|
||||||
|
HOW THE BIG PLAYERS DO IT
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
See detailed breakdowns:
|
||||||
|
- vscode-liveshare.txt (host-guest model, SSH relay, no CRDT)
|
||||||
|
- zed-collab.txt (true CRDT, anchors, tombstones, SumTree)
|
||||||
|
|
||||||
|
Quick comparison:
|
||||||
|
|
||||||
|
VSCode Live Share:
|
||||||
|
- Host-guest model (not true P2P)
|
||||||
|
- All content stays on host machine
|
||||||
|
- SSH tunnel (P2P or via Microsoft relay)
|
||||||
|
- No conflict resolution needed - only one source of truth
|
||||||
|
- Simpler but dependent on host connection
|
||||||
|
|
||||||
|
Zed:
|
||||||
|
- True CRDT - every replica is equal
|
||||||
|
- Anchors instead of offsets (insertion_id + offset)
|
||||||
|
- Tombstone deletions with version vectors
|
||||||
|
- Lamport timestamps for ordering concurrent edits
|
||||||
|
- Per-user undo via undo map
|
||||||
|
- SumTree (copy-on-write B+ tree) everywhere
|
||||||
|
|
||||||
|
Key insight:
|
||||||
|
VSCode = "remote desktop for code"
|
||||||
|
Zed = "Google Docs for code"
|
||||||
|
|
||||||
|
For CLI collab, the Zed approach is more interesting because it's
|
||||||
|
truly decentralized and doesn't require a persistent host.
|
||||||
|
|
||||||
|
|
||||||
|
NEOVIM-SPECIFIC
|
||||||
|
---------------
|
||||||
|
|
||||||
|
instant.nvim
|
||||||
|
https://github.com/jbyuki/instant.nvim
|
||||||
|
- Pure Lua, no dependencies, CRDT-based
|
||||||
|
- Run a server, others connect, real-time sync
|
||||||
|
- Virtual cursors show where others are editing
|
||||||
|
- Can share single buffer or entire session
|
||||||
|
- Built-in localhost server, default port 8080
|
||||||
|
- Commands: :InstantStartSingle, :InstantJoinSingle, :InstantStartSession
|
||||||
|
- Separate undo/redo per user
|
||||||
|
- This is probably the closest to Zed collab for terminal users
|
||||||
|
|
||||||
|
live-share.nvim
|
||||||
|
https://github.com/azratul/live-share.nvim
|
||||||
|
https://dev.to/azratul/live-sharenvim-real-time-collaboration-for-neovim-1kn2
|
||||||
|
- Builds on instant.nvim with nicer UX
|
||||||
|
- Still actively developed
|
||||||
|
|
||||||
|
|
||||||
|
TERMINAL SHARING (any editor)
|
||||||
|
-----------------------------
|
||||||
|
|
||||||
|
Upterm
|
||||||
|
https://github.com/owenthereal/upterm
|
||||||
|
https://upterm.dev
|
||||||
|
- Modern tmate alternative, written in Go
|
||||||
|
- NOT a tmux fork so you keep your tmux config
|
||||||
|
- GitHub/GitLab/SourceHut/Codeberg auth
|
||||||
|
- Community server: uptermd.upterm.dev
|
||||||
|
- Supports scp/sftp file transfer
|
||||||
|
- WebSocket fallback when SSH blocked
|
||||||
|
- Can integrate with GitHub Actions for SSH debugging
|
||||||
|
|
||||||
|
tmate
|
||||||
|
https://tmate.io
|
||||||
|
- Fork of tmux 2.x, shares terminal sessions
|
||||||
|
- Simple but stuck on old tmux, config conflicts
|
||||||
|
|
||||||
|
bottlerocketlabs/pair
|
||||||
|
https://github.com/bottlerocketlabs/pair
|
||||||
|
- Wrapper around tmux for quick pairing
|
||||||
|
- Good for vim/emacs users
|
||||||
|
|
||||||
|
|
||||||
|
CRDT LIBRARIES (build your own)
|
||||||
|
-------------------------------
|
||||||
|
|
||||||
|
Cola (Rust)
|
||||||
|
https://github.com/nomad/cola
|
||||||
|
- Text CRDT for real-time collaborative editing
|
||||||
|
- Peer-to-peer, no central server required
|
||||||
|
- Could theoretically power an editor-agnostic collab layer
|
||||||
|
- See: cola.txt in this dir for deep dive
|
||||||
|
|
||||||
|
Automerge
|
||||||
|
https://automerge.org
|
||||||
|
- More general CRDT library
|
||||||
|
- Has bindings for many languages
|
||||||
|
|
||||||
|
|
||||||
|
THE MISSING PIECE
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
Nobody has built the "any editor" dream yet. Would need:
|
||||||
|
1. Shared CRDT document layer (cola/automerge)
|
||||||
|
2. LSP forwarding to share language intelligence
|
||||||
|
3. Thin clients for each editor connecting to shared state
|
||||||
|
|
||||||
|
This could be a fun project to explore.
|
||||||
|
|
||||||
|
|
||||||
|
QUICK START
|
||||||
|
-----------
|
||||||
|
|
||||||
|
To try instant.nvim:
|
||||||
|
1. Install the plugin
|
||||||
|
2. One person runs :InstantStartServer 0.0.0.0 8080
|
||||||
|
3. Same person runs :InstantStartSession [ip] 8080
|
||||||
|
4. Others run :InstantJoinSession [ip] 8080
|
||||||
|
|
||||||
|
To try Upterm:
|
||||||
|
1. brew install owenthereal/upterm/upterm (or build from source)
|
||||||
|
2. upterm host -- tmux new -s shared
|
||||||
|
3. Share the SSH connection string with your pair
|
||||||
217
docs/synthesis.txt
Normal file
217
docs/synthesis.txt
Normal file
|
|
@ -0,0 +1,217 @@
|
||||||
|
Research Synthesis - Editor-Agnostic CLI Collaboration
|
||||||
|
|
||||||
|
THE CORE PROBLEM:
|
||||||
|
|
||||||
|
Zed and VSCode have beautiful real-time collaboration. But they lock you into their editors. If you're a vim/helix/kakoune user and want to pair program with a friend, you shouldn't have to make them switch editors. The goal: divorce collaborative editing from any specific editor.
|
||||||
|
|
||||||
|
EXISTING APPROACHES ANALYZED:
|
||||||
|
|
||||||
|
1. Terminal Multiplexing (upterm, tmate, tmux sharing)
|
||||||
|
|
||||||
|
How it works: Share a PTY over the network. Everyone sees the same terminal output, keystrokes forwarded to the shell.
|
||||||
|
|
||||||
|
Upterm specifically: Reverse SSH tunnel to a central server, clients connect through it. MultiWriter pattern broadcasts output to all connected clients.
|
||||||
|
|
||||||
|
Pros: Works TODAY with any CLI editor. Zero editor integration needed. Good for "let me show you something" pair programming.
|
||||||
|
|
||||||
|
Cons: No concurrent editing (everyone's typing goes to same shell). No offline. No semantic awareness. Last keystroke wins. Not true collaborative editing.
|
||||||
|
|
||||||
|
Verdict: Great for terminal screenshare, not for document collaboration.
|
||||||
|
|
||||||
|
2. File-Level Sync (VSCode LiveShare style)
|
||||||
|
|
||||||
|
How it works: Host owns the workspace. Guests get proxied file access. SSH protocol with relay fallback.
|
||||||
|
|
||||||
|
Not actually CRDT-based - more like remote desktop for code.
|
||||||
|
|
||||||
|
Sessions expire after 24 hours. P2P when possible, Microsoft relay otherwise.
|
||||||
|
|
||||||
|
Verdict: Doesn't solve editor-agnostic problem. Guests are still locked to host's environment.
|
||||||
|
|
||||||
|
3. CRDT-Based Document Sync (Zed, instant.nvim)
|
||||||
|
|
||||||
|
How it works: Each character gets a unique ID. Operations are "insert after ID xyz" not "insert at position 5". Concurrent edits automatically merge correctly.
|
||||||
|
|
||||||
|
Zed's architecture: Anchors (logical positions), tombstone deletions, Lamport timestamps, version vectors, per-user undo maps. Server for auth/discovery, CRDT for document state.
|
||||||
|
|
||||||
|
instant.nvim: Pure Lua implementation for Neovim. WebSocket server routes messages. Position IDs (tombstone vector clocks) for conflict-free ordering.
|
||||||
|
|
||||||
|
Key insight from instant.nvim: 70% of the code is editor-agnostic (transport + CRDT algorithm). Only 30% is neovim-specific (buffer events, manipulation, cursor display).
|
||||||
|
|
||||||
|
THE PROPOSED ARCHITECTURE:
|
||||||
|
|
||||||
|
CRDT Daemon + Thin Editor Adapters
|
||||||
|
|
||||||
|
The daemon handles all the hard parts:
|
||||||
|
- CRDT text buffer (using cola or diamond-types)
|
||||||
|
- Network sync (WebSocket for remote, Unix socket for local)
|
||||||
|
- Session management
|
||||||
|
- Peer discovery/auth
|
||||||
|
|
||||||
|
Each editor gets a minimal adapter that:
|
||||||
|
1. Hooks into buffer change events
|
||||||
|
2. Serializes changes as (offset, length, text)
|
||||||
|
3. Sends to daemon
|
||||||
|
4. Receives remote operations from daemon
|
||||||
|
5. Applies changes to local buffer
|
||||||
|
6. Optionally: displays peer cursors
|
||||||
|
|
||||||
|
Why this split works:
|
||||||
|
- Solving CRDT correctly is hard. Do it once in the daemon.
|
||||||
|
- Each editor's adapter is simple. Just event hooks and buffer manipulation.
|
||||||
|
- Adding new editors is cheap. Write a small plugin, done.
|
||||||
|
- Multiple different editors can collaborate simultaneously.
|
||||||
|
|
||||||
|
THE EDITOR ADAPTER REQUIREMENTS:
|
||||||
|
|
||||||
|
For any CLI editor to participate, the adapter needs:
|
||||||
|
|
||||||
|
1. Change event hook - Know when user edits the buffer
|
||||||
|
- Neovim: nvim_buf_attach with on_lines callback
|
||||||
|
- Helix: LSP-based or custom events
|
||||||
|
- Kakoune: FIFO-based extension system
|
||||||
|
- Vim: +clientserver or plugin
|
||||||
|
|
||||||
|
2. Buffer manipulation - Apply remote changes
|
||||||
|
- Neovim: nvim_buf_set_lines
|
||||||
|
- Others: Similar APIs exist
|
||||||
|
|
||||||
|
3. Cursor visualization (optional but nice) - Show where peers are editing
|
||||||
|
- Neovim: nvim_buf_set_extmark with virtual text
|
||||||
|
- Others: Editor-specific
|
||||||
|
|
||||||
|
THE LSP ANGLE:
|
||||||
|
|
||||||
|
Many CLI editors already speak LSP (Language Server Protocol). This is interesting because:
|
||||||
|
|
||||||
|
- textDocument/didChange already notifies of edits
|
||||||
|
- textDocument/didOpen and didClose handle lifecycle
|
||||||
|
- workspace/executeCommand can carry custom operations
|
||||||
|
|
||||||
|
A "collaboration language server" could:
|
||||||
|
1. Receive didChange notifications
|
||||||
|
2. Run them through CRDT
|
||||||
|
3. Push remote changes back via workspace edits
|
||||||
|
|
||||||
|
This would reduce per-editor work to almost zero - editors already have LSP clients. Worth exploring.
|
||||||
|
|
||||||
|
CRDT LIBRARY CHOICE:
|
||||||
|
|
||||||
|
Cola (https://github.com/nomad/cola):
|
||||||
|
- Operation-based CRDT for text
|
||||||
|
- Buffer-agnostic: doesn't store text, just manages coordinates
|
||||||
|
- Clean API: Replica, Insertion, Deletion
|
||||||
|
- Real-time P2P focus
|
||||||
|
- Serialization via serde or custom encode
|
||||||
|
- Handles out-of-order delivery via backlog
|
||||||
|
- Benchmarks show 1.4-2x faster than diamond-types in some cases
|
||||||
|
|
||||||
|
Diamond-types (https://github.com/josephg/diamond-types):
|
||||||
|
- "World's fastest CRDT"
|
||||||
|
- 5000x-80000x speedup through aggressive RLE
|
||||||
|
- Stores full history (temporal DAG + spatial state)
|
||||||
|
- More complex (OpLog, Branch, CausalGraph concepts)
|
||||||
|
- Great for: large documents, offline-first, audit trails
|
||||||
|
- WASM support for browser
|
||||||
|
|
||||||
|
For our use case: Cola wins.
|
||||||
|
- Simpler API, easier to integrate
|
||||||
|
- Real-time focus matches our needs
|
||||||
|
- We don't need full history storage
|
||||||
|
- Less cognitive overhead to work with
|
||||||
|
|
||||||
|
Diamond-types is overkill for initial prototyping. Could revisit for optimization later.
|
||||||
|
|
||||||
|
COMMUNICATION PROTOCOL OPTIONS:
|
||||||
|
|
||||||
|
1. Unix socket - Simple, local only. Good for same-machine testing.
|
||||||
|
|
||||||
|
2. WebSocket - Works remote. Browser-friendly if we ever want web UI. Good default.
|
||||||
|
|
||||||
|
3. stdio pipe - Simplest for CLI tools. Editor spawns daemon, communicates via stdin/stdout.
|
||||||
|
|
||||||
|
4. LSP protocol - Leverage existing infrastructure. Interesting but might be awkward fit.
|
||||||
|
|
||||||
|
Recommendation: WebSocket as primary (works local and remote), Unix socket as fast local alternative.
|
||||||
|
|
||||||
|
REFERENCE IMPLEMENTATIONS:
|
||||||
|
|
||||||
|
repos/cola/
|
||||||
|
- src/replica.rs: Main API, 1200+ lines of docs
|
||||||
|
- src/insertion.rs, deletion.rs: Operation types
|
||||||
|
- examples/basic.rs: Simple Document wrapper pattern
|
||||||
|
- Key pattern: editor maintains buffer + Replica, calls inserted/deleted for local ops, integrate_* for remote ops
|
||||||
|
|
||||||
|
repos/instant.nvim/
|
||||||
|
- lua/instant.lua: Main logic, mixed nvim + algorithm
|
||||||
|
- lua/instant/websocket_*.lua: Transport layer (portable)
|
||||||
|
- Position ID generation (genPID): Tombstone vector clocks
|
||||||
|
- Shows exactly what adapters need to do
|
||||||
|
|
||||||
|
repos/upterm/
|
||||||
|
- host/host.go: Session lifecycle
|
||||||
|
- io/writer.go: MultiWriter for output broadcast
|
||||||
|
- Different paradigm but useful for understanding terminal collaboration UX
|
||||||
|
|
||||||
|
repos/diamond-types/
|
||||||
|
- Complex internals, good for understanding CRDT optimization
|
||||||
|
- INTERNALS.md, BINARY.md explain the RLE approach
|
||||||
|
|
||||||
|
NEXT STEPS TO PROTOTYPE:
|
||||||
|
|
||||||
|
Phase 1: Minimal daemon
|
||||||
|
- Rust binary using cola
|
||||||
|
- Single document support
|
||||||
|
- WebSocket server
|
||||||
|
- Two clients can connect, edits sync
|
||||||
|
|
||||||
|
Phase 2: Neovim adapter
|
||||||
|
- Lua plugin
|
||||||
|
- Connects to daemon via WebSocket
|
||||||
|
- Hooks nvim_buf_attach for changes
|
||||||
|
- Applies remote changes via nvim_buf_set_lines
|
||||||
|
- Test: two neovim instances editing same file
|
||||||
|
|
||||||
|
Phase 3: Multi-document
|
||||||
|
- Session management
|
||||||
|
- File path mapping
|
||||||
|
- Join/leave notifications
|
||||||
|
|
||||||
|
Phase 4: Second editor
|
||||||
|
- Helix adapter (or kakoune, or vim)
|
||||||
|
- Prove the architecture works across editors
|
||||||
|
|
||||||
|
Phase 5: Polish
|
||||||
|
- Peer cursors
|
||||||
|
- User presence indicators
|
||||||
|
- Better auth (SSH keys, GitHub)
|
||||||
|
- Discovery service
|
||||||
|
|
||||||
|
OPEN QUESTIONS:
|
||||||
|
|
||||||
|
1. Where does the daemon run?
|
||||||
|
- Local daemon per machine? Central server? Hybrid?
|
||||||
|
- For local-first: daemon on each machine, P2P sync
|
||||||
|
- For easy setup: central server handles routing
|
||||||
|
|
||||||
|
2. How to handle file paths?
|
||||||
|
- Relative to project root? Absolute? UUID-based?
|
||||||
|
- Need consistent naming across different machines
|
||||||
|
|
||||||
|
3. Undo/redo coordination?
|
||||||
|
- Per-user undo (like Zed) or global?
|
||||||
|
- Cola doesn't handle this - need to build on top
|
||||||
|
|
||||||
|
4. Cursor/selection sync?
|
||||||
|
- Nice to have, not essential for MVP
|
||||||
|
- Adds complexity (need to track peer positions)
|
||||||
|
|
||||||
|
5. Permissions?
|
||||||
|
- Can anyone edit anything? Read-only viewers?
|
||||||
|
- Future concern, not MVP
|
||||||
|
|
||||||
|
THE DREAM:
|
||||||
|
|
||||||
|
You're in helix. Friend is in neovim. Another friend is in kakoune. You all open the same project, connect to a session, and just... edit together. Changes flow seamlessly. Each person uses their preferred editor with their preferred config. No one had to install anything they don't normally use.
|
||||||
|
|
||||||
|
That's the goal.
|
||||||
106
docs/vscode-liveshare.txt
Normal file
106
docs/vscode-liveshare.txt
Normal file
|
|
@ -0,0 +1,106 @@
|
||||||
|
VSCode Live Share - Technical Architecture
|
||||||
|
==========================================
|
||||||
|
|
||||||
|
https://learn.microsoft.com/en-us/visualstudio/liveshare/
|
||||||
|
|
||||||
|
|
||||||
|
ARCHITECTURE MODEL
|
||||||
|
------------------
|
||||||
|
|
||||||
|
Host-Guest model, NOT peer-to-peer CRDT:
|
||||||
|
- One host owns the workspace
|
||||||
|
- Guests connect to host's machine
|
||||||
|
- All content stays on host, never synced to cloud or guest machines
|
||||||
|
- Sessions expire after 24 hours
|
||||||
|
|
||||||
|
Connection flow:
|
||||||
|
1. Host starts session, gets unique URL
|
||||||
|
2. Guests join via URL
|
||||||
|
3. Live Share attempts P2P connection first
|
||||||
|
4. Falls back to Microsoft cloud relay if P2P fails (firewalls/NATs)
|
||||||
|
5. Some guests can be P2P while others relay in same session
|
||||||
|
|
||||||
|
|
||||||
|
SYNCHRONIZATION
|
||||||
|
---------------
|
||||||
|
|
||||||
|
NOT using CRDTs - this is a remote workspace model:
|
||||||
|
- File system level sync, not document-level CRDT
|
||||||
|
- Host's LSP, terminals, debuggers are shared
|
||||||
|
- Guests get proxied access to host's environment
|
||||||
|
- More like "remote desktop for code" than true collaborative editing
|
||||||
|
|
||||||
|
Why this matters:
|
||||||
|
- Simpler to implement (no conflict resolution needed)
|
||||||
|
- But requires constant connection to host
|
||||||
|
- If host disconnects, session ends
|
||||||
|
- Latency depends on connection to host
|
||||||
|
|
||||||
|
|
||||||
|
PROTOCOL & SECURITY
|
||||||
|
-------------------
|
||||||
|
|
||||||
|
Transport:
|
||||||
|
- SSH protocol for all data
|
||||||
|
- P2P: direct SSH connection (ports 5990-5999)
|
||||||
|
- Relay: SSH over TLS-encrypted WebSockets
|
||||||
|
|
||||||
|
Encryption:
|
||||||
|
- Diffie-Hellman key exchange for shared secret
|
||||||
|
- AES symmetric encryption derived from shared secret
|
||||||
|
- Keys rotated periodically during session
|
||||||
|
- Keys only in memory, never persisted
|
||||||
|
|
||||||
|
Authentication:
|
||||||
|
- JWT tokens signed by Live Share service
|
||||||
|
- Claims include user identity (MSA/AAD/GitHub)
|
||||||
|
- Session-specific RSA keypair generated by host
|
||||||
|
- Private key never leaves host memory
|
||||||
|
|
||||||
|
|
||||||
|
RELAY SERVICE
|
||||||
|
-------------
|
||||||
|
|
||||||
|
Microsoft's cloud relay:
|
||||||
|
- Only used when P2P fails
|
||||||
|
- Does NOT store or inspect content
|
||||||
|
- Just routes encrypted SSH packets
|
||||||
|
- End-to-end encryption means relay can't read traffic
|
||||||
|
|
||||||
|
Enterprise option:
|
||||||
|
- Private relay servers possible
|
||||||
|
- Requires additional infrastructure
|
||||||
|
|
||||||
|
|
||||||
|
WHAT GETS SHARED
|
||||||
|
----------------
|
||||||
|
|
||||||
|
- File system (read/write based on permissions)
|
||||||
|
- Language services (IntelliSense, go-to-definition)
|
||||||
|
- Debugging sessions
|
||||||
|
- Terminal instances (optional, read-only or read-write)
|
||||||
|
- Localhost servers (port forwarding)
|
||||||
|
- Cursor positions and selections
|
||||||
|
|
||||||
|
|
||||||
|
IMPLICATIONS FOR CLI COLLAB
|
||||||
|
---------------------------
|
||||||
|
|
||||||
|
Live Share's approach could work for terminal editors:
|
||||||
|
1. One person hosts their tmux/vim session
|
||||||
|
2. Others connect via relay or P2P
|
||||||
|
3. All editing happens on host machine
|
||||||
|
4. No conflict resolution needed
|
||||||
|
|
||||||
|
But:
|
||||||
|
- Not truly decentralized
|
||||||
|
- Dependent on host's connection
|
||||||
|
- Less elegant than CRDT approach
|
||||||
|
|
||||||
|
|
||||||
|
LINKS
|
||||||
|
-----
|
||||||
|
|
||||||
|
Docs: https://learn.microsoft.com/en-us/visualstudio/liveshare/
|
||||||
|
Security: https://learn.microsoft.com/en-us/visualstudio/liveshare/reference/security
|
||||||
|
Connect: https://learn.microsoft.com/en-us/visualstudio/liveshare/reference/connectivity
|
||||||
175
docs/zed-collab.txt
Normal file
175
docs/zed-collab.txt
Normal file
|
|
@ -0,0 +1,175 @@
|
||||||
|
Zed Editor - Collaboration Architecture
|
||||||
|
=======================================
|
||||||
|
|
||||||
|
https://zed.dev/docs/collaboration/overview
|
||||||
|
https://zed.dev/blog/crdts
|
||||||
|
|
||||||
|
|
||||||
|
PHILOSOPHY
|
||||||
|
----------
|
||||||
|
|
||||||
|
Collaboration is "part of Zed's DNA" - not bolted on.
|
||||||
|
Built from ground up with multiplayer in mind.
|
||||||
|
Uses CRDTs instead of Operational Transformation.
|
||||||
|
|
||||||
|
|
||||||
|
WHY CRDT OVER OT
|
||||||
|
----------------
|
||||||
|
|
||||||
|
Operational Transformation (OT):
|
||||||
|
- Transform concurrent operations to apply in different orders
|
||||||
|
- Requires central server to sequence operations
|
||||||
|
- Complex correctness proofs
|
||||||
|
- What Google Docs uses
|
||||||
|
|
||||||
|
CRDT approach:
|
||||||
|
- Structure data so operations are inherently commutative
|
||||||
|
- No transformation needed - apply directly on any replica
|
||||||
|
- Express edits in terms of logical locations, not absolute offsets
|
||||||
|
- Decentralized by nature
|
||||||
|
|
||||||
|
|
||||||
|
CRDT IMPLEMENTATION DETAILS
|
||||||
|
---------------------------
|
||||||
|
|
||||||
|
Anchors (logical positions):
|
||||||
|
- Each position is (insertion_id, offset) pair
|
||||||
|
- insertion_id = replica_id + sequence_number
|
||||||
|
- Replicas get unique IDs from server, then generate IDs locally
|
||||||
|
- No collision risk for concurrent operations
|
||||||
|
|
||||||
|
Fragments:
|
||||||
|
- Text organized into fragments
|
||||||
|
- Each fragment knows its insertion ID and offset
|
||||||
|
- Remote operations can find insertion points regardless of local changes
|
||||||
|
|
||||||
|
Immutable insertions:
|
||||||
|
- Every piece of inserted text is immutable forever
|
||||||
|
- Edits mark deletions, they don't remove content
|
||||||
|
- This is key to conflict-free merging
|
||||||
|
|
||||||
|
|
||||||
|
DELETION HANDLING
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
Tombstones:
|
||||||
|
- Deleted text gets tombstone marker
|
||||||
|
- Text is hidden, not removed
|
||||||
|
- Allows insertions within deleted ranges to resolve correctly
|
||||||
|
- "the deleted text is merely hidden rather than actually thrown away"
|
||||||
|
|
||||||
|
Version vectors:
|
||||||
|
- Each deletion has version vector
|
||||||
|
- Encodes "latest observed sequence number for each replica"
|
||||||
|
- Prevents concurrent insertions from being incorrectly tombstoned
|
||||||
|
|
||||||
|
|
||||||
|
CONFLICT RESOLUTION
|
||||||
|
-------------------
|
||||||
|
|
||||||
|
Concurrent insertions at same location:
|
||||||
|
- Sorted by Lamport timestamps (descending)
|
||||||
|
- Preserves user intent
|
||||||
|
- Guarantees same ordering on all replicas
|
||||||
|
|
||||||
|
Lamport clocks:
|
||||||
|
- Logical timestamps for causal ordering
|
||||||
|
- Each operation increments local clock
|
||||||
|
- Receiving operation updates clock to max(local, received) + 1
|
||||||
|
|
||||||
|
|
||||||
|
UNDO/REDO
|
||||||
|
---------
|
||||||
|
|
||||||
|
Undo map:
|
||||||
|
- Associates operation IDs with counts
|
||||||
|
- Odd count = undone
|
||||||
|
- Even count = redone (or never undone)
|
||||||
|
- Enables arbitrary-order undo in collaborative context
|
||||||
|
- Your undo doesn't affect others' operations
|
||||||
|
|
||||||
|
|
||||||
|
DATA STRUCTURES
|
||||||
|
---------------
|
||||||
|
|
||||||
|
SumTree (the "soul of Zed"):
|
||||||
|
- Thread-safe, snapshot-friendly, copy-on-write B+ tree
|
||||||
|
- Leaf nodes contain items + summaries
|
||||||
|
- Internal nodes contain summary of subtree
|
||||||
|
- Used EVERYWHERE in Zed (20+ uses)
|
||||||
|
|
||||||
|
Rope:
|
||||||
|
- B-tree of 128-byte string chunks (fixed size)
|
||||||
|
- Summaries enable fast offset-to-line/column conversion
|
||||||
|
- Concurrent access safe via copy-on-write snapshots
|
||||||
|
|
||||||
|
Where SumTree is used:
|
||||||
|
- Text buffers (via Rope)
|
||||||
|
- File lists in project
|
||||||
|
- Git blame info
|
||||||
|
- Chat messages
|
||||||
|
- Diagnostics
|
||||||
|
- Syntax trees
|
||||||
|
|
||||||
|
|
||||||
|
SERVER ARCHITECTURE
|
||||||
|
-------------------
|
||||||
|
|
||||||
|
Components:
|
||||||
|
- Collaboration server (Rust)
|
||||||
|
- PostgreSQL database
|
||||||
|
- LiveKit for voice/screenshare (optional)
|
||||||
|
|
||||||
|
Protocol:
|
||||||
|
- Protocol buffers (proto/zed.proto)
|
||||||
|
- RPC over WebSocket
|
||||||
|
- Server routes messages, manages rooms, auth
|
||||||
|
|
||||||
|
Key crates:
|
||||||
|
- crates/proto/ - message definitions
|
||||||
|
- crates/rpc/ - generic RPC framework
|
||||||
|
- crates/collab/ - collaboration server
|
||||||
|
|
||||||
|
|
||||||
|
CHANNELS
|
||||||
|
--------
|
||||||
|
|
||||||
|
IRC-like but for code:
|
||||||
|
- Each channel = ongoing project or work-stream
|
||||||
|
- Join channel = enter shared room
|
||||||
|
- See what everyone is working on (ambient awareness)
|
||||||
|
- Easy to jump into someone's context
|
||||||
|
|
||||||
|
Features:
|
||||||
|
- Shared cursors with zero latency
|
||||||
|
- Following (your view follows their navigation)
|
||||||
|
- Voice chat built-in
|
||||||
|
- Text chat in editor
|
||||||
|
- Screen sharing
|
||||||
|
|
||||||
|
|
||||||
|
WHAT A CLI VERSION WOULD NEED
|
||||||
|
-----------------------------
|
||||||
|
|
||||||
|
From Zed's approach:
|
||||||
|
1. CRDT text buffer (like Cola)
|
||||||
|
2. Anchor-based positions instead of offsets
|
||||||
|
3. Tombstone deletions with version vectors
|
||||||
|
4. Lamport timestamps for ordering
|
||||||
|
5. Undo map for per-user undo
|
||||||
|
6. Some transport (WebSocket, WebRTC, etc)
|
||||||
|
7. Optional: server for discovery/auth, or pure P2P
|
||||||
|
|
||||||
|
The SumTree/Rope is optional but helps with:
|
||||||
|
- Large file performance
|
||||||
|
- Efficient snapshot creation for background tasks
|
||||||
|
|
||||||
|
|
||||||
|
LINKS
|
||||||
|
-----
|
||||||
|
|
||||||
|
CRDT blog: https://zed.dev/blog/crdts
|
||||||
|
Rope/SumTree: https://zed.dev/blog/zed-decoded-rope-sumtree
|
||||||
|
Channels: https://zed.dev/blog/channels
|
||||||
|
Collab docs: https://zed.dev/docs/collaboration/overview
|
||||||
|
Local dev: https://zed.dev/docs/development/local-collaboration
|
||||||
Loading…
Reference in a new issue