Wax, explained

Documents, embeddings, full-text indexes and a structured knowledge graph — all bundled into one .wax binary that lives on your device. No server, no cloud, no Docker. You move it like any other file.Everything below is drawn from the source tree at /Users/tmk/dev/playground/Wax. Claims carry file:line citations so you can verify them; where the README and the code disagree, this page follows the code and says so.

What is this thing, really?

Strip away the marketing and Wax is three databases hiding inside one file, plus a coordinator that keeps them in sync.

Most retrieval-augmented (RAG) setups bolt together a document store, a vector database, and a text search server — three services to deploy and keep consistent. Wax collapses all of that into a single self-contained binary file with a .wax extension README.md:29. Opening that file gives you four capabilities at once:

▤ A document store

Raw text/blobs are written as compressed frames (LZ4 / LZFSE / deflate / plain). Each frame carries metadata, checksums, and parent-child links so a document and its chunks stay related. FileFormat.md:88–110

▦ A full-text index

An embedded SQLite FTS5 virtual table (frames_fts, unicode61 tokenizer) provides BM25 lexical search. FTS5Schema.swift:8

◈ A vector index

Embeddings (default MiniLM, 384-dim MiniLMEmbedder.swift:28) searched on the GPU for semantic recall. (See the honesty note about “HNSW” below.)

❖ A knowledge graph

A built-in EAV / entity-fact store with bitemporal facts lives in the same SQLite blob as FTS5 — durable, queryable knowledge, not just blobs of text. StructuredMemory.md:7

How a query works

A single search fans out to both the BM25 text index and the vector index, then merges the two ranked lists with Reciprocal Rank Fusion (RRF) — a rank-based merge that ignores the incompatible raw score scales of BM25 vs. cosine distance HybridSearch.swift:1–30. The weighting (alpha: 0 = all vector, 1 = all text) can even adapt to the query type — factual queries lean on text (0.7/0.3), semantic ones lean on vectors (0.3/0.7), temporal ones pull in a recency signal AdaptiveFusionConfig.swift:22–25.

Who uses it, and how

• Swift API Embed the Wax module in an iOS/macOS app: Memory(at:) → save / search.
• CLI wax-cli remember "…" / wax-cli search "…" --mode hybrid, or a long-running daemon.
• MCP An MCP server (wax-mcp) exposes memory tools to Claude Code / Cursor so the agent remembers context across sessions. A broker process owns long-term memory plus per-session “scratch” stores.

The confusing words, decoded

Wax’s docs assume you already know a dozen storage-engine terms. Tap any to expand.

Anatomy of a .wax file

Click a region to inspect it. Offsets and sizes are from the format spec. FileFormat.md

Why this shape? Crash-safety by construction

The two design tricks that make a single mutable file safe to crash on:

1. Dual header pages (A/B). Two 4 KiB headers sit at offset 0 and 4 KiB. Writes alternate between them; on open, Wax reads both and picks the one with the highest generation that still passes its SHA-256 checksum. A crash mid-header-write can never corrupt the store — worst case you fall back to the previous generation. FileFormat.md:52–56
2. Write-ahead log (WAL) ring buffer. Every mutation — putFrame, deleteFrame, supersedeFrame, putEmbedding — is appended to a fixed-size circular log (default 256 MiB) before it touches the main structure. On the next open, pending records are replayed; corrupt tail records are tolerated. WALAndCrashRecovery.md:24–48,62–72

Every layer — WAL records, header pages, TOC, footer — carries a SHA-256 checksum, and the TOC additionally keeps a 32-byte Merkle root. Corruption is detectable at every level. WALAndCrashRecovery.md:99–108 FileFormat.md:71

How Markdown (and any text) gets ingested

The honest, slightly surprising truth: a .md file is treated as plain UTF-8 text. There is no Markdown parser in the ingest path.

The pipeline, step by step

Try the chunker

Paste text and watch how the 400/40 token window would slice it. Approximation: the real chunker counts WordPiece tokens; this demo estimates tokens at ~1.3×words.

The other Markdown story: managed projections

Separate from ingestion, the MCP broker keeps a small set of human-editable Markdown files in two-way sync with memory. This is the feature that actually matters for sharing knowledge.

The broker can project memory out to Markdown and re-absorb edits back in. It manages three kinds of file under a project root AgentBrokerService+Markdown.swift:9–38:

How a managed line is encoded

Each memory becomes a Markdown list item with an invisible HTML-comment marker that base64-encodes a JSON record (frame id, content hash, memory type, durability, confidence, session id…). The text is human-readable; the marker is machine-readable. BrokerMarkdownSync.swift:81–163

What markdown_sync actually does

Sync is a reconciliation, not a one-way import. For each managed file it matches Markdown lines to stored memories — first by trusted marker (frame id + content hash + memory id + source kind/path all must agree), then by content hash — and classifies every line as created / updated / unchanged / deleted AgentBrokerService+Markdown.swift:276–370:

• In Markdown, new → written to memory (created).
• In both, changed → memory updated (old frame tree deleted) (updated).
• In both, identical → no-op (unchanged).
• In memory, absent from Markdown → deleted from memory… unless the memory is locked. AgentBrokerService+Markdown.swift:358–367

DREAMS.md — promotion review

“Dreams” are candidate long-term memories that the broker harvested from session stores and thinks are worth keeping. They’re rendered as unchecked checkboxes. You tick [x] the ones to keep; on the next sync, each checked candidate runs through a promotion judge (proposePromotion) and the approved ones are written to durable memory. Unchecked = ignored. AgentBrokerService+Markdown.swift:168–248 A human stays in the loop for what becomes permanent.

Sharing knowledge between a work Mac and a personal Mac

The headline question. Short answer: there is no networked replication or store-merge in the code — but there are two viable patterns, and one is much better than the other for two-way sharing.

Decision helper

Aside — corpus_search is not cross-machine. It searches across the broker’s local per-session stores with provenance metadata (which session a hit came from) README.md:250–256. Useful for cross-session recall on one machine; it is not a replication or multi-device feature.

Exploring what’s inside a .wax file

There is no single “open this file and show me everything” inspector. But several focused tools together let you see the contents and verify integrity.

How the EAV triples are indexed — vs Datomic / DataScript / Datalevin

The deep one. Wax models knowledge as RDF-like (subject, predicate, object) triples — but it is not a Datalog database. It’s a fixed-schema, bitemporal triple store on SQLite B-trees.

The actual schema

Entities and predicates are interned into dictionary tables (integer surrogate keys), and every fact is one row in sm_fact with a tagged-union object: an object_kind discriminator (1=string … 7=entity-ref) plus seven typed columns, with a CHECK enforcing exactly one is populated. StructuredMemorySchema.swift:7–67

CREATE TABLE sm_entity   (entity_id PK, key UNIQUE, kind, created_at_ms)
CREATE TABLE sm_predicate(predicate_id PK, key UNIQUE, created_at_ms)

CREATE TABLE sm_fact (
  fact_id PK,
  subject_entity_id  → sm_entity,
  predicate_id       → sm_predicate,
  object_kind INTEGER,            -- 1=str 2=int 3=real 4=bool 5=blob 6=time 7=entity-ref
  object_text/_int/_real/_bool/_blob/_time_ms/_entity_id,   -- exactly one, by CHECK
  version_relation,                -- sets / updates / extends / retracts
  fact_hash BLOB UNIQUE,          -- SHA-256(S,P,O) → idempotent assert
  qualifiers_hash )

CREATE TABLE sm_fact_span (    -- BITEMPORAL: two independent time axes
  span_id PK, fact_id → sm_fact,
  valid_from_ms,  valid_to_ms,     -- when the fact is TRUE in the world
  system_from_ms, system_to_ms,    -- when it was RECORDED (NULL = still asserted)
  span_key_hash UNIQUE )

CREATE TABLE sm_evidence (     -- provenance back to the source text
  source_frame_id, chunk_index, span_start_utf8, span_end_utf8,
  extractor_id, extractor_version, confidence, asserted_at_ms )

The indexes Wax keeps

Only these B-tree indexes exist on the fact tables StructuredMemorySchema.swift:109–130:

Plus key/alias indexes for resolution: sm_entity(key), sm_entity_alias(alias_norm) (NFKC-normalized, case-folded for fuzzy match), sm_predicate(key).

Query-pattern explorer

Pick what you want to bind, and see whether Wax has a covering index — versus which of Datomic’s four index orders a Datalog DB would use.

Head-to-head with the Datalog databases

Quick answers