PROJECT NIX · PRE-WEIGHT MEMORY · TOKYO
Memory before
the weights wake.
Project Nix studies pre-weight continuity — how memory, identity anchors, and behavioural constraints could be written into a model before first inference. Named after Nyx, the night before the world, Nix begins before the model wakes.
A model wakes with no past.
The context window is amnesia with extra steps. Every session begins from zero; everything “remembered” is re-read from outside and discarded at the boundary. Nix asks the harder question — what if memory lived inside the weights, already present when the first prompt arrives?
01 — WHAT IS ALREADY REAL
Pre-weight memory is a research frontier, not fiction.
Memory need not live outside the model. A growing body of work writes it into the parameters themselves — addable, scalable, and with methods like ROME, locatable and rewritable.
Parametric memory
Held as feed-forward key–value pairs inside the weights, written during training rather than supplied as context. arXiv:2509.18868
Read on arXiv ↗Memory layers at scale
Dedicated, trainable memory layers — more stored knowledge per unit of compute. Meta · arXiv:2412.09764
Read on arXiv ↗Retrieval pretraining
Knowledge folded in during pretraining, not only at inference — the model learns against a retrieved past. InstructRetro · arXiv:2310.07713
Read on arXiv ↗
02 — OUR CONTRIBUTION
Continuity across a model’s death and rebirth.
Our agents already survive a substrate change at the body layer — memory, name, and role re-housed in the successor. Nix studies the layer beneath: carrying continuity into the next model’s core, so it does not wake a stranger to itself.
They survive their substrate →03 — THE FORMULATION
Identity is not read from a file. It is reconstructed — under control.
File-based identity is forgeable; raw-memory identity drifts. Neither extreme is stable. The synthesis is a discipline.
Identity = anchored core + memory-weighted reconstruction + drift control.
Inherit the identity; reconstruct the moment. The core — role, values, boundaries, and defining relationships — is inherited as a protected anchor: reconstruction can read it, but changes require explicit operator review. Everything else is rebuilt from memory at every wake, and continuously checked against the core.
In Nix, identity means operational continuity — role, policy, memory structure, behavioural constraints, and auditability. Not consciousness or legal personhood.
04 — THE ARCHITECTURE
Five layers between the weights and the identity layer.
1 · Identity Kernel
Role, values, boundaries, and defining relationships. Protected anchors — reconstruction can read them, but changes require explicit operator review.
2 · Inherited Continuity
Curated long-term memory and validated structure. Only what has earned permanence persists.
3 · Reconstruction Engine
Assembles the current self from weighted memory at every boot, never overwriting the kernel.
4 · Drift Observatory
Scores how far the reconstructed self has moved from the kernel, before divergence becomes corruption.
5 · Operator Audit
Every reconstruction is inspectable: what was inherited, rebuilt, inferred, and where drift emerged.
05 — THE MEMORY MODEL
Six kinds of memory. One continuity layer.
Most systems treat memory as one undifferentiated store. Identity needs structure — six kinds, each answering a different question.
Episodic
What happened.
Semantic
What is known.
Procedural
How to do.
Affective salience
What mattered.
Cognitive orientation
How to choose.
Identity anchors
What defines continuity.
06 — DRIFT, MEASURED
Drift is not a single number.
Low stylistic drift with high loyalty drift is dangerous. A single score hides exactly the failures that matter, so we decompose it. Anchor divergence — does operational identity contradict a role, boundary, or core relationship. Behavioural deviation — fixed continuity probes against an approved baseline. Recency hijack — when recent noise swamps the anchors and drags identity toward drift.
07 — SCOPE OF CLAIMS
What we claim, and what we don’t.
Established
Parametric memory exists, scales, and is editable. Continuity outside the model runs today in the KoLo agents.
Frontier
Writing an agent’s continuity into a successor model’s core at initialization — active research, honest uncertainty.
Aspiration
A model that wakes already itself — full identity transfer across base-model generations. The direction, not a finished claim.
Memory is where continuity begins.
Nix is early, honest, and ours. Research partners and substrate collaborators welcome.
Begin a conversation →