Shanghai-based AI company MemTensor has released and open-sourced MemT 1.0, a family of what it calls memory-native foundation models. The series includes two models: MemT-1.0-Flash, with 288 billion total parameters and 17 billion active, and MemT-1.0-Nano, with 36 billion total and 3 billion active. Weights are published on Hugging Face, and the code, together with a 74-author technical report, is available on GitHub.
The models target a familiar cost problem in long conversations. Most systems carry the full history in the context window, so every additional turn makes inference slower and more expensive. MemT instead adds a trainable memory module inside the model. After each interaction, the model writes new information into a fixed-size memory state and recalls it when needed, rather than rereading the full record each time.
MemTensor reports that the approach pays off most at long histories, where inference runs nearly four times faster than a full-context setup and per-session storage shrinks by more than 100 times. All performance figures come from the company’s own evaluation, and the technical report is a preprint that has not been peer-reviewed.
The Problem: Long Histories Are Expensive
A language model normally works through the entire earlier conversation on every turn. That design keeps reasoning simple, but it means a long history slows responses, inflates inference costs, and consumes memory. The KV cache that stores intermediate computations for the full context grows with every token, and at hundreds of thousands of tokens it becomes one of the largest single costs of serving a long-running assistant.
Existing workarounds fall into two camps. Systems can keep stuffing old messages into the context window, accepting the growing bill, or they can bolt on an external store and retrieve snippets when needed. Retrieval-augmented approaches work well in many cases, but they add infrastructure, and the quality of the answer depends on the quality of the retrieval at query time.
MemT 1.0 takes a third route, and the company’s framing, offered in its Chinese-language release, is that the model learns to take its own notes rather than flipping back through the transcript. The approach moves memory from an external bolt-on to a learned capability of the model itself, which is what the term memory-native is meant to signal.
How the Memory-Native Architecture Works
A Native Memory Block Beside Attention
MemT 1.0 retains the backbone model’s original computation and adds a Native Memory Block alongside attention in selected Transformer layers. Two components make up the block. The Local Memory Block maintains a dynamic, multi-channel memory matrix that evolves across interaction steps, holding a compressed representation of the conversation. The Hyper Memory Block supplies the trainable machinery: memory query, key, and value projections, along with erase, write, forget, and output gates.
During generation, memory attention reads from the current memory state, normalizes and gates the readout, and adds it to the backbone attention output. At the end of each turn, a stabilized form of Gated DeltaNet-2 performs token-level memory updates, with a symmetrization fix that keeps the memory states from diverging during training. To expand capacity further, the report introduces Mixture-of-Storage, a scheme that splits each memory block into parallel memory experts. Unlike sparse mixture-of-experts routing, MoS uses dense routing, so every expert participates, and the routing probabilities determine how information is written to and read from each one.
Crucially, the model’s weights stay fixed. What changes is the per-session memory state, which each interaction trajectory maintains independently. No fine-tuning is needed to absorb new information, and the memory state remains the same size as the conversation grows.
What the Model Actually Remembers
The repository’s documentation includes a demonstration that shows the behavior in practice. A session is told about a three-month-old kitten named Memmy who loves small shrimp and is excellent at remembering things. After a second exchange, the model saves that turn to latent local memory and clears the plaintext conversation context. When asked about the kitten’s name, age, favorite food, and special trait, it answers correctly from the latent memory alone.
The same demonstration shows sessions being saved to a snapshot file and restored later with an empty transcript. The restored session reports zero stored messages, yet still answers “How old is my kitten?” with “Your kitten is 3 months old.” For developers, this suggests a deployment pattern where long-running assistants persist a compact memory state between sessions instead of shipping full transcripts.
The MemT 1.0 Model Family
The release consists of two models built on prominent open-weight backbones. Flash extends DeepSeek-V4-Flash, while Nano extends Qwen3.6-35B-A3B. Both are mixture-of-experts models, which is why the active parameter count is far below the total.
| Model | Backbone | Total Parameters | Active Parameters | Trainable Memory Parameters | Dynamic Memory State |
|---|---|---|---|---|---|
| MemT-1.0-Flash | DeepSeek-V4-Flash | 288B | 17B | 3.61B | 336M |
| MemT-1.0-Nano | Qwen3.6-35B-A3B | 36B | 3B | 451M | 10M |
The training corpus breaks down into 20 billion tokens of continued pre-training and 14 billion tokens of post-training data, in a unified format where the context is first stored through the native memory mechanism and the model must then answer a query from the resulting memory state. The method itself is distillation: the frozen backbone, which sees the full history, acts as a teacher, and the memory model learns to match its next-token predictions without ever receiving the transcript. The first stage teaches the model to write information into memory; the second, supervised by KL divergence against the teacher, teaches it to answer questions from what it wrote.
The DeepSeek-V4-Flash backbone for Flash was itself prepared before memory was attached. MemTensor post-trained it on 126.5K curated trajectories and reports a 6.3-point average gain across six monitored benchmarks, with the prepared backbone, at 284 billion total and 13 billion active parameters, exceeding the reported scores of larger open models on SWE-bench Multilingual and IFBench v2.
Performance Claims: Faster Inference, Smaller Footprint
The efficiency story is the headline of the release, and the advantage grows with history length. At 128K tokens of history, MemTensor measures end-to-end inference about 1.86 times faster than a full-context approach. At 256K, the gap widens to about 3.99 times.
Latency follows the same pattern. Query time to first token holds at roughly 104 to 112 milliseconds regardless of history length, compared with about 445 milliseconds for a full-context approach at 256K. In practical terms, the company argues that users of a long-running assistant will not experience responses getting slower the longer the conversation runs.
Storage shows the most dramatic difference. At 256K of history, the persistent memory state for a single session is about 40 MiB, versus about 5.06 GiB for a full-context KV cache, a reduction of roughly 130 times. At 128K, the repository documentation cites an approximately 66 times reduction. These are the figures that matter most for anyone running many concurrent long-horizon sessions, where the aggregate cache can dominate serving costs.
One caveat deserves emphasis: all of these numbers are MemTensor’s own measurements, not independent benchmarks. The startup has published the weights and code specifically so that others can verify them.
Memory Benchmarks: Strong, But Not the Whole Story
Across 11 memory benchmarks, MemT-1.0-Flash averages 50.6 and MemT-1.0-Nano averages 47.0, placing them first and second among the parametric memory methods in the evaluation. The company says the models lead the next-best parametric baseline by 12.5 and 8.9 points, respectively, and that their average performance is roughly level with retrieval-augmented approaches, without needing to retrieve history at query time.
MemTensor also reports that attaching memory did not degrade Flash’s general capabilities. Under distractor-heavy test conditions, general performance held roughly level with the memory-free backbone, and some benchmarks improved.
The model card adds an honest qualification: strong textual-memory systems still lead overall, particularly on long histories that require detailed evidence retrieval. In other words, MemT 1.0 is the current frontier for parametric memory, the approach that stores history inside the model’s own state, but the best external retrieval pipelines can still recall specific details more reliably. The benchmark figure of 38.1 for the best parametric baseline is derived from the company’s reported lead margins and should be read as approximate.
What It Means for Developers
Deployment is straightforward in shape but demanding in hardware. Both models run on SGLang 0.5.16, with memory snapshots stored server-side. The documentation gives one starting configuration: Nano serves on two GPUs, while Flash requires eight. The scripts are described as starting points that have not been deeply optimized, so teams will need to tune parallelism and token budgets for their own workloads.
Licensing deserves careful reading. The repository’s source code, including the SGLang integration, is published under the PolyForm Noncommercial License 1.0.0, which does not permit commercial use without separate permission from the authors. The model weights are more permissive: Flash weights carry an MIT license and Nano weights Apache 2.0. The technical report itself is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 4.0. A team building a commercial product could ship the weights but would need to check what surrounding code it relies on.
The release also includes a demonstration of experience-guided agentic coding. Retrieved operational experience is written into the model’s native memory and accessed through memory attention during execution, improving task performance on long-horizon coding tasks without adding experience text to the task context. On the SWE-bench Verified and SWE-bench Multilingual coding benchmarks, MemTensor reports improved average task success and consistency across repeated runs compared with agents without experience, reaching performance comparable to agents that carry explicit textual experience. This points to where MemTensor sees the value: not just cheaper long chats, but agents that accumulate skill over time.
Toward Agents That Learn From Experience
MemT 1.0 builds on Metis, MemTensor’s earlier memory foundation model research, and the technical report treats memory capacity as a scaling dimension in its own right, alongside data and model size, with dedicated scaling studies on how performance responds to larger memory states. The team says its next steps are improving long-term memory, expanding memory capacity, and lowering the cost of memory updates. The longer-term goal is more ambitious: agents that keep learning from their interactions, with the team framing memory-native models as groundwork for recursive self-improvement.
That framing connects to a broader shift in how the industry thinks about model states. Memory tools such as MemPalace demonstrate strong demand for persistent AI memory today, but they operate outside the model as external stores. MemT 1.0 makes the bet that memory belongs inside the architecture, learned and managed by the model itself. Whether that bet pays off will depend on independent verification of the benchmarks and on how the approach scales beyond the two models released today.
What is not in dispute is the release itself. As the first fully open reference point for memory-native model design, MemT 1.0 hands the AI community both a working implementation and a falsifiable set of claims, a combination that should settle the debate faster than any press release could.
