Search, Don't Compute: How Engram Is Rewriting AI Architecture
Imagine asking an ultra-advanced language model to complete the sentence "United States of". It doesn't take a genius to figure out that the answer is "America", yet behind the scenes something paradoxical happens: the model activates billions of parameters, runs through dozens of computation layers, consumes real electricity, to arrive at a result that anyone who has ever read a newspaper knows by heart. It's like asking a math professor to learn the Pythagorean theorem from scratch every time they need it, instead of applying it as they recall it from middle school. The paradox of modern artificial intelligence lies right here: excellent at 'reasoning' through new problems, surprisingly inefficient at retrieving facts it has already seen thousands of times during training.
In recent years, the industry has responded to this problem with Mixture-of-Experts models—so-called MoEs—which allow a model's capacity to be increased by activating only a portion of its parameters for each token, much like having a hundred specialized consultants in a company but calling only three or four to each meeting. It is a great solution for managing compute power, but it does not solve the fundamental issue: even a MoE model, when completing "United States of", continues to 'think' the answer rather than 'remember' it. And this is precisely where the proposal from DeepSeek fits in—published in January 2026 and updated in July of the same year by a team led by Xin Cheng alongside colleagues from DeepSeek-AI and Peking University: a new axis of specialization, conditional memory, which complements the conditional computation of MoEs rather than replacing it.
Engram, the Memory That Isn't Computed but Searched
The module is called Engram, a name that intentionally recalls the biological trace of memory theorized by neuroscientists a century ago—the idea that a memory leaves a physical, traceable imprint in the brain. In the paper, the authors describe it as a module that modernizes classic N-gram techniques by transforming them into a constant-time search system, what in computer science is called O(1): the time to retrieve information remains the same regardless of how large the lookup table is.
How does it work in practice, without wading through formulas? The model looks at the current token and a handful of immediately preceding ones, combines them through a hashing function that generates a unique index, and uses that index to fetch directly from a huge table pre-compiled during training. It is no longer solving an equation; it is consulting an analog index, much like in an old library when you don't scroll through all the shelves but go straight to the card catalog indicating the right shelf. The difference compared to a paper index is that here the "index" isn't consulted by a patient librarian, but by an algorithm that does so in a predictable and consistently low time, whatever the volume of stored data.
The numbers speak for themselves. Comparing an Engram-27B model with a pure MoE model at equal parameters and computation (what researchers call an iso-parameter and iso-FLOPs comparison), Engram achieves an improvement of 3.4 points on MMLU, 5.0 points on BBH, and 3.0 points on HumanEval, alongside gains on CMMLU, MATH, GSM8K, and DROP. The most interesting piece of data, according to the paper, is not even the factual retrieval in itself, but rather that the most marked improvements are observed in general reasoning and in coding and math tasks: freeing the network from the burden of having to reconstruct static facts seems to leave it with more "mental space" to reason through complex problems—much like a student who doesn't have to compute a simple multiplication every time, but draws on memorized times tables and can finally focus on geometry problems.
The U-shaped Law: How Much Memory, How Much Compute
The paper's most subtle discovery, however, concerns a problem that at first glance seems purely accounting-driven: given a fixed parameter budget, how many should be allocated to Engram and how many to the traditional MoE? The researchers formulated what they call the Sparsity Allocation problem, testing different combinations on models ranging from a few billion parameters up to 27 billion parameters.
The result is a U-shaped curve. Using only MoE, without any dedicated memory, yields below-average performance. But the opposite extreme—a model almost entirely based on memory lookups and lacking computational capacity—performs even worse. The optimal point lies in the middle, and according to the experiments described in the project's official repository, it sits at around 20-25% of sparse parameters dedicated to Engram, with the remaining 75-80% allocated to MoE. What makes this discovery particularly useful is its stability: the optimal ratio remains nearly identical whether training a 5-billion-parameter model or a 27-billion-parameter model, offering engineers a true design recipe rather than an intuition to verify case by case.
This is no mere detail for insiders: it is a compass. And some have already followed it to the letter.
Qwen3.8-Flash-Next: When Theory Becomes a Downloadable Model
What distinguishes this story from so many other academic proposals left on paper is that, just a few months later, someone actually built it at an industrial scale. On August 26, 2026, Alibaba's Qwen team released Qwen3.8-Flash-Next, officially presented as an experimental preview of the architecture that will underpin Qwen4—so much so that in the model's configuration files, the architecture is internally labeled as "qwen4_exp".
The numbers are dizzying just to read: a 125-billion total parameter multimodal MoE model, with only 6 billion parameters activated per token, featuring a native context window of 262,144 tokens extendable up to one million. The most surprising part, however, is what interests us most here: alongside the 125 billion main parameters, Qwen added an additional 51-billion-parameter N-gram embedding table, applying almost to the letter the U-shaped law recipe described by DeepSeek. A terminological clarification: Engram is the name DeepSeek gave to its module in the original paper, whereas Qwen, while taking inspiration from that idea, calls its implementation simply "N-gram Embedding". For this reason, in the rest of the article, "Engram" indicates the module described by DeepSeek, while "N-gram" refers to the table integrated into Qwen3.8-Flash-Next. To the total is also added a small 4-billion-parameter module dedicated to speculative decoding—the technique allowing the model to "guess" multiple tokens in advance to accelerate generation, a less flashy piece than the 51 billion parameters of the N-gram table, but one that in practice heavily impacts final performance.
What if a consumer GPU could run a model that, on paper, would require a data center? That is precisely what this architecture promises. The 51-billion-parameter N-gram table does not necessarily need to live on the GPU: it can stay in system RAM, or even on an NVMe drive, and be loaded only when needed thanks to an asynchronous prefetch mechanism. It's like having a huge warehouse outside town rather than in the downtown store, with a delivery system so efficient that it still feels within easy reach. This isn't theory: pull requests already integrated into SGLang and official vLLM recipes show that NVMe offloading is already supported by mainstream inference frameworks, not a laboratory experiment.
On the compression front, the open-source community moved extremely fast: the N-gram table, which in BF16 format occupies roughly 95 gigabytes, can be quantized down to 28.8 gigabytes in NVFP4 format or 32 gigabytes in INT4, making offloading possible even on systems with just 64 gigabytes of available RAM, as documented in several quantized checkpoints published on Hugging Face. Translated into plain terms: what on paper looks like a data center model can, with the right adaptations, run on an enthusiast's powerful PC, not a hundred-thousand-dollar server.
The Speed of Search vs. the Slowness of Computation
Here comes the second ingredient of the architecture, Qwen Sparse Attention, which tackles a complementary problem: how to handle extremely long contexts without blowing up compute times. The idea is to group context into micro-blocks instead of analyzing every single token—an approach that, according to Alibaba's official benchmarks, accelerates attention by up to 7.6x in the prefill phase and 4.9x in the decoding phase on 1-million-token contexts, though it's worth noting that different sources, such as SGLang cookbooks and vLLM recipes, report slightly different figures (up to 10.2x and 6.6x), confirming that these numbers should always be read as manufacturer-claimed until independent measurements arrive.
The overall architecture alternates, every four layers, three passes of Gated DeltaNet—a technique that compresses context history into a fixed-size state—with one pass of sparse attention for precise retrieval across the entire context, for a total of 48 layers organized into blocks of 12. Completing the picture is the Gated Residual, a mechanism that opens four parallel "lanes" in the network's residual stream instead of just one, reducing bottlenecks during training much like a multi-lane highway avoids traffic jams compared to a two-lane country road.
The practical result of all this engineering? According to reports on NVIDIA's technical blog, on DGX hardware the model achieves throughputs exceeding 16,000 tokens per second per GPU in high-concurrency agentic scenarios, while quantized versions released by the community allow Qwen3.8-Flash-Next to run offloaded on systems with a single consumer GPU, such as an RTX 4090 with 24 gigabytes of VRAM, provided it is paired with around a hundred gigabytes of system RAM to host the N-gram table. You don't need a supercomputer to take a close look at this technology; a good gaming PC and a little patience with configuration are enough.
Why It Pays to Expect More from "Smaller" Models
The comparison that stands out most, according to statements by the Qwen team on GitHub, is with its predecessor Qwen3.7-Plus, a 397-billion-parameter model: Qwen3.8-Flash-Next achieves comparable or superior performance despite being trained with roughly one-ninth of the computational expenditure, with particularly marked improvements in programming tasks and office productivity scenarios. It is a signal worth taking seriously: for years the industry has pursued the "bigger is better" logic, accumulating parameters as if they were the only lever available to improve a model. This story suggests that architectural efficiency can compete with, and sometimes beat, the raw brute force of scale.
Not everything is sunshine and roses, however. It is worth asking: who actually gains, and who stands to lose? Companies that sell compute capacity at scale today—the big cloud providers used to billing based on the gigawatt-hours consumed by training runs—see architectures like this as a direct threat to their business model, because a training run nine times cheaper also means nine times less margin on compute sales. Conversely, those developing applications on limited budgets—from independent developers to startups that cannot afford a cluster of professional GPUs—gain concrete access to capabilities previously reserved for a select few players. There is also a less discussed but relevant issue: memory tables of this size, fifty-one billion parameters fixed by training, raise questions about their updateability. If a fact changes, how do you update the model's memory without repeating the entire training process? It is an open question that DeepSeek's paper touches upon only superficially, pointing to dynamic, real-time updateable memory as one of the most promising future research directions.
The Future of Agents Runs Through Here
There is one last piece of the puzzle that makes this architecture more than a mere academic exercise: the rise of AI agents—systems capable of managing extremely long contexts, invoking external tools, and carrying out complex tasks autonomously. For an agent that must keep track of hundreds of thousands of tokens of conversation, technical documentation, and tool outputs, having fast memory with low computational cost is not a luxury; it is a structural necessity. It is no surprise that Qwen already lists native support with Claude Code, Codex, and other agentic development environments among the highlights of the release.
Those familiar with niche science fiction will recognize in this idea a distant echo of the narrative in Ghost in the Shell: Stand Alone Complex, where memory is not a monolithic block but a distributed fabric updated and consulted without ever pausing the flow of conscious thought. And those who know software development might think of a much more down-to-earth but equally central pattern: the cache. Engram, at its core, is conceptually a giant cache of linguistic knowledge, built not to be invalidated at runtime, but to be queried at (nearly) zero cost.
That DeepSeek and Qwen chose to openly share both theoretical research and model weights, rather than keeping them locked inside a proprietary product, inevitably accelerates the adoption of these techniques across the entire open-source community—from small university labs to indie developers who can download, study, and modify the code directly on GitHub. It remains to be seen whether this path—that of memory that searches rather than computes—will become the standard for the next generation of models, or if it will remain one of many brilliant insights destined to be superseded by the next idea. For now, however, the promise is concrete and verifiable: an artificial intelligence that stops reinventing the wheel every time it is asked "United States of", and finally has room to 'think' about something more interesting.