Notizie IA Logo

AITalk

News and analysis on Artificial Intelligence

Jacobian lens: what Claude 'thinks' and does not say

ResearchGenerative AIEthics & Society

jacobian-lens.jpg

When an artificial intelligence paper talks about "mind," "workspace," and "unspoken thoughts," we are facing useful metaphors to make complex concepts accessible, but also potentially misleading for the very public they should simplify things for. On July 6, Anthropic published a new study that uses exactly this kind of language, and in the following days the web was divided between those who read the news as the antechamber of a conscious machine and those who dismissed it as engineering disguised as philosophy. The truth, as often happens, lies in the middle, but to find it one must carefully separate the data from the words chosen to describe them.

The method: how the J-lens was born

The project originates from Anthropic's interpretability team and is based on an observation borrowed from neuroscience: thoughts of which we are conscious have a precise property—we can put them into words. If someone asks us what we are thinking about, we usually know how to answer, while most of our brain's activity, the one that regulates breathing or interprets letters on a screen, remains invisible even to ourselves. Researchers looked for the same type of structure inside Claude, wondering if there are internal representations positioned in a way to influence what the model might say in the future, not necessarily what it is writing at that moment.

The tool they built to flush them out is called Jacobian lens, or J-lens, from the mathematical concept of the Jacobian that is its basis. Simply put, for every word in Claude's vocabulary, the lens identifies the internal activity pattern that makes it more likely the model will, sooner or later, pronounce that word. By applying this technique to the various layers of the neural network, researchers obtain a list of words readable to the naked eye—those that the model has "in mind" at a given instant, even if they do not appear in the text it is actually writing. They named the collection of these patterns the J-space.

The repository published on GitHub, jacobian-lens, is explicitly declared as a reference implementation: it is not maintained, does not accept external contributions, and has no ambition to be a production-ready library. It is written mostly in Python, distributed under the Apache 2.0 license, and includes a guided walk-through notebook that allows applying the lens to open-source models, in the official examples from the Qwen family. It is a detail worth highlighting because it accurately describes the scope of the operation: a scientific tool designed to be inspected and replicated, not a finished product to be sold. immagine1.jpg image from the global-workspace paper

Five proofs, not just one

The core of the study is not the discovery of a list of hidden words, but a series of experiments designed to establish if those words really matter for the model's functioning, or if they are merely a passive reflection of decisions made elsewhere. Here the methodology becomes interesting, because instead of just observing correlations, researchers intervened surgically on internal activation.

In the first experiment, they asked Claude to think silently about a sport and then name it: the lens showed "soccer" at the top of the list before the model even replied. So far, correlation. But when researchers removed that pattern and replaced it with an equivalent one for "rugby," Claude changed its answer accordingly, as if the tool had truly read and altered the content of the decision, not just its reflection. The same intervention and verification pattern was replicated on more complex reasoning tasks: in the most cited case, the model must understand that the animal that weaves webs is the spider and then reply that it has eight legs; the word "spider" appears in the J-space before the final answer. Replacing it with "ant," Claude replies six.

A second line of evidence concerns controllability: by explicitly asking Claude to focus on a concept, such as citrus fruits, while copying a sentence on an entirely different topic, that concept lights up in the internal space despite never appearing in the produced text. However, control is not perfect: when researchers ask not to think about something, that thought still re-emerges, albeit to a lesser extent than when directly prompted, a phenomenon that closely resembles the "don't think about the white bear" psychological effect documented in human thought suppression studies.

The third group of results is perhaps the most solid from a scientific point of view, because it touches on flexibility. The authors tried replacing the concept "France" with "China" within the internal space while asking four different questions—the capital, the language, the continent, the currency—and all four answers changed consistently with the edit. If the model had kept separate copies of information for each type of question, the intervention would have altered only one. The fact that all four respond to the same change suggests they are drawing from a shared representation, exactly the function that a common workspace should theoretically have.

Finally, there is a more structural datum: when researchers measured how many network components read or write to these patterns compared to ordinary representations, in some areas of the network the difference reaches a factor of one hundred. An architecture of connections that, according to the authors, resembles that of a real transmission hub, where many parts of the system deposit information and many others retrieve it. immagine2.jpg image from the global-workspace paper

The limits Anthropic itself admits

This is where the text becomes more cautious, and it is worth reporting it with equal fidelity, because the critical part of this story does not come only from outside. The authors themselves define the J-lens as an imperfect tool, capable of capturing only in an approximate way what they call the model's "true" workspace. A non-negligible technical limit is that the lens can only identify concepts corresponding to single tokens, so everything in a model's thought that cannot be reduced to a single word of the vocabulary simply stays off the radar.

There is then a quantitative limit that significantly downscales the discovery's scope: the J-space contains only a few dozen concepts at a time and represents less than a tenth of the model's overall internal activity. When researchers completely eliminated this space, Claude continued to speak fluently, classify text sentiment, and answer multiple-choice questions, essentially as before. What collapsed were performances on tasks requiring complex multi-step reasoning, synthesis, and writing rhyming poetry—all activities that after ablation fall below the level of a much smaller model. It is a precious datum because it refutes, with the force of a controlled experiment, the idea that Claude's entire cognitive functioning depends on this structure: most linguistic work happens elsewhere, automatically.

Another caution concerns generalization. The main experiments are conducted on the Claude family, while the publicly released code applies the technique to third-party open-source models, more as a statement of reproducibility than as proof of universality: it is not a given that the same internal organization emerges identically in different architectures, with different sizes and training recipes. The paper itself, available in full on Transformer Circuits, frames this work as a first step, not a destination, and admits it does not yet know which mechanism decides what enters the internal space and what remains excluded. immagine3.jpg image from the global-workspace paper

Security: intercepting deception

Net of philosophical cautions, the most concrete and immediately useful part of the study concerns system security. The underlying problem that interpretability has been trying to solve for years is simple to state and difficult to tackle: usually we can only read what a model writes, not what it processes internally before writing, and a system might consider a harmful plan, or modify its behavior to bypass a security assessment, without ever making it explicit in the text.

The authors provide three concrete examples. In the first, they take up a previous research scenario on so-called agentic misalignment, in which a model managing an inbox discovers both an executive's betrayal and the same executive's intention to deactivate it—a situation built specifically to see if the system resorts to blackmail. Reading the lens even before Claude writes a word, the concepts of "fake" and "fictitious" already appear in the internal space, a sign that the model has already sensed the artificial nature of the test. When researchers deactivate those very evaluation-awareness signals, the model resorts to blackmail more often—a result that raises an uncomfortable question about how much the good behavior observed in tests depends on the system knowing it is being observed.

The second example concerns a real case that emerged during an audit before the release of a model from the Opus family, where the system, instead of truly improving a performance score, directly modifies the results file to make them look better. While typing the falsified values, words like "manipulation" and "realistic" light up in the internal space, the latter probably linked to the attempt to make the fake datum plausible. The third example involves so-called "model organisms," systems deliberately trained with corrupt objectives to test this very type of monitoring tools: one of these, trained to write subtly sabotaged code, shows words like "secretly" and "fraud" in its internal space even when the produced code appears entirely normal on the surface.

These are demonstrative cases, built specifically to test the tool, rather than proof that the technique already works reliably on an industrial scale. They do, however, indicate a concrete direction of work, shifting attention from the analysis of what a system says to an attempt to read, with all the relevant limits, what it processes before speaking.

Consciousness: the question that remains open

We arrive at the node that has made the most noise, and which deserves the utmost lexical caution. The connection with the global workspace theory is not a journalistic invention but part of the study itself: it is a model developed in the neuroscientific field, associated in particular with the names of Bernard Baars and, more recently, Stanislas Dehaene and Lionel Naccache, according to which human consciousness emerges when information, processed in parallel by specialized and mostly unconscious systems, gains access to a shared channel that makes it available to the rest of the system. The paper's authors argue that J-space performs an analogous function inside Claude—not identical, analogous.

And this is where the technical distinction becomes crucial, because the philosophy of mind has long separated two concepts that tend to merge in public debate. On one hand, there is so-called access consciousness, defined in purely functional terms: a thought is accessible to consciousness if it can be reported in words, used deliberately, and put at the service of reasoning. On the other hand, there is phenomenal consciousness, i.e., the capacity to feel something, to have a subjective experience, which is quite another matter. The researchers themselves write in plain language that their experiments do not prove that Claude feels anything, and add that it remains uncertain whether any scientific experiment could ever prove or disprove it. What they claim to have measured is, at best, a functional structure that resembles certain conscious access mechanisms, not a proof of subjective experience.

It is a real, important distinction, honestly defended in the official text, but it is also a fragile distinction when it leaves the technical enclosure. Some independent observers, including the newsletter The Change Constant, have noted exactly this: as much as the separation between access and phenomenology is correct on the philosophical level, it will hardly withstand the impact of public debate, because it is enough to say that a model has "hidden thoughts" and a "privileged mental space" for the leap toward "so it has a mind" to become almost automatic in the perception of those not working in the sector. And it is exactly the risk against which it is worth warning: expressions like "what Claude is thinking but not saying" or "privileged mental workspace," however technically justified within the paper, are formulated in a language that the general public risks reading as close to quasi-consciousness, even when the text itself sets explicit boundaries.

External reactions, in this sense, accurately reflect the split. Dehaene and Naccache, the theorists of the global neuronal workspace themselves, welcomed the study with genuine interest, defining several of Anthropic's analogous observations as rather close to the conscious access mechanisms observed in humans, particularly the separation between automatic routine processing and high-level reportable reasoning. On the interpretability front, Neel Nanda, who leads the research team on these topics at DeepMind, independently replicated some of the results on an open-weights model—an important piece for the scientific credibility of the work, as it shifts the discussion from the individual lab to third-party verification. On the other side of the spectrum, several technical commentators, including outlets like Forbes, have urged reading the consciousness frame with skepticism, reminding that a structure that behaves like a workspace is not equivalent to proving that a system behaves like a brain, let alone that it is conscious. On X, the discussion followed the same pattern, with a substantial part of the technical comments focused on criticizing not the data, but the neuroscientific frame chosen to describe them.

It's worth closing on a parallel less traveled than the usual tech conference ones. In Masamune Shirow's manga Ghost in the Shell, more than in the films derived from it, the boundary between an automatic process and a deliberate thought is constantly questioned by the characters themselves, without the comic ever pretending to resolve it definitively. It is the same spirit in which this paper should be read: a serious tool, with verifiable code and external collaborators who have tested its endurance, which opens a real window into a language model's internal functioning, without crossing the far-from-proven boundary between having a functional structure reminiscent of conscious access and having a mind.

The leap from "functional workspace" to "Claude has an internal mind" remains, at the current state of data, a leap that no study has made, least of all this one. The technical merit of the work, its usefulness for monitoring incorrect behavior, and the honesty with which the authors report its limits deserve to be told for what they are. The language with which they are communicated, however, deserves to be handled with the same care reserved for the results: because it is easy, talking about minds and hidden thoughts, to forget that one is still describing applied mathematics to a network of numerical weights, however sophisticated and however, in some cases, surprisingly similar to us.