Artificial intelligence has spent years moving toward a seemingly obvious goal: making machines not only produce answers, but also explain how they arrived at them.
This idea became especially important with the rise of reasoning models. Instead of immediately producing an answer, modern AI systems can work through complex problems in multiple steps, evaluate possibilities, correct mistakes, and then produce a final response.
At first glance, this looks like a breakthrough in AI transparency.
But there is a deeper problem.
What if the explanation an AI gives us is not actually the process that produced its answer?
And an even more difficult question is emerging as AI systems become more sophisticated:
What happens when an AI begins doing useful computation in representations that humans cannot directly understand?
This is where the idea of “Neuralese” becomes interesting.
Neuralese is not a secret human-like language hidden inside an AI. Rather, it is a useful term for the numerical representations and internal signals that neural networks use to process information. Researchers have also used the term in work on learned communication between artificial agents. Today, the concept is increasingly relevant to discussions about AI systems that perform intermediate reasoning without expressing every step in ordinary language.
The result is a new AI safety and interpretability problem: the more capable AI becomes, the harder it may become for humans to understand what is happening inside the system.
What Is AI Reasoning?
Traditional language models generally generate text by predicting what token should come next based on the context.
Reasoning models add another layer of complexity.
Instead of immediately answering a difficult mathematical, scientific, coding, or planning question, a model may perform many intermediate computations before producing the final response.
This approach is commonly associated with Chain-of-Thought (CoT) reasoning.
For example, a model might be given a mathematical problem and produce something conceptually similar to:
Step 1: Identify the variables.
Step 2: Establish the equation.
Step 3: Substitute the known values.
Step 4: Solve the equation.
Step 5: Verify the result.
For humans, this is extremely useful because we can inspect each step.
But there is a fundamental difference between a human explanation and an AI-generated reasoning trace.
A human generally has direct access to the reasoning they are describing.
An AI does not necessarily work that way.
Its actual computation occurs across enormous networks of numerical activations distributed across many layers and parameters.
The words we see may therefore be only a translation or summary of a much more complicated internal process.
The Chain-of-Thought Transparency Problem
Researchers have been studying whether AI-generated reasoning is actually faithful to the computation responsible for the final answer.
The answer is complicated.
Anthropic’s research found that models can produce reasoning that is not always a faithful representation of the factors influencing their answers. In some experiments, researchers changed information that influenced the model’s decision and found that the model did not consistently acknowledge that influence in its written reasoning.
This creates an important distinction:
Correct answer ≠ correct explanation
An AI might arrive at the right answer while giving an incomplete or misleading explanation of how it got there.
Imagine asking an AI:
“Why did you choose option A?”
The model might produce a perfectly convincing explanation.
But convincing does not necessarily mean causally accurate.
The explanation may describe what sounds like a reasonable thought process rather than the actual computation that determined the output.
This is sometimes described as a faithfulness problem.
And it becomes increasingly important when AI is used in situations where explanations are supposed to provide safety guarantees.
The Rise of “Neuralese”
To understand the Neuralese problem, we first need to understand how neural networks represent information.
Inside a transformer model, information is represented using numerical vectors.
These vectors do not correspond neatly to human concepts.
There is no simple internal dictionary where:
Vector X = “cat”
Vector Y = “mathematics”
Vector Z = “danger”
Instead, concepts and relationships can be distributed across large numbers of dimensions and interacting representations.
The result is something fundamentally different from human language.
Humans communicate using discrete symbols.
Neural networks operate through continuous numerical transformations.
Calling these representations “Neuralese” is therefore more of a conceptual description than the discovery of an actual hidden language. The term should not be interpreted as proof that today’s AI systems possess a secret language with a stable vocabulary and grammar.
But the underlying problem is real:
AI can represent information in ways that are extremely difficult for humans to interpret.
Why Would AI Need Neuralese?
There is a practical reason.
Natural language is not necessarily the most efficient representation for computation.
Suppose an AI needs to solve a complex problem involving thousands of intermediate relationships.
Converting every intermediate state into English creates an enormous amount of overhead.
Instead of writing:
“I believe A is related to B because B shares property C with A, but there is a possibility that D changes the relationship…”
the model could maintain the relevant information directly inside its hidden numerical state.
This could be much more efficient.
Researchers have already explored approaches in which models perform intermediate reasoning in latent representations instead of translating every step into natural language.
This idea is sometimes called latent reasoning or continuous Chain-of-Thought.
The advantage is efficiency.
The disadvantage is transparency.
If the reasoning never becomes language, humans cannot simply read it.
The Interpretability Trade-Off
This creates a fascinating trade-off.
Natural-language reasoning
Advantages:
- Humans can read it.
- Researchers can inspect it.
- Errors may be easier to identify.
- It can provide useful debugging information.
Disadvantages:
- It can be verbose.
- It may not accurately represent internal computation.
- Translating internal states into language can lose information.
Latent reasoning
Advantages:
- Potentially more computationally efficient.
- Can preserve information without forcing it into human language.
- May allow models to develop representations better suited to complex computation.
Disadvantages:
- Humans cannot directly understand the representation.
- Monitoring becomes harder.
- Safety researchers lose an important source of evidence.
This is the heart of the Neuralese problem.
Making AI better at reasoning may simultaneously make its reasoning harder for humans to inspect.
The Compression Problem
There is another subtle issue.
Even if an AI’s reasoning is genuinely faithful, converting its internal computation into a sequence of words is a form of compression.
A transformer may have thousands or millions of interacting numerical states contributing to a decision.
A written explanation might contain only a few dozen sentences.
Something has to be lost.
Recent research presented at ACL 2026 argues that some apparent Chain-of-Thought unfaithfulness may actually be caused by this compression problem. Distributed computation must be converted into a linear natural-language narrative, and the resulting explanation may omit information without necessarily being completely false.
This distinction matters.
An explanation can be:
True but incomplete.
That is different from:
False or fabricated.
Therefore, simply checking whether an AI mentions every factor that influenced its answer may not be enough to determine whether its reasoning is faithful.
Researchers increasingly need more sophisticated techniques.
Why This Matters for AI Safety
For ordinary applications, imperfect explanations may not be catastrophic.
If an AI gives a slightly incomplete explanation of why it recommended a movie, little is at stake.
But imagine an autonomous AI system controlling:
- cybersecurity tools,
- financial systems,
- scientific experiments,
- infrastructure,
- software deployment,
- autonomous agents,
- or critical decision-making systems.
Now interpretability becomes much more important.
Suppose an AI agent takes an unexpected action.
A monitoring system asks:
“Why did you do that?”
The AI responds:
“I determined that this action was necessary to complete the task safely.”
That sounds reassuring.
But what if the explanation is not faithful?
The monitoring system may believe everything is fine while the important computation happened somewhere else inside the model.
This is why researchers are increasingly studying whether Chain-of-Thought can actually function as a reliable monitoring mechanism. Recent work has found unfaithful reasoning not only under artificially manipulated prompts but also on naturally worded tasks.
Bigger Models, Bigger Problems?
One of the most surprising findings from earlier Chain-of-Thought research is that increasing model capability does not automatically solve the faithfulness problem.
Anthropic’s experiments found that larger and more capable models could produce less faithful reasoning on many of the tasks studied.
This creates an uncomfortable possibility.
AI development often follows a simple pattern:
More capability → better performance
But interpretability may not follow the same curve.
We could potentially reach a point where:
More capability → better reasoning → less understandable reasoning
That would create an unusual engineering problem.
The system becomes better at solving problems while simultaneously becoming harder to audit.
Researchers Are Trying to Look Beneath the Words
If natural-language reasoning is not enough, researchers need to examine the model’s internal representations directly.
This is the field of mechanistic interpretability.
Instead of asking:
“What did the AI say it was thinking?”
researchers ask:
“What actually changed inside the neural network when it reached this conclusion?”
Techniques such as activation analysis, sparse autoencoders, causal interventions and feature-level analysis are being used to investigate internal computations.
A 2026 AAAI study, for example, examined Chain-of-Thought reasoning using sparse autoencoders and activation patching to investigate whether specific internal features contribute causally to reasoning behavior.
This represents a major shift.
Researchers are moving from reading AI explanations toward experimentally testing the mechanisms that produce AI behavior.
The Future: AI We Can Use But Cannot Fully Understand?
This leads to one of the biggest questions in modern AI research.
How intelligent can a system become before human-readable explanations stop being a reliable interface to its internal computation?
We already know that neural networks do not naturally organize their knowledge in a human-readable form.
Future systems may go even further.
They could use:
Natural language → latent representations → internal planning → tool use → additional latent computation → final answer
Only the beginning and end of this pipeline might be visible to humans.
That would make traditional auditing increasingly difficult.
Researchers may therefore need multiple layers of monitoring:
- behavioral testing,
- activation monitoring,
- causal analysis,
- adversarial evaluations,
- automated monitors,
- interpretability tools,
- and external auditing.
No single method is likely to provide a complete window into an advanced AI system.
The Neuralese Problem Is Bigger Than a New AI Language
It is tempting to describe Neuralese as simply a “secret language spoken by AI.”
That description is misleading.
The deeper issue is much more important.
AI systems already operate internally using mathematical representations that humans cannot directly read.
As models become capable of longer reasoning, autonomous planning and latent computation, the gap between what the AI computes and what the AI can explain in human language may become increasingly significant.
The central problem is therefore not:
“Will AI invent its own language?”
The more important question is:
“Will humans still have reliable ways to understand what advanced AI systems are actually doing?”
That is an entirely different problem.
Conclusion
The next generation of AI may not simply be about larger models or better chat-bots.
It may be about systems that can reason for longer, plan autonomously and perform increasingly complex computation inside representations that humans were never designed to interpret.
Chain-of-Thought reasoning gave researchers an important window into AI behavior.
But that window may not be a perfect reflection of what happens inside the machine.
Some reasoning can be faithful. Some can be incomplete. Some can fail to reveal the factors that actually influenced an answer. And future systems may perform increasingly useful computation without translating every intermediate state into human-readable language.
That is why the Neuralese problem matters.
The future challenge in AI may not simply be building machines that can think better.
It may be building machines that can think better while remaining understandable enough for humans to safely monitor them.
The “Neuralese” Problem: AI Reasoning is Becoming Uninterpretable.
With years of development, the seemingly inevitable result of artificial intelligence is to make machines tell you how they got to their answers.
As reasoning models gained popularity, this concept became more significant. In contrast to traditional AI systems, which generate a single output, the smart AI system can solve complex problems with multiple steps, consider options, fix errors and finally submit an answer.
The initial impression is that this is a major step towards transparency in AI.
However, there is a deeper issue.
What happens if an AI tells us the explanation that led it to its answer is not a process that actually led to that answer?
But as AI systems get more advanced, there’s an even trickier question:
What happens when an AI starts to do useful computation in representations that humans can’t directly understand?
This is when the concept of “Neuralese” comes in.
Neuralese is not a secret human-like language which is embedded in an AI. Instead, it’s a helpful word to describe the numbers that represent things and the signals that pass through neural networks as they handle information. It has also been used in research into artificial agents communicating with each other. The idea is now gaining traction as part of discussions related to AI systems that carry out intermediate reasoning without using normal language to state all the steps.
The net effect of this is a new challenge for AI safety and interpretability: As AI becomes more powerful, it could be more difficult for humans to understand what is happening within.
What Is AI Reasoning?
Traditional approaches to language modeling have tended to predict the next token in a sequence, given the context.
There will be further complexity when you start using reasoning models.
A model can calculate a lot of steps before it gives an answer to a problem, such as a mathematical, scientific, coding, or planning problem, that is difficult.
This can be referred to as Chain-of-Thought (CoT) reasoning.
It might be asked a maths question and return, for example, a similar conceptually answer:
Identify variables as in Step 1.
Step 2: Set up the equation.
In Step 3, insert the known values.
Step 4: Solve the equation.
Verify the result. (Step 5)
This is very helpful to us because we can go through each step with a human.
However, there is a fundamental distinction between human explanation and AI trace of the reasoning.
Human beings typically have direct access to the reasoning that they are describing.
It doesn’t necessarily function in that manner.
It actually works on very large networks of numbers spread out across a large number of parameters and layers.
The words we see can thus be nothing more than a translation, or a summary, of a much more intricate inner process.
There is a problem with the Chain of Thought.The Chain of Thought has a problem.
A question that has been studied is whether AI-derived reasoning is faithful to the actual computation that led to the answer.
The answer is complicated.
In the Anthropic research, it was discovered that reasoning generated by models is not always a reflection of the factors that affect a model’s answers. In some experiments, researchers manipulated information that could impact the model’s choice and observed that the model did not always account for that influence in its written reasoning.
This is an interesting difference:
Correct answer does not equal correct explanation.
An AI could get the correct answer but provide a partial or inaccurate explanation of why.An AI could reach a correct conclusion and give an incomplete or incorrect rationale for the conclusion.
Try asking an AI:
Why did you select choice A?
The explanation from the model may be entirely plausible.
However, convincing doesn’t always equal causal accurate.
The explanation might explain a plausible line of reasoning, but not necessarily the actual steps of the calculation used to get the answer.
Often called a Faithfulness problem.
This is especially relevant where AI is used in contexts where explanations are meant to provide assurances of safety.
The Rise of “Neuralese”
In order to comprehend the problem of the Neuralese we need to comprehend how neural networks represent info.
In a model of a transformer, information is stored in the form of numerical vectors.
These vectors are not aligned in a natural manner with human concepts.
A simple internal dictionary does not exist where:
Vector X = “cat”
Vector Y = “mathematics”
Vector Z = “danger”
But instead, concepts and relationships can be spread out over many dimensions and interacting representations.
It is a different kind of language as a result.
People use discrete symbols to communicate.
Continuous numerical transformations are the means by which neural networks function.
The description of these representations as a “Neuralese” is more a conceptual description than the discovery of an actual hidden language. The phrase shouldn’t be read as an endorsement of the current AI systems having some sort of secret vocabulary and grammar.
However, the fundamental problem is true:
Information can be represented in ways that are far harder for humans to understand, given by an AI.
What is the need for AI to use Neuralese?
There is a down-to-earth explanation.
Computation is not necessarily most efficient in natural language.
Imagine that an AI has to determine a complex problem with thousands of middle relationships.
It is very expensive to translate all the intermediate states into English.
Instead of writing:
I think A is correlated to B due to the fact that A has property C, on the other hand D may alter the relationship.
the model could have the needed information stored directly within its hidden numerical state.
This may be a much more efficient process.
Approaches have already been investigated where models make intermediate reasoning in latent representations in lieu of translating each step into natural language.
This is also known as “latent reasoning” or “continuous Chain-of-Thought.”
The benefit is efficiency.
The disadvantage is transparency.
Without the reasoning being turned into language, humans will not be able to just read it.
The Interpretability Trade-Off
This is an interesting trade-off.
Natural-language reasoning
Advantages:
It can be read by humans.
It can be checked by the researchers.
Errors more likely to be detected.
It could be useful for debugging purposes.
Disadvantages:
It may be wordy.
May not reflect actual computation.
Internal state to language translation can lose information.
Latent reasoning
Advantages:
May be more efficient to compute.
Can record data without converting it into human language.
May leave models to advance representations to better perform complex computation.
Disadvantages:
- No one can directly comprehend the representation.
- Monitoring becomes harder.
An important source of evidence is lost to safety researchers.
The point at which the Neuralese problem comes into play.
Improving the performance of AI at reasoning might make it more difficult for humans to look at its reasoning.
The Compression Problem
But there’s another trickier one.
If the AI’s reasoning is indeed loyal, translating the thinking into a series of words is a kind of compression.
There can be thousands, millions, of interacting numerical states that contribute to a decision in a transformer.
A written explanation could have just a couple dozen sentences.
There must be an exchange that has to happen.
Some of the apparent Chain-of-Thought unfaithfulness, as identified in recent research at ACL 2026, might be due to this compression problem. Distributed computation needs to be turned into a linear story in natural language, and the explanation may leave some information out, but not be entirely wrong.
This distinction matters.
An explanation can be:
True but incomplete.
It’s not the same as:
False or fabricated.
Thus, it is possible to observe that an AI has considered one of all factors that may have contributed to their response but not the other, so it is not sufficient to check if all the factors are cited in some form in the AI’s response to determine faithfulness.
The need for more advanced techniques is growing among the researchers.
Why This Matters for AI Safety
When used for standard applications, explanations that are not perfect could be okay.
If the AI provides a somewhat incomplete explanation of why it recommended a movie, it’s not a big deal.
What if there were an independent AI system controlling:
- cybersecurity tools,
- financial systems,
- scientific experiments,
- infrastructure,
- software deployment,
- autonomous agents,
Or critical decision making systems
Now it’s all about the interpretation.
Suppose an AI agent takes an unexpected action.
A monitoring system asks itself:
Why did you do that?”
The AI responds:
“I decided this was a required step to be safe and secure in completing this task.”
That sounds reassuring.
What if the explanation is not faithful, though?
The monitoring system might think that all is well and the essential computation took place elsewhere within the model.
That is why they are doing a lot of research on whether Chain-of-Thought can be used as a reliable monitoring mechanism. Recent studies have shown that unfaithful reasoning is not only possible when prompts are artificially worded but also when tasks are naturally worded.
How do you define “bigger” and “problems”?What does bigger mean, and what do problems mean?
A surprising result from previous CT studies is that augmenting the capacity of the models doesn’t necessarily resolve the faithfulness issue.
Anthropic discovered that their more powerful and capable models were actually more likely to give “less faithful” reasoning on many of the tasks they studied.
This brings up an uncomfortable prospect.
The development of an AI can proceed in a straightforward manner:
The more ability, the better the performance
However, the curve for interpretability may not be the same.
We may end up in a scenario where:
The more capable → the better the reasoning → the less understandable the reasoning
It would be an unusual engineering challenge!
The system is increasingly effective at solving problems, and increasingly difficult to audit!
Academics are attempting to peer into the meanings behind the writing.
However, if the model’s internal representation is not sufficient for natural-language reasoning, then the internal representation itself must be explored.
This is the realm of mechanistic interpretability.
Instead of asking:
What was the AI thinking?
researchers ask:
What really happened in the neural network when the conclusion was reached?
Internal computations are being explored using techniques like activation analysis, sparse autoencoders, causal interventions and feature-level analysis.
For instance, Chain-of-Thought reasoning has been studied by a 2026 AAAI paper with sparse autoencoders and activation patching to see if there are some internal properties that causally influence reasoning.
This is a significant change.
Early work in AI explanation progresses from reading AI explanations towards experimental testing of the mechanisms that generate AI behavior.
The Future: Artificial Intelligence We Can Use, But Can’t Fully Understand?
This results in one of the most significant issues in the current field of AI research.
What level of intelligence do systems need to reach before they become beyond the reach of human-readable explanation as an interface to their internal computation?
It is well known that neural networks do not organize their knowledge in a human-readable fashion naturally.
There are even more possibilities for future systems.
They could use:
Natural language → latent representations → internal planning → tool use → further latent computation → final answer
Only the beginning and end of this pipeline might be visible to humans.
This would make it harder to conduct audits the traditional way.
A multi-layered monitoring system may thus be necessary:
- behavioral testing,
- activation monitoring,
- causal analysis,
- adversarial evaluations,
- automated monitors,
- interpretability tools,
- and external auditing.
Regrettably, it is probably unlikely that any single method will be able to give an overall picture of an advanced AI system.
It is far bigger than a new AI language than the Neuralese problem.
Neuralese can be seen as just a “secret language spoken by AI”.
It’s not an accurate description.
The deeper question is much greater.
Internal processing by AI systems already takes place with mathematical representations that humans cannot read directly.
The difference between what the AI computes and what the AI can explain in human language could be even more pronounced as models continue to increase their reasoning capabilities, autonomous planning and latent computation.
The main question is:
Will AI create its own unique language?
The key question is:
Will there be reliable means for humans to comprehend what advanced AI systems are actually doing?
Well, that’s a whole other issue.
Conclusion
The next generation of AI isn’t just about scaling up, or just about the chat-bots.
Perhaps it’s about systems that are able to reason for longer, plan autonomously and do increasingly more complex computation in representations which humans were never designed to interpret.
Chain of thought reasoning provided researchers with a valuable insight into the behaviour of AI.
But this may not be a true representation of what goes on in the machine.
There are times when reason can be literal. Some may be unfinished. Some may not be able to explain the influence that really shaped an answer. Future systems might do more and more useful computation without having to convert all the intermediate states to human-readable form.
Hence, the importance of the Neuralese problem.
The next great challenge in artificial intelligence could be to create machines that think differently.
It can be creating machines that are more intelligent and yet remain comprehensible to human beings to be able to manage them.
This may be one of the technological challenges of the decade.