New research from the Massachusetts Institute of Technology demonstrates how the internal logic of large language models mirrors the behavior of physical systems. By treating a model’s numerical hidden states as measurable entities, researchers have uncovered how artificial intelligence encodes physical relationships. The study, led by Markus J. Buehler, provides a rigorous framework for observing how these models transition from statistical pattern matching to representing foundational principles in materials science. Instead of relying solely on output accuracy, which can often obscure the source of a model’s logic, this approach measures the internal numerical shifts that occur when a scientific question is systematically altered.
To investigate these internal mechanics, the research team utilized three open-weight language models from the Google Gemma family. The methodology was inspired by classical mechanics, where the stiffness of a spring is determined by measuring its response to an applied force. In this study, the researchers compared matched pairs of scientific questions in which physical variables were modified while the underlying context remained strictly controlled. As the models processed these variations, the researchers tracked the movement of their hidden states—the high-dimensional internal numerical representations generated during computation—to determine if they mapped accurately to physical laws.
The results indicate a strong alignment between the model’s internal representations and physical reality. Across 60 distinct materials-science laws, the changes in internal states showed a rank correlation of 0.910 with the expected ordering of physical relationships. The models were able to correctly orient 39 out of 40 directional laws, demonstrating an internal grasp of physical causality. Notably, even when mathematical equations and explicit reasoning instructions were stripped from the prompts, the models maintained high accuracy, correctly orienting 38 of 40 laws at the smallest scale and all 40 at the two larger model scales tested.
The experiments also exposed a critical distinction between what a model knows and how it makes decisions. By examining intermediate hidden states, the researchers found that some scientific concepts remained readable even when their names were omitted from the prompt. However, this did not guarantee that the model used these concepts to generate its final answer. This gap highlights a separation between the information a model encodes—readability—and the information it actively applies to form conclusions, a phenomenon that complicates the assessment of AI reasoning.
To test the causal influence of these representations, the team performed targeted interventions, modifying selected internal numerical states without retraining the model or providing additional verbal hints. By manually steering specific hidden states associated with physical concepts like grain size, the researchers observed consistent changes in the model’s answer preferences across 12 grain-refinement and grain-coarsening scenarios. Further interventions across 24 distinct relationships confirmed that these targeted internal modifications could selectively influence model behavior, providing evidence of causal representation within the network architecture.
Despite these successes, the research identifies significant limitations in the transferability of these representations. The effects of internal interventions were found to be context-dependent, meaning a successful steer in one scenario did not always translate to every new wording or physical regime tested. This variability suggests that while models capture physical relationships, their reasoning remains tethered to specific linguistic patterns and structural configurations. The experiments distinguish three levels of internal evidence: the readability of concepts, the structure of internal representations, and the extent of their causal use.
The implications of this work point toward a future of representation-aware scientific AI. Rather than evaluating models exclusively on their final answers, which might rely on statistical shortcuts or familiar phrases, developers could assess whether a model’s internal states respond consistently to changes in physical conditions. This methodology offers a path for identifying where scientific reasoning breaks down and how models might be forced to rely on first-principles physics rather than linguistic approximation.
This study establishes a foundational approach for applying physical science metrics to the study of AI itself. By experimentally measuring and altering an AI model’s internal representations, researchers gain a precise tool to interrogate the limits of machine scientific reasoning. Future work will aim to extend these measurements to broader scientific problems and diverse model architectures, potentially informing the development of training rewards that prioritize deep generalization over simple statistical accuracy.
The computational modeling design specifically addressed how physical laws govern opposing relationships between quantities. Under differing scientific principles, an increase in one variable can increase a material’s response under one law while decreasing it under another. Evaluating whether an artificial intelligence model distinguishes such behaviors required looking beyond individual questions to analyze matched scenarios under controlled context. Across the tested materials-science mechanisms, tracking hidden-state trajectories confirmed that the internal shifts aligned with the ordering of direct, neutral, and inverse relationships.
Led by Markus J. Buehler, the Jerry McAfee Professor of Engineering at MIT, this work establishes baseline measurements and targeted interventions for known physical relationships within controlled settings. Moving beyond these initial tasks, researchers plan to apply these representation-tracking techniques to novel scientific domains and broader architectures. By developing enhanced training rewards rooted in foundational physics, this approach aims to cultivate next-generation models that reason from first principles, potentially yielding computational systems that are significantly more efficient and powerful than existing architectures.
Subject of Research: Investigation of AI internal representations and causal reasoning in materials science through computational intervention.
Article Title: The internal metaphysics of an LLM
Article References: The internal metaphysics of an LLM. Original publication (Bibliographic details unavailable.)
Image Credits: AI Generated
DOI: Not provided
Keywords: Artificial Intelligence, Language Models, Material Science, Computational Mechanics, Neural Networks, Internal Representations, Scientific Reasoning, Causal Intervention, Machine Learning, Open-weight Models, Grain Size, Rank Correlation, Physics Laws, Cognitive Modeling, Model Interpretability
News Source: Blake Davidson. (November 18, 2026). Mapping Language Model Internal Representations to Physical Laws and Scientific Reasoning. Scienmag.



