Researchers at the Georgia Institute of Technology have developed and validated a new computational framework for simulating nuclear thermal propulsion (NTP) engines, addressing one of the most persistent challenges in reactor engineering: the strong coupling between neutron physics and thermal-hydraulic behavior. The framework, called ANTHMM, is a Python package designed to externally couple Monte Carlo neutronics solvers with commercial computational fluid dynamics (CFD) software. In its current form, ANTHMM is optimized for the Serpent Monte Carlo code and Ansys Fluent, two widely used tools in nuclear analysis. The team applied the framework to the XE-Prime NTP engine, a flight-ready design from the NERVA program era, and demonstrated that their coupled simulations reproduce experimental data with excellent agreement while dramatically reducing computational cost.
Nuclear thermal propulsion has attracted renewed interest from government and industry, particularly for crewed Mars missions. The power density of nuclear materials is sufficient to greatly reduce interplanetary transit time, thereby lowering crew exposure to bone-embrittling microgravity and high-energy ionizing radiation. Unlike conventional propulsion systems, which must carry both fuel and oxidizer, nuclear systems require only the transport of a low-mass working fluid such as hydrogen. However, testing NTP designs is challenging due to environmental concerns and consequent regulatory constraints, which require that experimental efforts be heavily complemented and guided by modeling and simulation. Modern computational tools are orders of magnitude more powerful than those available during NTP’s infancy, yet even the smallest high-fidelity models require hours or days on supercomputers to solve the discretized momentum and energy transport equations.
The research team, led by Jonathon Thomsen, Camden Eck, and Dan Kotlyar at Georgia Tech’s George W. Woodruff School of Nuclear and Radiological Engineering, approached the problem using two complementary philosophies. High-fidelity models optimize the accuracy of the system’s physics at substantial computational cost, often requiring the system to be reduced and analyzed piecewise. Reduced-order models, in contrast, seek to balance accuracy and simulation time, characterizing the bulk behavior of large systems that cannot feasibly be represented with CFD due to extraordinary mesh and computational requirements. For both approaches, the authors emphasize that validation against experimental data is a necessary condition for using results at a project management level, and verification serves as a sufficient condition when experimental data is unavailable.
The XE-Prime engine was chosen as the validation target because it was the most flight-ready design of its time. Developed as a joint effort primarily between NASA and the Westinghouse Astronuclear Laboratory (WANL) under the NERVA program, comprehensive reports from this era can serve as the basis for validating modern codes. The NTP development effort in the United States began in 1953 with the ROVER program, which built thirteen reactors with each iteration improving on the last. Several material selection and flow instability problems were solved during that program, informing the seven systems developed under NERVA, which aimed to demonstrate engines that maintained structural integrity, could be restarted, and were reliable enough for crewed space operations.
The team’s analysis focused on a reduced-geometry supercell of the XE-Prime core, consisting of a hexagonal outer domain enclosing a central moderating element and six primary fueled elements, with six partial fueled elements at the outer vertices. The central moderating element comprises seven radial layers and five materials, including an Inconel tie rod included as a structural element to counteract the high axial loads induced by the large total pressure drop over the length of the active core region. The fueled elements are structurally simpler, with nineteen coolant channels directing hydrogen from the low-temperature plenum to the nozzle chamber, and the fuel itself being a graphite-uranium matrix. The final 8.255 centimeters of each fueled element is an unfueled tip, where the graphite-uranium matrix is replaced with unfueled graphite material.
Material property definition proved challenging due to the range of values reported in the literature and the actual properties of materials used in test articles. The researchers defined material properties as functions of temperature, subject to each solver’s capabilities. Ansys Fluent supports material definitions using polynomial correlations and field functions, while CoRE-ThM supports a variety of definitions including polynomial and exponential correlations. Notably, polynomial fits of material properties with complex temperature-dependent behavior are truncated in Ansys Fluent if they require more than eight coefficients, which is the primary reason piecewise linear functions were used for the fuel region. The temperature-dependent density of hydrogen was modeled with a field function expressing density as a function of temperature only, with separate correlations developed for each analyzed operating condition using the corresponding channel inlet pressure.
The reduced-order thermal-hydraulic code, CoRE-ThM, was developed in the Computational Reactor Engineering (CoRE) laboratory at Georgia Tech and enables rapid, computationally inexpensive analysis of a wide range of reactor architectures with a specific focus on NTP reactors. The most basic component within any CoRE-ThM model is the Element, a two-dimensional cross-section of a region within the core containing dimensional and geometric information. Elements are then used to construct a Channel, composed of an arbitrary number of axial layers of user-defined length. Channels are dynamically linked into a Flow Scheme, which contains all boundary conditions shared between channels and determines the order in which channels are solved. The computational sequence relies on reducing a model into a 1.5D conduction-convection solution, where each channel is approximated as an equivalent cylinder conserving key hydraulic quantities including heated and wetted perimeter, flow area, and hydraulic diameter.
The CFD model, built in Ansys Fluent using a finite-volume approach, employed the steady-state Reynolds-Averaged Navier-Stokes (RANS) formulation with shear stress transport (SST) turbulence equations and enhanced wall treatment (EWT) to capture turbulent fluctuations of velocity and temperature fields. The analysis assumed all flow regions are incompressible, as in-channel shocks would damage the core itself. The CFD mesh consists of 3,968,070 cells, twelve cell zones, and fifteen shell conduction regions, with symmetry planes reducing the total number of mesh cells required. Thin material regions within the moderating element caused some difficulty, as cell refinement in these regions can greatly increase the total number of cells in the mesh. Due to the lack of available documentation characterizing heat transfer within the hydrogen gaps, the hydrogen in these gaps was assumed to be stagnant, with heat transfer across the gaps considered solely through conduction while radiative heat transfer was neglected.
The validation results show excellent agreement between the CFD model, CoRE-ThM, and reference data across three different regions of the core, recorded during the XE-Prime experiment at radial power factors of 0.8, 1.0, and 1.2. Temperature profiles in the fueled element’s channels vary only slightly between cases, with maximum deviations from the reference occurring in the near-inlet region, likely due to increased mixing and improved heat transfer that would be difficult to accurately capture without a turbulence model. The pyrolytic graphite region, intended to insulate the inner tie rod, is characterized by extreme radial temperature gradients, with a 0.25 millimeter distance corresponding to a temperature difference of 50 Kelvin at the outlet plane. The moderating element’s coolant channel, thermally downstream of all other regions except the Inconel tie rod, shows the CFD model and CoRE-ThM within 5 and 10 Kelvin of the reference curve respectively, indicating that the applied material models and heat transfer methods have a propagated thermal error well within the range of the uncertainty of the reference data.
In the coupled verification, power shares are well preserved across all cases, with differences occurring in materials where statistical uncertainty from the Monte Carlo simulations is high, such as regions containing hydrogen and those located several mean free paths from the materials where most power is generated. The maximum temperature deviation between the two thermal-hydraulic models is in the Inconel region, with a 55 Kelvin overprediction of the outlet temperature in CoRE-ThM’s results for the 1.2 radial power factor case. As this is a very small portion of the total domain, the researchers demonstrated there is no effect on the neutronic results. Perhaps most significantly, the team showed that informed prediction from a reduced-order solver of first-iteration temperatures and densities used in the Monte Carlo simulation can reduce the coupled simulation from several iterations to a single predictor-corrector step. The maximum deviation from the converged density profile decreases from approximately 6 percent in the first iteration to below 0.3 percent after three iterations, indicating rapid convergence of the coupled framework. The researchers suggest that future development of the ANTHMM package may increase the fidelity of results by including radiative heat transfer, and that as interest in NTP grows in both industry and academia, an increased number of institutions will seek higher-fidelity solutions using larger CFD models and higher resolution of coupled effects.
Subject of Research: Coupled neutronic and thermal-hydraulic simulation and validation for the XE-Prime nuclear thermal propulsion engine
Article Title: Application and validation of a higher-fidelity architecture for coupled neutronic and thermal-hydraulic analysis in the XE-prime nuclear thermal propulsion system
Article References: Thomsen, J., Eck, C., & Kotlyar, D. (2026). Application and validation of a higher-fidelity architecture for coupled neutronic and thermal-hydraulic analysis in the XE-prime nuclear thermal propulsion system. Frontiers in Nuclear Engineering, 5, Article 1884654. https://doi.org/10.3389/fnuen.2026.1884654
Image Credits: AI Generated
DOI: 10.3389/fnuen.2026.1884654
Keywords: nuclear thermal propulsion, XE-Prime, Monte Carlo neutronics, CFD, reduced-order modeling, multiphysics coupling, Serpent, Ansys Fluent, NERVA, reactor simulation, Application, validation
News Source: Denise Maddox. (October 16, 2026). New Simulation Framework Validates Coupled Neutronic and Thermal-Hydraulic Analysis for Nuclear Thermal Propulsion. Scienmag.
