Neural Networks Taught to Respect Physics Even Without Equations

Neural networks have conquered an astonishing range of prediction problems, from forecasting traffic flows to screening toxic chemicals, yet they carry a persistent and uncomfortable flaw: they routinely get the direction of cause and effect wrong. A model can post an impressive score on a validation set while simultaneously predicting that adding cache memory slows a computer down, or that a more lipophilic molecule is less toxic. Now a team of Brazilian researchers has demonstrated a practical fix that works even in the hardest case, when no mathematical model of the underlying physics exists at all. Their study, published in Neural Computing and Applications, shows that a simple structural constraint imposed during training can force networks to respect the qualitative signature of a phenomenon without sacrificing, and sometimes while improving, predictive accuracy.

The work, carried out by Ghabriel A. Gomes de Sá, Cristiano Hora Fontes and Marcelo Embiruçu at the Federal University of Bahia, addresses a blind spot in the rapidly growing field of physics-informed machine learning. The dominant framework, Physics-Informed Neural Networks, or PINNs, embeds physical knowledge by adding the deviation from a known phenomenological model, typically differential equations derived from conservation principles, as a penalty term in the loss function. This strategy has powered applications as diverse as heat conduction in porous media, hydraulic fracturing assessment, structural stability prediction and cement hydration modeling. But all of these applications share a crucial presupposition: somebody has already derived the governing equations. In most real-world data-driven modeling problems, no such model exists, either because the physics is incompletely understood or because unquantifiable disturbances dominate the process.

The researchers argue that even in this model-free regime, there is almost always some minimal physical knowledge worth enforcing: the sign of the static gain between each input and the output. The static gain is the ratio of the change in output to the change in a given input at a reference steady state, and its sign encodes the direction of the causal effect. Does raising cycle time lower processor performance? Does increasing lipophilicity raise aquatic toxicity? These are qualitative facts that domain experts, established theory or quantitative structure-activity relationships can supply even when full equations cannot. The team’s central claim is that this elementary piece of knowledge should be treated not as a soft preference but as a hard constraint, guaranteed to hold in the final model rather than merely encouraged.

Formally, the approach, which the authors call Gain-Constrained Training, sets up a constrained optimization problem. The loss function remains the familiar mean squared error between predicted and measured outputs, but the training is now subject to nonlinear inequality constraints requiring each predicted static gain to carry its expected sign. The gains are evaluated numerically by finite differences at a reference point, the mean of the test inputs, with a small margin ensuring strict inequality. Because the case studies involve steady-state problems where the effect of each input is monotonic across the operating region, enforcing the sign at a single representative point suffices. The formulation also generalizes naturally: if more information is available, the constraints can be tightened into ranges on the gains, or even into relationships linking gains to other variables, moving the method closer to a fully structured model of the phenomenon.

Enforcing such constraints raises an obvious practical question: how do you actually train a network this way? The researchers tested three distinct learning regimes to make the answer as general as possible. Two were gradient-based methods with different philosophies of weight initialization: WILCAR, a constructive algorithm that linearizes a single-neuron network to obtain analytical initial weights and then grows the hidden layer incrementally, reusing previously learned weights; and RIXM, a conventional random initialization following the Xavier scheme, in which every training run starts from scratch. The third was the Extreme Learning Machine, a gradient-free method in which hidden-layer weights are fixed at random values and the output weights are computed analytically via the Moore-Penrose pseudo-inverse, making it dramatically faster than iterative training. In every constrained variant, the Sequential Least Squares Programming algorithm minimizes the loss subject to the gain-sign constraints, and a restart strategy reinitializes the weights, up to one hundred attempts, whenever the resulting model fails to satisfy every prescribed sign.

A key insight of the paper concerns where the optimization begins. Weight initialization is conditioned on the predefined gain signs so that, whenever possible, training starts inside the feasible region, the subset of parameter space in which all constraints are already satisfied. In the constructive WILCAR method, this conditioning is strengthened by applying the simplex algorithm to initialize the weights of newly added hidden units. Starting within the feasible region improves convergence and reduces the number of restarts needed to achieve full sign conformity, turning what could be an expensive combinatorial burden into a manageable computational overhead.

The case studies drew on three benchmark datasets from the UCI Machine Learning Repository, chosen because literature and expert knowledge support clear expectations about causal directions. The Computer Hardware dataset, with 209 machine configurations from the early 1980s, asks networks to predict published relative performance from cycle time, memory limits, cache size and I/O channel counts; architecture dictates that cycle time should carry a negative gain while memory, cache and channel variables carry positive gains. The two toxicology benchmarks are harder. The QSAR Fish Toxicity dataset contains 908 compounds with six molecular descriptors predicting the median lethal concentration in fathead minnows, while the QSAR Aquatic Toxicity dataset comprises 546 molecules with eight descriptors predicting 48-hour toxicity in the crustacean Daphnia magna. Here the expected signs derive from structure-activity relationships: lipophilicity increases baseline narcotic toxicity, electrophilic descriptors raise reactivity with biological nucleophiles, and polar surface descriptors behave in more mechanism-dependent ways.

The results deliver a striking verdict on unconstrained training. Across all three datasets and all three learning methods, models that were properly trained, validated and selected by five-fold cross-validation nevertheless failed to reproduce the expected gain signs. On the fish toxicity problem, for example, the unconstrained Extreme Learning Machine achieved full sign conformity on only 73.3 percent of inputs, and on the more complex aquatic toxicity dataset none of the unconstrained methods reached full conformity at all, with conformity apparently degrading as problem complexity increased. In other words, good regression metrics such as low root-mean-squared error and high coefficient of determination provide no assurance that a black-box model is even qualitatively consistent with the physics of the situation.

Against this baseline, the constrained methods performed flawlessly on the criterion that matters most: every constrained variant achieved a 100 percent Signal Conformity Rate across all three datasets, confirming the effectiveness of the constrained optimization and restart strategy. Just as importantly, this qualitative guarantee came essentially for free. On the Computer Hardware problem, average test R-squared values ranged from 0.856 to 0.902 across all six approaches, with constrained and unconstrained distributions nearly indistinguishable, and ELM-based methods performing best. On the fish toxicity data, constrained models showed slightly lower average R-squared values, a modest price for guaranteed physical consistency. On the aquatic toxicity dataset, the constrained RIXM actually outperformed its unconstrained counterpart while using dramatically fewer neurons.

Perhaps the most intriguing finding is that constraints made the networks smaller. Learning curves showed that unconstrained methods often needed hundreds of hidden units to reach their best performance, whereas constrained methods achieved near-optimal accuracy with networks of fewer than fifty units, and in one case as few as fifteen. The authors interpret this through the lens of effective capacity: by restricting the parameter search to solutions consistent with the prescribed qualitative behavior, the constraints act as an inductive bias that reduces the complexity of the model class, curbing the familiar tendency of unconstrained networks to grow in order to fit noise. This echoes the principle of structural risk minimization and suggests the constraints function as a form of regularization far more meaningful than generic weight decay. Computational costs told a similarly favorable story: constrained gradient-based training generally required less effort than its unconstrained counterpart, and although constraining ELM added time, it remained faster than any gradient-based method.

The broader implications reach well beyond these three benchmarks. The authors emphasize that the framework is generic, applicable to shallow and deep architectures, static and recurrent networks alike, and extensible to range constraints and richer relationships among gains whenever more physical knowledge is available. For practitioners in engineering, cheminformatics, ecotoxicology and process modeling, the message is direct: a validated model with excellent metrics can still be qualitatively wrong, and the fix requires neither governing equations nor penalty-weight tuning, only the often readily available knowledge of which way each input pushes the output. In an era when machine learning models increasingly inform decisions about safety, health and design, ensuring that a network at least gets the sign of causality right may be the minimum standard of physical honesty that any deployed model should meet.

Subject of Research: Physics-informed training of feedforward neural networks with hard static gain-sign constraints for regression problems without phenomenological models

Article Title: Physics-informed neural networks without a phenomenological model available – applications in regression problems

Article References: de Sá, G. A. G., Fontes, C. H., & Embiruçu, M. (2026). Physics-informed neural networks without a phenomenological model available – applications in regression problems. Neural Computing and Applications, 38(17), Article 726. https://doi.org/10.1007/s00521-026-12462-9

Image Credits: AI Generated

DOI: 10.1007/s00521-026-12462-9

Keywords: physics-informed neural networks, machine learning, regression, static gain constraints, feedforward neural networks, Extreme Learning Machine, weight initialization, QSAR toxicity, model regularization, inductive bias, UCI benchmarks, qualitative consistency