AI Learns to Design Waveguides from Just 421 Simulations

Engineers designing the millimeter-wave components that future sixth-generation wireless systems will depend on often face a brutal bottleneck: every candidate design must be checked with a full-wave electromagnetic solver such as Ansys HFSS, and a single simulation can take ten to twenty minutes. Sweeping through thousands of geometric possibilities quickly becomes computationally prohibitive. A new study published in Advanced Modeling and Simulation in Engineering Sciences shows how a machine-learning surrogate model, trained on only 421 such simulations, can compress that exploration from weeks of computing into a couple of hours — and, more strikingly, can propose entirely new device geometries that outperform the best design found in the original simulation campaign.

The work, led by Amit Kumar, Clément Mailhé, Jad Mounayer, Dominique Baillargeat and Francisco Chinesta, targets a specific and demanding application: an air-filled waveguide whose sidewalls are made not of metal but of vertically aligned carbon nanotubes, or VACNTs. Such waveguides have recently been introduced as a promising alternative to conventional metallic structures, offering reduced propagation losses and compatibility with wafer-level fabrication. The researchers set out to optimize one of these devices for operation in the D-band, the frequency range around 135 to 155 GHz that is considered ideal for the ultra-high data rates — up to one terabit per second — envisioned for 6G networks.

The core of the method is a rank-reduction autoencoder, or RRAE. A conventional autoencoder compresses high-dimensional data, such as a full frequency-response curve, into a compact latent representation and then reconstructs it. The trouble, as the authors illustrate with a simple example of circles sliding along a diagonal, is that the latent space learned by an ordinary autoencoder does not reliably support interpolation: sampling between two known points can fall off the learned manifold and produce nonsensical outputs. Variational autoencoders fix this by forcing the latent space to follow a statistical distribution, but they are expensive to train and often need to be paired with generative adversarial networks, demanding large datasets.

The RRAE takes a different route. It inserts a truncated principal component analysis stage between the encoder and the decoder, forcing the latent space to be a linear vector space spanned by a small set of orthonormal modes. The result is a low-rank manifold in which interpolation and generative sampling remain robust even when the model is trained on very little data. In the waveguide application, the design latent space of dimension 256 was reduced to rank 6, while the response latent space of dimension 128 was reduced to rank 14. The full surrogate consists of three parts: a design RRAE that maps the five geometric parameters to reduced coefficients, a response RRAE that maps the scattering-parameter curves to their own reduced coefficients, and a multi-layer perceptron that learns the mapping between the two coefficient sets.

The device itself is a five-parameter problem. The designers varied the depth to which the coplanar waveguide feed penetrates the carbon-nanotube cavity, the horizontal and vertical offsets of a taper center, the taper width that governs the aperture coupling energy into the waveguide, and the spacing between the feed and adjacent shielding vias. These parameters are not independent: some combinations of taper position and aperture width produce geometrically inconsistent transitions, and the feed depth and via spacing are bounded by fabrication constraints. The design space is therefore low-dimensional but correlated and partially constrained — exactly the situation, the authors argue, in which a surrogate that learns feasible regions from a limited dataset beats manually refined parameter sweeps.

Training data came from a design of experiments run in HFSS with a driven-modal solution, adaptive meshing at 150 GHz and a convergence criterion of 0.02. Each of the 421 designs yielded reflection and transmission magnitude curves sampled every 200 MHz across 110 to 160 GHz, giving 251 points per curve; ten sine-cosine frequency pairs were concatenated to the two S-parameter channels, producing 22 input channels. The database was split 70/30 for training and testing, and both the RRAE and a control model — a direct encoder-decoder regression baseline sharing the same backbone but lacking the rank-reduction stage — were trained with identical settings: the Adam optimizer, a mean-square-error reconstruction loss, a learning rate of 10 to the minus 4 with cosine annealing and warm restarts, and a batch size of 40.

As pure regression models, the two performed almost identically. On the test set, the RRAE reached a median mean absolute error of 0.41 dB for the reflection coefficient, with a 90th-percentile error below 1 dB; the baseline posted 0.42 dB and 1.00 dB respectively. Transmission was harder in absolute terms, with median errors of 3.25 dB for the RRAE and 3.37 dB for the baseline, but median full-curve correlations remained around 95 percent for both, meaning the overall frequency-dependent trends were preserved. Standardized errors corresponded to roughly 15 percent of test-set variability for reflection and 20 percent for transmission. The authors are explicit that the RRAE’s advantage is not pointwise accuracy but the structure of its latent space.

That advantage emerged decisively in the optimization stage. The design goal was a reflection coefficient below minus 10 dB and transmission above minus 5 dB over at least 10 GHz within the 135 to 155 GHz band. A review of all 421 simulated designs found only one that met the target, underscoring how scarce suitable examples were in the training data. On the hardware used for surrogate modeling — a Ryzen 7 6800H processor with an RTX 3060 GPU — a full prediction takes about 4.9 milliseconds with the baseline and 5.9 milliseconds with the RRAE, versus ten to twenty minutes for one HFSS run on the simulation workstation. The baseline search assessed nearly 9.8 million sampled parameter combinations in about three hours, at 872 designs per second, while the RRAE screened candidates by latent-space interpolation at 1,102 designs per second for one hour and fifty minutes.

The outcomes diverged sharply. The RRAE flagged 807,539 potentially valid designs against 548 for the baseline; after applying a uniqueness criterion requiring parameter sets to differ by at least 10 percent, the RRAE yielded 8 unique candidates and the baseline 10. When the candidates were verified in HFSS, all 8 RRAE designs proved geometrically consistent, and 5 of them met the optimization objective. The baseline fared worse: 5 of its 10 candidates were geometrically inconsistent, and the remaining 5 failed the 10 GHz bandwidth condition, so the direct regression workflow produced no HFSS-confirmed valid design at all. The authors attribute this to the baseline’s limited ability to generalize from 421 examples, noting that comparable inverse-design studies in the literature typically rely on tens of thousands to hundreds of thousands of simulations.

Most of the unique designs retrieved by the RRAE outperformed the single valid design found in the original simulation campaign, demonstrating that the model generalized beyond its training data rather than merely reproducing it. The authors also frame the economics: the 421 HFSS simulations represent an equivalent serial cost of roughly 70 to 140 hours, but once that upfront investment is paid, the trained surrogate could screen on the order of hundreds of millions of candidate designs in the same time. HFSS validation is not replaced — every final design still requires high-fidelity confirmation — but a limited simulation budget is converted into a large-scale exploration capability. The researchers suggest the framework extends naturally to other RF components such as cavities, filters and antennas, wherever data efficiency and design explorability are critical.

Subject of Research: A machine-learning surrogate model for optimizing carbon-nanotube-based air-filled waveguides at D-band frequencies

Article Title: Generative surrogate modelling for the efficient design optimization of an RF component

Article References: Kumar, A., Mailhé, C., Mounayer, J., Baillargeat, D., & Chinesta, F. (2026). Generative surrogate modelling for the efficient design optimization of an RF component. Advanced Modeling and Simulation in Engineering Sciences, 13(1), Article 21. https://doi.org/10.1186/s40323-026-00340-x

Image Credits: AI Generated

DOI: 10.1186/s40323-026-00340-x

Keywords: machine learning, surrogate modelling, autoencoder, RF design, waveguides, carbon nanotubes, D-band, 6G, HFSS, design optimization, electromagnetic simulation, principal component analysis