AI Reveals Why Pathologists Disagree on HER2 Scoring in Breast Cancer

An artificial intelligence model designed to score HER2 immunohistochemistry in breast cancer has exposed a striking weakness in one of the most consequential tests in oncology: pathologists themselves often cannot agree on what they are seeing, and the AI can now predict exactly which samples are most likely to trigger that disagreement. The finding, published in BMC Cancer by a team of Brazilian researchers, suggests that the variability long tolerated as an unavoidable feature of diagnostic pathology may be rooted in a measurable property of the tumors themselves — the spatial heterogeneity of HER2 protein expression within a single tissue sample.

HER2 testing sits at the heart of modern breast cancer treatment. The HER2 protein, encoded by the ERBB2 gene, drives one of the most aggressive molecular subtypes of the disease, and its detection determines eligibility for targeted therapies ranging from trastuzumab to the newer antibody–drug conjugates. The stakes have grown even higher since the recognition of HER2-low tumors, which express the protein at levels below the traditional positive threshold but still respond to certain conjugate drugs. That expansion has pushed the scoring system into a gray zone where small differences in interpretation can change a patient’s entire treatment trajectory, making reproducibility not an academic nicety but a clinical imperative.

The study, led by Pedro Ferrari and colleagues at the D’Or Institute for Research and Education and the Rede D’Or pathology network in São Paulo, set out to quantify just how reproducible HER2 immunohistochemistry scoring really is, and to test whether an AI model could shed light on why disagreements arise. The researchers recruited both generalist and specialist pathologists from Rede D’Or centers across Brazil and asked them to assess digitized whole slide images of 126 breast cancer samples stained for HER2. In a clever design choice, the same images were presented to each pathologist twice, separated by an interval of one month, allowing the team to measure not only how much pathologists disagreed with one another but also how much each individual disagreed with his or her own earlier judgment.

The results were sobering. The median intraobserver agreement — the consistency of each pathologist with their own prior scoring — was just 66.67 percent. Agreement between pathologists was similarly modest, with a median interobserver agreement of 67.65 percent, and full-threshold agreement above 85 percent was achieved in only about a quarter of the samples, 25.4 percent. In other words, for roughly three out of four tumors, the panel of experts could not converge on a single interpretation. Perhaps most telling was the comparison with the machine: the median agreement between pathologists and the AIM-HER2 model, developed by PathAI in Boston, was 60.8 percent, indicating that the algorithm’s readings diverged from human consensus nearly as often as humans diverged from each other.

Yet the study’s most original contribution lies in what happened next. Rather than treating the disagreement as statistical noise, the researchers mined the AI model’s spatial breakdown data — its ability to map HER2 staining intensity across different regions of the same slide — to compute a measure of intra-sample heterogeneity. When they correlated this measure with the agreement metrics, a clear pattern emerged: samples with lower heterogeneity, meaning the staining pattern was relatively uniform across the tissue, were associated with significantly higher agreement among pathologists. Conversely, tumors in which the AI detected patchy, mixed, or geographically variable staining were precisely the ones where human observers clashed, both with each other and with themselves a month earlier.

This correlation across all agreement metrics — intraobserver, interobserver, and human–AI agreement — points to a mechanistic explanation for a problem that has plagued HER2 testing since its inception. Immunohistochemistry scoring depends on the pathologist integrating membrane staining intensity and the proportion of positive tumor cells across a slide, a task that becomes genuinely ambiguous when a tumor contains intermingled areas of 1+, 2+, and 3+ staining. In such cases, different observers may legitimately weight different fields of view differently, and even the same observer may make different sampling decisions on different days. The AI’s spatial analysis essentially quantifies that ambiguity, converting a subjective sense that a case is difficult into a reproducible numerical score.

The technical architecture underlying this capability is worth appreciating. Modern computational pathology models such as AIM-HER2 are trained on large corpora of digitized whole slide images, learning to recognize staining patterns at the level of individual cells and tissue regions rather than issuing a single holistic judgment. This granular output is what enabled the Brazilian team to dissect each slide into spatially resolved components and derive a heterogeneity score. Notably, the authors report that PathAI provided the model and technical support but did not interfere with the design, execution, or writing of the study, and the project was entirely self-funded, with the authors declaring no competing interests.

The clinical implications are substantial. With antibody–drug conjugates now indicated for HER2-low disease, the boundary between HER2-negative and HER2-low has become a treatment decision of real consequence, and it sits squarely in the region where observer variability is worst. A patient whose tumor is scored 1+ by one pathologist and 2+ by another may either receive or be denied therapy with demonstrated benefit. The study’s authors argue that this variability reflects a potential limitation of current scoring practices, and they propose a practical role for AI: rather than replacing pathologists, models like AIM-HER2 could serve as a triage tool, flagging samples whose spatial heterogeneity scores predict diagnostic discordance so that those cases receive additional scrutiny, second reads, or reflex testing by more definitive methods such as in situ hybridization.

Such a workflow would represent a meaningful shift in how digital pathology is deployed. Much of the current enthusiasm for AI in diagnostics centers on automation — having algorithms perform repetitive tasks at scale. This study suggests a complementary value proposition: AI as a quality-control instrument that identifies the cases where human judgment is least reliable. Because the heterogeneity signal is computed from the same slide images already generated in routine digital workflows, implementing such triage would not require new tissue, new stains, or new instrumentation, only the computational infrastructure to run the model and a protocol for handling flagged cases.

The research also carries a broader message about the nature of diagnostic expertise. The finding that pathologists disagreed with their own month-earlier assessments nearly as often as they disagreed with colleagues underscores that scoring variability is not primarily a matter of individual skill or training level, since generalists and specialists alike participated. Instead, it reflects genuine ambiguity in the biological signal itself — tumors that are, in a measurable sense, internally inconsistent. Recognizing that ambiguity, and quantifying it with tools that never fatigue and never second-guess themselves, may be the key to making HER2 testing, and biomarker assessment more generally, as reproducible as the therapies it is meant to guide. The Brazilian team’s work offers an early but compelling demonstration that the path to diagnostic consistency may run not through more stringent human consensus, but through machines that can see what humans cannot agree on.

Subject of Research: Artificial intelligence assessment of HER2 immunohistochemistry heterogeneity and pathologist agreement in breast cancer diagnostics

Article Title: Artificial Intelligence Model’s assessment of intra-sample heterogeneity of HER2 IHC in breast cancer is related to interobserver and intraobserver agreement among pathologists

Article References: Ferrari, P., Petaccia de Macedo, M., Werneck da Cunha, I., Soares-Souza, G. B., Soares, F. A., Breast Cancer Cooperative Study Group, Alencar Giongo, A., Da Silva Camacho, A. H., Souza, A., de Brito Rodrigues, A. L., da Cunha Mercante, A. M., Karlinski, Â., Monnerat, A. C., de Almeida Verdolin, A., Castro, A. A., Nicolli, A., Volpato, A. H., Aquino Garibaldi, B., de Carvalho Santos, C., … Peixoto, R. (2026). Artificial Intelligence Model’s assessment of intra-sample heterogeneity of HER2 IHC in breast cancer is related to interobserver and intraobserver agreement among pathologists. BMC Cancer. https://doi.org/10.1186/s12885-026-17049-0

Image Credits: AI Generated

DOI: 10.1186/s12885-026-17049-0

Keywords: HER2, breast cancer, artificial intelligence, digital pathology, immunohistochemistry, interobserver agreement, intraobserver agreement, tumor heterogeneity, pathology, AIM-HER2, antibody-drug conjugates, diagnostic reproducibility