Rigorous hypothyroidism guidelines hedge while weaker ones give decisive advice

The most methodologically rigorous clinical guidelines for adult hypothyroidism tend to give the most cautious, conditional advice, while lower-scoring documents often deliver the most decisive directives, according to a systematic appraisal published in Frontiers in Endocrinology. The study applied the Appraisal of Guidelines for Research and Evaluation II instrument, known as AGREE II, to 15 adult hypothyroidism guidelines and standard documents published between 2012 and 2025, and found that only eight of the 15, or 53 percent, earned an unqualified recommendation for use. Mean overall quality scores ranged from 4.77 to 6.25 on a seven-point scale, and the highest-scoring documents, including the BMJ Rapid Recommendations of 2019, the joint American Association of Clinical Endocrinologists and American Thyroid Association guideline of 2012, and the 2025 European Thyroid Association guideline, generally avoided firm treatment mandates in contested areas such as the thyroid-stimulating hormone range of 4 to 10 milli-international units per liter.

The research team, led by Peng Zhi of the Department of General Practice at Henan Provincial Chest Hospital in Zhengzhou, China, set out to test an assumption that clinicians often make implicitly: that guidelines built through more rigorous methods provide more trustworthy recommendations. That assumption has been under strain because adult hypothyroidism guidelines disagree on screening strategy, on the treatment threshold for subclinical disease, and on whether levothyroxine should be combined with liothyronine, the synthetic versions of the two main thyroid hormones. Although pregnancy-specific hypothyroidism guidelines had previously been appraised with AGREE II, no earlier study had applied the instrument to adult guidelines outside pregnancy, including recent documents such as the 2025 European Thyroid Association guideline. The new appraisal was registered prospectively with PROSPERO under registration number CRD420261487294 and was reported according to the PRISMA 2020 statement for systematic reviews.

The search covered PubMed, Web of Science, Scopus, CNKI, and VIP from January 1, 2010 to August 25, 2026, and returned 295 records in total, of which 113 came from PubMed, 72 from Web of Science, 69 from Scopus, 21 from CNKI, and 20 from VIP. Eligible documents were clinical practice guidelines or systematically developed standard documents covering the diagnosis, screening, or management of adult hypothyroidism, published in Chinese or English. Pregnancy-specific, pediatric, and congenital guidelines were excluded, as were guidelines for other thyroid conditions, standalone position statements without formal recommendations, narrative reviews, primary studies, and meeting abstracts. After duplicate removal and screening, 15 documents met the criteria: 12 in English and 3 in Chinese. The final set included the 2025 European Thyroid Association guideline on levothyroxine monotherapy, the 2012 AACE/ATA guideline, the 2014 American Thyroid Association guideline, the 2013 and 2018 European Thyroid Association guidelines, the 2019 BMJ Rapid Recommendations, the 2025 Italian guideline, the 2024 American Society for Reproductive Medicine guideline, the 2016 British Thyroid Association guideline, the 2013 Brazilian guideline, the 2012 European Thyroid Association document on levothyroxine plus liothyronine, the 2021 LT4/LT3 consensus document, the 2017 Chinese Medical Association guideline, and two Chinese primary care documents from 2019 and 2021.

Four raters independently scored all 23 AGREE II items on a one-to-seven scale after a training session and a calibration exercise on one guideline. The panel comprised one associate chief physician of general practice, two attending physicians of general practice, and one rehabilitation medicine supervisor nurse with methodological training, a composition chosen deliberately because hypothyroidism is managed mainly in primary care. The instrument assesses six domains: scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, and editorial independence. Standardized domain scores were computed as the obtained score minus the minimum possible, divided by the maximum minus the minimum possible, and multiplied by 100 percent. Each rater also assigned an overall quality score calculated as the mean of that rater’s 23 item scores, and the guideline-level overall score was the mean across the four raters. A guideline was classified as recommended for use when its mean overall score reached at least 5.5, as uncertain between 4.0 and 5.49, and as not recommended below 4.0.

The BMJ 2019 Rapid Recommendations led the field with an overall score of 6.25, followed by AACE/ATA 2012 at 6.20 and ETA 2025 at 6.10. ATA 2014 scored 5.92, ETA 2018 scored 5.85, the Italian 2025 guideline scored 5.83, ETA 2013 scored 5.76, and ASRM 2024 scored 5.58, all above the threshold for an unqualified recommendation. The remaining seven documents fell into the uncertain category: the Brazilian 2013 guideline at 5.37, CMA 2017 at 5.29, the LT4/LT3 Consensus 2021 at 5.22, ETA 2012 at 5.15, BTA 2016 at 5.09, and the two Chinese primary care guidelines at 4.86 and 4.77, the lowest scores in the set. Averaged across the 15 documents, mean domain scores ranged from 67.8 percent for stakeholder involvement to 83.0 percent for scope and purpose, and only scope and purpose exceeded 80 percent.

Inter-rater reliability, measured with two-way random-effects, absolute-agreement intraclass correlation coefficients, was moderate to good when the four raters’ scores were averaged. Average-measure ICC values ranged from 0.651 to 0.895 across the six domains, with stakeholder involvement highest at 0.895 and scope and purpose lowest at 0.651. Single-rater ICC values told a different story: the estimate for scope and purpose was 0.318 and for clarity of presentation 0.348, both below 0.50 and in the poor range, a reminder that judgments about how explicitly objectives are stated and how easily recommendations can be identified are subjective. The authors argue that multiple raters, standardized training, and calibration remain the sensible remedy for this variability.

On the clinical questions, recommendations were more consistent for screening strategy and first-line treatment than for treatment thresholds. Most guidelines favored case finding over universal screening and identified levothyroxine monotherapy as standard replacement therapy. The main disagreements concerned subclinical hypothyroidism, a mild biochemical state in which thyroid-stimulating hormone is elevated while the thyroid hormone levels themselves may still be normal. Most guidelines reserved treatment for TSH values of at least 10 milli-international units per liter or individualized the 4 to 10 range. ETA 2013 recommended a more active approach in younger patients or those positive for thyroid peroxidase antibodies, whereas BMJ 2019 was explicitly conservative. Policies on combination therapy with levothyroxine plus liothyronine also diverged: ATA 2014, ETA 2025, and the Chinese primary care guidelines advised against routine use, while ETA 2012 and the LT4/LT3 consensus permitted conditional use in selected patients. ASRM 2024 set a lower treatment threshold for infertile women, at TSH 2.5 to 4.0 milli-international units per liter, than the threshold used for the general population.

The appraisal also traced how guideline series evolved over time and found two distinct updating models. In the European Thyroid Association series, the 2025 guideline replaced conditional acceptance of combination therapy with explicit levothyroxine monotherapy, a shift the authors describe as driven by new evidence. In the Chinese primary care series, the 2019 and 2021 versions simplified the treatment threshold to TSH of at least 10 milli-international units per liter and refined medication management and referral pathways, an evolution the authors characterize as simplification-driven. An exploratory, ad hoc framework called EndoCompass was used to scan full texts for emerging topics: artificial intelligence or machine learning terms appeared in 10 of 15 guidelines, laboratory standardization terms in 10 of 15, and personalized reference interval terms in 14 of 15, but most mentions were passing references rather than implementation-oriented guidance.

Citation impact did not track closely with quality. Citation counts were available for 13 of the 15 guidelines, retrieved from OpenAlex on August 27, 2026 with Crossref as a fallback; the two Chinese primary care guidelines were not indexed in the international databases. Citations ranged from 3 for the Italian 2025 guideline to 1,896 for ATA 2014, with a median of 171. The highest-scoring document, BMJ 2019, had 234 citations. The Spearman rank correlation between overall quality and citation count was weak to moderate, at rho = 0.368 with n = 13. The authors stress that this analysis was exploratory, because citation counts are strongly influenced by publication age, journal visibility, and clinical controversy, and they report it chiefly to caution against using citation counts as a proxy for guideline quality. ATA 2014 was the most cited guideline but not the highest scoring, and several high-quality guidelines left the TSH 4 to 10 range unspecified while lower-quality primary care documents provided operational thresholds.

The authors label the central pattern a quality-recommendation paradox and are careful about what the term means. It describes an observation, not a logical contradiction or a causal relationship: higher methodological rigor accompanied more conservative recommendations, while lower-quality documents supplied more decisive directives. GRADE-based guidelines such as BMJ 2019 make uncertainty explicit, whereas consensus documents often resolve uncertainty through expert opinion. The practical message for clinicians is to distinguish decisiveness from trustworthiness, since conditional recommendations can reflect a more transparent appraisal of evidence while firm recommendations may rest on weaker synthesis, and to pair guideline appraisal with explicit evidence summaries in gray-zone decisions such as the TSH 4 to 10 range. The authors also note that AGREE II assesses process and reporting quality rather than the clinical validity of recommendations, that no patient-outcome or adherence data were available to link higher quality to better outcomes, that only three Chinese guidelines were included so the language-stratified comparison is descriptive, and that patient and public representatives were not part of the appraisal panel. For guideline developers, the study points to concrete gaps: stakeholder involvement and applicability scored lowest on average, follow-up frequency and referral pathways are often missing, and fixed review cycles of roughly three to five years would let new evidence enter systematically. The authors propose a testable prediction for future work: conditional recommendations from rigorous guidelines should remain more stable than decisive directives from consensus documents when new evidence emerges.

Subject of Research: Systematic appraisal of the methodological quality, recommendation consistency, and citation impact of adult hypothyroidism clinical practice guidelines

Article Title: Quality-recommendation paradox in adult hypothyroidism guidelines: a systematic appraisal of methodological quality, recommendation consistency, and citation impact

Article References: Zhi, P., Yan, X., Zhang, C., & Zhao, J. (2026). Quality-recommendation paradox in adult hypothyroidism guidelines: a systematic appraisal of methodological quality, recommendation consistency, and citation impact. Frontiers in Endocrinology, 17, 1998518. https://doi.org/10.3389/fendo.2026.1998518

Image Credits: AI Generated

DOI: 10.3389/fendo.2026.1998518

Keywords: hypothyroidism, clinical practice guidelines, AGREE II, guideline appraisal, subclinical hypothyroidism, levothyroxine, TSH, evidence-based medicine, citation impact, inter-rater reliability, quality-recommendation paradox, systematic review