PharmAI: Small Language Models Master Pharmaceutical QA on a 6 GB GPU

…

…

The distinction between small language models and their larger counterparts is more than a matter of parameter counts. Models in the one-to-two billion parameter range occupy a practical sweet spot for domain adaptation: they retain enough linguistic competence to follow instructional prompts and compose coherent multi-sentence answers, yet remain small enough that their weights can be quantized to four bits and held comfortably within the memory envelope of a laptop-class graphics card. This is why the three models examined in the study—SmolLM2-1.7B-Instruct, Qwen2.5-1.5B-Instruct, and Llama-3.2-1B-Instruct—could all be fine-tuned under identical quantized low-rank adaptation settings on a single 6 GB GPU, a constraint that would make full-parameter fine-tuning of even these compact architectures infeasible. Quantized Low-Rank Adaptation works by freezing the base weights in a compressed four-bit NF4 representation and training only small low-rank adapter matrices inserted into attention and feed-forward layers, which reduces the trainable parameter footprint by orders of magnitude while preserving most of the model’s capacity to absorb new domain knowledge.

The choice of NF4 quantization deserves particular attention. Four-bit NormalFloat is a data type designed specifically for normally distributed weights, and it tends to preserve the information content of quantized neural network parameters better than standard integer formats at the same bit width. Combined with double quantization of the quantization constants themselves, it allows a 1.7-billion-parameter model to occupy roughly one to two gigabytes of memory in its frozen form, leaving the remainder of the GPU budget for activations, gradients on the adapter weights, and optimizer state. For researchers and students without access to data-center hardware, this class of techniques effectively democratizes domain-specific model development, shifting the bottleneck from compute procurement to data curation and evaluation design.

Completion-only loss masking addresses a subtle but consequential failure mode in fine-tuning language models on question-answer pairs. When the training objective computes loss over the entire sequence, including the question tokens, the model is implicitly rewarded for memorizing and reproducing question phrasings rather than for producing accurate answers. In extreme cases, a small model fine-tuned naively on textbook text learns to continue passages in the style of the source material rather than to respond to prompts. By masking the loss so that gradients flow only through the completion tokens, the training signal is concentrated on answer generation, which aligns the optimization target with the actual task the model must perform at inference time. The ablation analysis in this study validates this design choice empirically, and it echoes a broader lesson from the instruction-tuning literature: what the model is trained to predict matters as much as what data it is trained on.

The retrieval side of the pipeline illustrates how classical information retrieval engineering remains highly relevant in the era of generative models. The system uses Sentence-Transformer embeddings, which produce dense vectors capturing semantic similarity between passages and queries, and indexes them with FAISS using a flat inner-product search structure. Because the vectors are L2-normalized before indexing, inner-product ranking is mathematically equivalent to cosine similarity ranking, which is the standard similarity measure for dense retrieval. The flat index performs exact search rather than approximate nearest-neighbor search, which is a deliberate trade-off: for a corpus of roughly 7,370 passages, exact search is fast enough on a CPU that the approximation errors of quantized or graph-based indexes offer no benefit. This stands in contrast to production-scale systems handling millions or billions of documents, where approximate indexes such as IVF or HNSW become necessary. The lesson is that retrieval architecture should scale with the corpus, not with fashion.

The retrieval metrics reported—Recall@1 of 0.517, Recall@3 of 0.732, Recall@5 of 0.797, and a Mean Reciprocal Rank of 0.628—can be interpreted in practical terms. Recall@K measures the fraction of queries for which the correct passage appears somewhere in the top K retrieved results, so a Recall@5 near 0.8 means that in four out of five cases the system surfaces the relevant textbook passage within its first five candidates. Mean Reciprocal Rank, which averages the reciprocal of the rank of the first correct result, sits between the Recall@1 and Recall@3 values, consistent with a distribution in which correct passages frequently appear in the second or third position. Because all three language models shared the same retrieval index and embedding model, their identical retrieval scores are expected and serve as a useful control: they isolate the generation stage as the sole source of performance differences between the fine-tuned models.

The generation metrics tell a more differentiated story. ROUGE-1, ROUGE-2, and ROUGE-L measure lexical overlap between generated and reference answers at the level of unigrams, bigrams, and longest common subsequences respectively. These metrics are cheap to compute and correlate reasonably well with human judgments for extractive or tightly constrained tasks, but they penalize valid paraphrases and cannot detect factual errors that happen to use different wording. BERTScore and cosine-based semantic similarity partially compensate by comparing contextual embeddings of the generated and reference texts, rewarding answers that are semantically equivalent even when phrased differently. The finding that Qwen2.5-1.5B-Instruct, after fine-tuning, achieved a ROUGE-1 of 0.47, ROUGE-2 of 0.25, ROUGE-L of 0.38, and BERTScore of 0.72—clearly ahead of the roughly 0.35 ROUGE-1 and 0.66 BERTScore posted by the other two models—suggests that the base model’s pretraining mixture and multilingual exposure gave it a stronger foundation for absorbing pharmaceutical terminology, even at a smaller parameter count than SmolLM2.

The category-wise breakdown is arguably the most instructive part of the evaluation. Comparison questions proved the most difficult for the single-passage retrieval architecture, and this difficulty has a structural explanation. A comparison question—asking, for instance, how two drug classes differ in mechanism or how two dosage forms contrast in clinical use—typically requires information distributed across multiple textbook passages, often in different chapters. A retriever that returns one passage per query, however accurate, cannot supply both sides of the comparison, and the generator is forced to answer from partial evidence. Definition and mechanism questions, by contrast, tend to be answered within a single coherent passage, which aligns naturally with the chunk-level retrieval granularity. This observation generalizes beyond pharmaceuticals: any RAG system evaluated on multi-hop or multi-source questions will underperform unless the retrieval stage is extended to multi-passage or iterative retrieval, and the study’s category-wise analysis makes this failure mode visible rather than averaging it away in aggregate scores.

The synthetic QA generation pipeline also merits scrutiny. Automated question generation from textbook content is attractive because manual annotation of 2,144 domain-specific QA pairs would require substantial expert time, but it introduces risks that the authors addressed with five sequential quality filters designed to reject hallucinated or malformed pairs. The four question categories—definition, mechanism, comparison, and clinical application—were chosen to mirror the cognitive structure of pharmaceutical education, ranging from factual recall to relational reasoning to applied judgment. This taxonomy matters for evaluation design: aggregate scores can mask large per-category disparities, and a system that excels at definitions but fails at clinical application is of limited educational value. Reporting per-category results, as this study does, is a practice worth emulating in other domain-specific QA efforts.

The corpus construction details also carry methodological weight. Four pharmaceutical textbooks totaling 3,977 pages were processed into 7,370 semantically coherent chunks, with the chunking pipeline respecting chapter and section boundaries across four different publisher heading conventions. This structure-preserving approach contrasts with naive fixed-length chunking, which can split a drug’s mechanism description mid-sentence or merge unrelated sections, degrading retrieval precision in ways that are difficult to diagnose later. The emphasis on page-aware extraction reflects a practical reality of working with real publisher PDFs, where heading formats vary widely and automated parsing must be robust to inconsistency. For teams building similar pipelines in adjacent domains—nursing, pharmacology, clinical chemistry—the lesson is that preprocessing quality often determines the ceiling on downstream retrieval performance more than model choice does.

It is worth situating these results against the broader landscape of biomedical language models. Large biomedical models fine-tuned on examination-style datasets such as MedQA, or on literature-based datasets such as PubMedQA, have demonstrated strong performance on medical licensing-exam-style tasks, but their training distributions emphasize clinical reasoning and research abstracts rather than the pharmacokinetic detail, dosage-form terminology, and mechanism-of-action content that dominate pharmacy curricula. A pharmacy student asking about first-pass metabolism or the differences between zero-order and first-order kinetics needs answers grounded in the standard textbooks used in their coursework, not in examination question banks. The PharmAI pipeline’s decision to build its corpus directly from those textbooks is therefore not merely a convenience but a deliberate alignment of the system’s knowledge source with its intended audience.

The hardware constraint that motivated much of the design—training and inference on a single consumer laptop GPU—also has implications for reproducibility and adoption. Research results that require multi-GPU clusters are difficult for small academic groups, and especially for students, to replicate or extend. By demonstrating that a complete pipeline from raw PDF ingestion through fine-tuning to RAG-based question answering fits within a 6 GB VRAM budget, the study lowers the barrier for replication at educational institutions in resource-constrained settings. The trade-offs involved—four-bit quantization, adapter-only training, CPU-based exact retrieval—are all individually well-established techniques, but their combination into a coherent, documented workflow under explicit hardware constraints is a contribution in itself.

Several limitations naturally follow from these design choices and point toward future work. The corpus is drawn from four textbooks, so coverage is bounded by those sources, and generalization to questions outside their scope is untested. The retrieval stage returns a single passage, which the comparison-question results show to be a structural bottleneck; multi-passage retrieval, query decomposition, or iterative retrieval strategies are natural extensions. The evaluation relies on automatic metrics against synthetic reference answers, and human expert evaluation of answer factual accuracy—particularly important in a medical domain where plausible-sounding errors carry real risk—remains an important next step. Finally, the identical retrieval scores across models, while methodologically clean, mean the study cannot speak to how model choice might interact with retrieval quality; joint optimization of the two stages is an open question. These caveats notwithstanding, the study offers a concrete, reproducible template for building trustworthy, domain-specific QA systems on hardware that a single researcher or student can already own.

Subject of Research: PharmAI: domain-specific QA generation by adaptive learning and fine-tuning of small language models

Article Title: PharmAI: domain-specific QA generation by adaptive learning and fine-tuning of small language models

Article References: Anantharajan, S., Loganathan, S., Soundarraj, R., & Emerson Nithiyaraj, E. (2026). PharmAI: domain-specific QA generation by adaptive learning and fine-tuning of small language models. Frontiers in Artificial Intelligence, 9, Article 1944175. https://doi.org/10.3389/frai.2026.1944175

Image Credits: AI Generated

DOI: 10.3389/frai.2026.1944175

Keywords: PharmAI, domain-specific, generation, adaptive, learning, fine-tuning, small, language, models, scientific research