Artificial intelligence is becoming a core instrument of scientific discovery, but it introduces a vulnerability that conventional cybersecurity often misses: models, datasets, and automated agents can be subtly compromised while still appearing “safe.” A system may evade malware detection, pass functional tests, and still produce results that are unreliable in ways that look like ordinary scientific variation.
A new National Science Foundation project, VERITAS (VERified Infrastructure for Trustworthy AI in Science), led by principal investigator Anita Nikolich, will tackle this blind spot by making AI Assurance a foundational capability of scientific research infrastructure. Funded for three years with $896,000 through the NSF Cybersecurity Innovation for Cyberinfrastructure program, VERITAS convenes specialists in adversarial AI, research cyberinfrastructure, data science, and workforce development.
The project’s premise is that AI-enabled research can fail without triggering alerts designed for unauthorized access or data theft. For example, a poisoned dataset may preserve statistical properties while shifting learned behavior. Likewise, a backdoored model can operate normally until a specific input activates hidden behavior. Autonomous agents add another layer of risk: over-permissioned systems may alter data, invoke lab-connected tools, or steer workflows without leaving obvious “security incident” traces.
Nikolich argues that simply “bolting on” traditional cybersecurity is insufficient. When a manipulated input yields plausible outputs that are subtly wrong, the usual defenses—firewalls, virus scanners, and access controls—may never notice. Researchers therefore need assurance mechanisms that validate that AI components are documented, tested, and behaving as intended before they enter high-impact pipelines.
VERITAS proposes three connected strategies to address these threats. First, it will pilot standardized documentation approaches akin to nutrition labels: model cards and dataset datasheets that describe provenance, intended use, and known limitations to support reproducibility and traceability. Second, it will pilot an AI Assurance Engineer role at the National Center for Supercomputing Applications (NCSA), responsible for reviewing novel projects, scanning model files for unsafe behavior, assessing software vulnerabilities, and evaluating risks tied to autonomous agents. Third, via the National Data Platform (NDP) Education Hub, it will create hands-on challenges that train students to detect poisoned data, inspect potentially compromised models, and evaluate agent permissions.
A key technical theme is proactive AI red-teaming—systematically stressing models and workflows before an attacker’s manipulation causes scientific failure. In research contexts, adversarial weaknesses can be difficult to distinguish from legitimate results, so VERITAS adapts red-teaming to scientific cyberinfrastructure with the aim of improving resilience.
Just as important, VERITAS targets the workforce. Participants will practice with realistic scientific models and infrastructure while learning responsible disclosure practices, helping build expertise at the intersection of machine learning, cybersecurity, and scientific computing.
By embedding documentation, security review, adversarial assessment, and training into everyday research infrastructure, VERITAS aims to deliver a replicable AI Assurance model that can scale from individual supercomputing centers to national research environments.
Subject of Research: Trustworthy AI in scientific research infrastructure, including model/data integrity and adversarial risk
Article Title: VERITAS Builds AI Assurance for Trustworthy Science
News Publication Date:
Web References:
References:
Image Credits:
Keywords
AI assurance, adversarial AI, red teaming, model cards, dataset datasheets, dataset poisoning, backdoored models, autonomous agents, scientific cyberinfrastructure, cybersecurity innovation
Tags: adversarial AI in scientific discoveryAI assurance in researchAI security in scientific researchAI system integrity and reliabilitychallenges of AI cybersecurity in sciencecybersecurity for AI modelsNSF VERITAS project on AI safetyprotecting automated research agentsrisks of poisoned datasets and backdoored modelssecuring AI-enabled scientific workflowstrustworthy AI infrastructurevulnerabilities in AI datasets and models
