VERITAS project could change the way scientists secure AI
The AI models, datasets, and automated systems researchers depend on can be compromised in ways conventional cybersecurity tools aren’t designed to detect. A new project called VERITAS (VERified Infrastructure for Trustworthy AI in Science) aims to close that gap by establishing AI Assurance as a core function of scientific research infrastructure.
Led by Anita Nikolich, research scientist and director of research and technology innovation at the University of Illinois School of Information Sciences, the initiative has received a three-year, $896,000 grant from the US National Science Foundation’s Cybersecurity Innovation for Cyberinfrastructure program. The project brings together experts in adversarial AI, research cyberinfrastructure, data science and workforce development to address security risks unique to AI-powered science.
“We cannot simply bolt traditional cybersecurity onto AI-driven science,” said Nikolich. “When a poisoned dataset or backdoored model produces an answer that looks plausible but is subtly wrong, no firewall or virus scanner is likely to catch it. The researchers doing our most important scientific work deserve assurance that the AI systems they rely on are documented, tested, and behaving as intended.”
Nikolich said VERITAS is built to fold AI Assurance into the research infrastructure scientists already use, so scientists don’t need to become cybersecurity experts and cybersecurity teams don’t need to become machine-learning specialists.
Three pieces of the plan
VERITAS is structured around three connected efforts.
The first is documentation. The project will pilot standardized model cards and dataset datasheets for large scientific computing allocations. These are records that lay out where a model or dataset came from, how it was built or altered, what it’s meant for, and what its limits are, making it possible to trace problems back through a workflow.
The second is a new operational role, an AI Assurance Engineer, piloted at the National Center for Supercomputing Applications. That person would review technically novel AI projects before they go live, scan model files for unsafe or malicious behavior, check software for vulnerabilities, and look at how much autonomy agents are given.
The third is education. Through the National Data Platform Education Hub, VERITAS will build hands-on challenges that teach students to spot poisoned data, inspect suspect models, evaluate agent permissions, and find weak points in scientific AI pipelines.
Bringing red-teaming into the lab
AI red-teaming, deliberately probing a system to expose its weaknesses, is common in the tech industry. However, it has rarely been applied to scientific research infrastructure, where compromised AI models or datasets can influence research without obvious warning signs. VERITAS is among the first projects to bring red-teaming into research computing.
“AI systems can fail in ways that are difficult to distinguish from legitimate scientific results,” noted Nikolich. “Proactive red teaming allows us to identify those weaknesses before a vulnerable model or agent becomes embedded in a research pipeline. The objective is to help research teams make their systems more trustworthy and resilient.”
Beyond the technical work, the project has a workforce component. Students taking part in the challenges will work with scientific models, datasets, and infrastructure through the National Data Platform, and get exposure to responsible disclosure practices along the way.
“AI is now part of the scientific workflow. We need to protect its integrity just as seriously as we protect the networks and computing systems around it,” Nikolich concluded.
If successful, VERITAS could help establish AI assurance as a standard component of research cyberinfrastructure, providing universities and research institutions with a framework for evaluating the security and trustworthiness of AI systems before they become embedded in critical scientific workflows.