The 6 challenges your business will face in implementing MLSecOps
Organizations that don’t adapt their security programs as they implement AI run the risk of being exposed to a variety of threats, both old and emerging ones.
MLSecOps addresses this critical gap in security perimeters by combining AI and ML development with rigorous security guidelines. Establishing a robust MLSecOps foundation is essential for both proactively mitigating vulnerabilities and simplifying the remediation of previously undiscovered flaws.
AI/ML systems must remain trustworthy, resilient, and secure. MLSecOps can help security teams bake in protections as their operations scale, according to a white paper from the Open Software Security Foundation.
MLSecOps implementation challenges
As organizations start to establish more robust ML and AI security, they will face six major challenges. It’s important that leadership and security strategists know how to identify the problems and what to do if they suspect risks in their models.
1.Defining the unique, changing threat landscape
Many of the security practices of DevSecOps, a cousin to MLSecOps, fall flat when applied to the AI threat landscape.
DevSecOps is built around conventional software vulnerabilities that security practitioners have understood for years: backdoors, bugs, glitches, etc. AI and ML systems, by virtue of being new to most organizations, come with a host of new threat vectors for security teams to consider in addition to their existing processes. These might include data poisoning, adversarial inputs, model theft or tampering, or even privacy-specific attacks like model inversion and membership inference.
Defending against these new threats means creating controls designed specifically for the ML lifecycle. Security professionals must be prepared for repeated, probing attacks instead of covert one-time hacking attemots. Stress testing models is crucial.
2. The hidden complexity of continuous training
AI models evolve, which adds another layer of complexity to MLSecOps security. Each time a model is trained and retrained on data, new vulnerabilities are potentially being introduced to the ML ecosystem. This means that a model can be secure one day and not the next.
To combat this, each retraining of the model should be treated as a net-new product version. IT and security leadership might even consider creating materials to accompany the latest version of their model – much like app makers share version details with each new release – to outline what data the model is trained on and how this version differs from the last. If model training is not continuously tracked, the security of MLSecOps programs will drift over time.
3. Managing opacity and interpretability in ML models
ML models are often “black boxes”, even to their creators, so there’s little visibility into how they arrive at answers. For security pros, this means limited ability to audit or verify behavior – traditionally a key aspect of cybersecurity.
There are ways to circumnavigate this opacity of AI and ML systems: with Trusted Execution Environments (TEEs). These are secure enclaves in which organizations can test models repeatedly in a controlled ecosystem, creating attestation data.
TEEs enable organizations to use attestation data to build pre-established standards and guides for appropriate model behavior, allowing researchers of the model to decide whether an AI system is trustworthy. While the TEE doesn’t make for transparent models, it ensures that the risks of erratic or unknown behaviors do not reach production environments.
4. Creating a secure training data pipeline
Models are not static and are shaped by the data they ingest. Thus, data poisoning is a constant threat for ML models that need to be retrained.
Organizations must embed automated checks into the training process to enforce a continuously secure pipeline of data. Using information from the TEE and guidelines on how models should behave, AI and ML models can be assessed for integrity and accuracy each time they are given new information.
The same can be said of the data users are giving the model as well, and security leaders should be regularly enforcing checks on the resilience of their MLSecOps program.
5. Model provenance and reproducibility
Updated training data, configuration drift and evolving libraries make it difficult to track the stability and performance of a model. Prior versions of the model can feel impossible to track or even reproduce, especially when a problem arises in model decisioning.
The solution is to create a lineage of a model, which will allow security teams to have visibility into version control and changes to the model over time. This might include datasets, detailed snapshots of training configurations and dependencies of a model. When exact reproduction isn’t possible due to the changing nature of the model and its data, organizations should aim for approximate reproducibility. The ability to retest a model and achieve comparable results will preserve trust in that model’s progression.
6. Difficulties in performing risk assessment
Risk assessment frameworks that work for traditional software will not be applicable to the changeable nature of AI and ML programs. Traditional assessments fail to account for tradeoffs specific to ML, e.g., accuracy vs fairness, security vs explainability, or transparency vs efficiency.
To navigate this difficulty, businesses must be evaluating models on a case-by-case basis, looking to their mission, use case and context to weigh their risks. This is certainly not a conventional way to conduct risk assessment, but decisions about the model’s operation must be guided by a culture of prioritization. Cross-functional collaboration is also key to assessment, and pulling in ML engineers, security teams and policy leaders to oversee the model will heighten security.
Hitting the moving target
If a business wants to capitalize on AI and ML workflows, they must consider MLSecOps as an integral part of that program as well.
These six challenges highlight the complexity of AI security, but they also create an opportunity for organizations to build truly airtight, trustworthy MLSecOps.
The first step towards security is acknowledging the limitations of existing security practices and building new rules to address the uniqueness of AI and ML. Eventually, organizations will view MLSecOps as a cornerstone of responsible AI deployment, but that requires security leaders taking action today for a higher standard for safety and trust.