PCI SSC calls for human approval of AI agent actions involving cardholder data
The PCI Security Standards Council (PCI SSC) has published Security Considerations for AI Systems, guidance covering the protection of data supplied to AI systems in payment environments and defenses against AI-assisted attacks.
Developed with industry stakeholders, the document addresses governance, deployment, access controls, testing and the application of PCI standards. Its recommendations are advisory, and existing PCI requirements take precedence.
“As AI is increasingly used in payment environments, there is an obligation for all parties to ensure the technology is used responsibly,” said Gina Gobeyn, Executive Director, PCI SSC. “This additional guidance provides a practical starting point for secure AI implementation.”
PCI DSS scoping for AI systems (Source: PCI SSC)
Define access and accountability
The Council recommends defining an AI system’s purpose, permissions and data access before selecting or deploying it. Its least agency approach limits each system to the access and capabilities needed for its assigned tasks.
A suitable human individual should formally accept responsibility for AI output, and organizations should specify which actions require human approval. Access restrictions should be enforced through independent controls, such as identity-management policies and network isolation.
Organizations should avoid combining sensitive data access, external communications and unrestricted input from untrusted sources in one AI system. Where a workflow needs all three, the guidance recommends separating responsibilities among agents with different permissions.
An AI inventory and bill of materials should identify models, versions, hosting, integrations, data-use and retention policies, and intended users. An acceptable-use policy and technical controls should help identify and restrict shadow AI, tools employees deploy or use without approval.
Test controls and prepare for failure
Safeguards should be tested before extensive functional or user-acceptance testing. This includes adversarial testing to determine whether restrictions can be bypassed.
Monitoring and revalidation should continue throughout deployment to detect changes in behavior. Review processes should also account for excessive trust in AI output, which can cause reviewers to overlook mistakes.
The guidance describes human approval for each task and an alternative in which AI systems perform authorized actions under monitoring without approval for every individual action. For monitored autonomy, organizations should define permitted actions, approval requirements, shutdown triggers and procedures for reversing changes. A suitable human individual should remain ultimately responsible.
For agents with access to cleartext cardholder data, explicit human approval for any actions involving that data is recommended.
Protect sensitive data and credentials
AI systems should not handle, generate or manage unprotected high-impact secrets, including passwords and cryptographic keys. Credentials should be managed through secrets-management tools and kept out of source code, prompts, AI context, outputs and logs.
When random values are required, the Council recommends using a trusted random-number generator. Values used for passwords or cryptographic keys may need to remain outside the AI system.
Organizations should use encrypted or tokenized payment data where possible. Data-loss prevention controls should operate independently of the AI, and logs should support investigations without retaining sensitive payment information.
PCI scoping should consider an AI system’s deployment, isolation, access to account data, including data used for training, and ability to affect the security of cardholder-data systems. Access to encrypted or tokenized data together with tools that decrypt or detokenize it should be treated as access to readable data.
Address AI-assisted attacks
The Council warns that AI can accelerate vulnerability discovery, exploit development and social engineering. It recommends ongoing vulnerability monitoring and a process for prioritizing findings and patches.
Defenses include limiting services and permissions, isolating legacy systems, using phishing-resistant authentication, encrypting sensitive data and containing the impact of breaches.
AI-generated code and patches should undergo security and functional testing. Reviews should check for embedded credentials, unsuitable dependencies, newly introduced weaknesses and whether fixes address the underlying problem.
One deployment example divides vulnerability management among agents that identify issues, prepare patching plans, test changes and deploy approved updates. Permissions are assigned by role, and rollback procedures are tested before deployment.
Assess external providers
External AI providers with access to sensitive data should be assessed under applicable third-party service provider requirements. Agreements should address responsibility for sensitive information, explicitly prohibit using the organization’s data for AI training, provide visibility into subcontractors and set breach notification terms.
Incident response plans and periodic testing should cover prompt injection, model poisoning, actions outside approved scope and unauthorized AI tools accessing sensitive data.
