What the first year of EU AI Act transparency enforcement could look like

In this Help Net Security interview, Edwin Weijdema, Field CTO at Veeam, answers questions on Article 50 of the EU AI Act and what the first year of enforcement might bring.

He explains why corrective orders will likely outnumber large fines, when an AI agent working through a ticket queue counts as interacting with a person, and how security teams should handle simulated phishing that uses cloned voices. He also weighs where the first enforcement case could start, and the accountability question clients keep asking with no good answer yet.

Article 50 breaches carry exposure up to 15 million euro or three percent of worldwide turnover. What is realistic first-year exposure? Do you expect monetary penalties, or corrective orders and withdrawal requirements?

As we’ve seen with other EU regulations like NIS2 and GDPR, the real-world enforcement, including the fines, will be up to individual member countries to enforce, with procedures varying depending on the country. So, it’s difficult to estimate precisely what sort of exposure and penalties we’ll see during the first year.

That being said, the first year of new regulation penalties can also often be treated as a bit of a ‘bedding in’ period. It’s more than likely that corrective orders will significantly outweigh the number of major financial penalties we see in this first year of enforcement. Especially for those organisations making a genuine effort to comply. Regulators are likely to look at proportionality, scale of impact, whether the breach was intentional or negligent, how quickly the organisation cooperated, and whether basic governance controls were already in place when making these decisions.

That said, we do also often see one or two big headline-making fines to show regulators mean business, but if this does happen, I wouldn’t expect it to land in year one. In fact, in the first year, the bigger practical exposure may well be operational rather than financial. For an organisation, being ordered to suspend, relabel, change, or withdraw an AI-enabled process at speed could be far more disruptive than receiving a fine.

In year one, the bigger risk likely won’t be the fine; it’ll be being told to stop using the system until you can prove it is compliant.

Agentic systems interact with people indirectly, through a ticketing queue, a shared inbox, or a supplier’s procurement portal. Which of those count as interacting directly with a natural person, and have you had to tell a client that their agent crossed a line they did not know existed?

For the EU AI Act, the channel is not decisive. A ticketing queue, shared inbox, or procurement portal does not automatically mean direct interaction with a person, but it can. The key question is whether the AI system itself is communicating with a natural person, or whether there is a human intermediary exercising meaningful review and control.

Under the Act itself, the transparency obligation applies in those situations where a person is interacting with an AI system and needs to be informed that they’re dealing with AI, unless it’s made obvious by the circumstances. So, if an AI drafts a response and a human reviews and sends it, that is a very different risk profile from an AI agent autonomously replying to a customer, supplier, or employee. The latter can start to look like direct interaction, even if it happens through a ticketing system or procurement portal rather than a chatbot window.

So, companies need to make deliberate choices to separate those agents that are internal and those that are customer-facing by setting up appropriate barriers for agents depending on their roles. And those same access and privacy controls should be present across the organisation, not just across the agents. Only then can you avoid any unintended access – it’s all well and good telling an agent ‘don’t go into this room’; you also need to put a lock on the door.

Ultimately, the AI Act does not care whether the interaction happens in a chatbot window or a ticket queue. It cares whether the human is effectively dealing with the machine.

Security teams run simulated phishing and vishing exercises, sometimes cloning an executive’s voice, and the exercise fails if the material carries a label. How have you advised those teams, and what does the internal documentation need to contain to justify the position?

Exercises like these are not automatically exempt from the AI Act’s transparency requirements, and organisations shouldn’t just assume they are. Understandably, there’s a reluctance to label AI-generated phishing emails or cloned voices – it’s not much of an exercise if there’s a big flag all but announcing that it’s a test.

But cloning an executive’s voice is particularly sensitive. If AI is used to make a real person appear to say something they did not say, that can quickly become a deepfake scenario. A security purpose does not automatically create an exemption, and “the exercise works better without disclosure” is not, by itself, a compliance justification.

So, if organisations are deciding not to label these AI-generated elements of these exercises, they should be able to demonstrate that the legal basis and the risk of that decision have been carefully assessed. Going forward, for best practice around these exercises, I’d suggest security teams involve their legal and compliance departments early to document their reasoning as a minimum. I would also include privacy, HR, and, where relevant, works council or employee representative input, especially if the exercise uses a real person’s voice, image, or likeness.

But in most cases, I would advise teams to consider alternatives such as fictional personas, synthetic voices that do not imitate real employees, prior general notice that simulations may use synthetic media, and immediate post-exercise disclosure. The goal is to preserve realism without normalising undisclosed executive impersonation inside the company.

The documentation should be able to show the purpose of the exercise, the scope, what AI tools were used, whether any real person was imitated, what disclosure was provided and when, what personal data was processed, why the approach was necessary and proportionate, what safeguards were in place, and how employees were debriefed afterwards.

What I normally tell security teams tasked with these kinds of exercises is: “A security objective does not magically turn an undisclosed deepfake into a compliant one. And If you have to clone the CEO’s voice to make the test work, legal should be in the room before anyone presses send.”

As of mid-June, only nine of the twenty-seven member states had designated both a market surveillance authority and a notifying authority, twelve had partial designations, and six had neither. Where does the first Article 50 action originate: a regulator, a competitor complaint, a consumer group, or a defamation claim in a national court?

It’s difficult to say for sure where that first Article 50 action is likely to come from. Formally, the first Article 50 action is most likely to come from a market surveillance authority, because that is where enforcement responsibility sits at national level. But practically, the trigger may come from somewhere else.

Working backwards, defamation is probably the least likely at this early stage. But a defamation claim is possible, especially where synthetic audio or video damages someone’s reputation, but that is more likely to be a parallel legal route than the first clean Article 50 enforcement case.

For those AI systems that could affect or interact with large numbers of people, consumer groups could be likely candidates for an early challenge. But regulator-led action seems the most likely, even if some markets are still setting these up. I would expect the first case to be regulator led on paper, but very possibly complaint-led in reality, triggered by a consumer group, competitor, employee, journalist, civil society organisation, or affected individual.

What is the question your clients keep asking that has no good answer yet?

The question I keep hearing is: How do we prove what an AI agent did, why it did it, and who was accountable?

That is still a hard question with no real good answer yet. In cybersecurity and GRC, evidence matters. We need logs, approvals, identities, access controls, retention, and audit trails. But agentic AI can reason, retrieve data, generate content, and take actions across multiple systems. That means governance has to move from policy documents into technical controls.

My advice to clients is to treat AI agents like privileged digital identities. Give them an owner, a defined role, least-privilege access, monitoring, approval gates, and a kill switch. The organisations that get this right will not just be more compliant. They will be more resilient.

Another question I keep hearing is directly tied to using AI with security testing: Where does transparency end and security testing begin?

Security teams need realistic simulations, but the AI Act pushes organisations toward disclosure when people interact with AI or are exposed to deepfakes. The hard part is designing exercises that remain realistic without crossing legal, ethical, or employee trust boundaries.

Security teams want realism. Regulators want transparency. The challenge is designing exercises that satisfy both.

And there are more questions out there that lack answers like:

• Who is ultimately accountable when an AI system causes harm, the vendor, the deployer, the business owner, or the executive team?
• How do we prove to regulators, customers, and the board that our AI governance is working in practice, not just documented in policy?
• How much business value are we willing to lose in order to stay compliant, transparent, and auditable when using AI at scale?

Download: The high-performance team playbook