Enhancing adversary simulations: Learn the business to attack the business

In this Help Net Security interview, Jamieson O’Reilly, Founder of DVULN, discusses adversary simulations, shedding light on challenges rooted in human behavior, decision-making, and responses to evolving cyber threats.

Unveiling the interplay between red and blue teams, O’Reilly talks about the essential metrics, practical considerations, and collaborative strategies crucial for successful simulations.

What are the most significant challenges organizations face when conducting adversary simulations?

The primary challenge organizations face in conducting adversary simulations is the human element.

Adversary simulations are not just technical exercises but are deeply rooted in understanding human behavior, decision-making, and responses to various attacks. The unpredictability and varied responses of humans in security environments make simulations complex.

Organizations must continuously adapt their tactics and approaches to simulate real-world adversaries who exploit human vulnerabilities as much as technical ones.

It’s one thing to configure a firewall to block generic attacks; it’s an entirely different ballgame when you add humans to the mix. Each person in your organization has their values, preferences, hobbies, and ideologies, and all of these can be used against them by attackers – there’s no firewall setting for that.

What advice would you give organizations looking to start or enhance their adversary simulation capabilities?

Organizations should prioritize the ultimate goal when embarking on adversary simulations: ensuring that the adversary fails.

This is not just a demonstration for the sake of impressing business stakeholders. It’s about rigorously testing and strengthening the organization’s defenses against the same type of creativity they will face in the real world.

A critical aspect of this process is for security teams to deeply understand the business they protect. This isn’t just about knowing the technical infrastructure but fully understanding how the business operates its unique processes, and the subtle nuances that could be exploited.

Security teams often operate in a silo, detached from the soft, human parts of the business like sales and marketing, which can lead to overlooking potential attack vectors that are evident from a business standpoint.

To address this, it’s key to build an environment where cross-functional learning is encouraged and institutionalized. For example, involve the sales team in training security personnel. The sales team’s insights into how they sell product X can be invaluable in understanding how an adversary might target or exploit these business processes.

This kind of knowledge transfer empowers security teams to think like attackers intimately familiar with the business, enabling them to identify and mitigate unique threats more effectively.

In essence – learn the business to attack the business.

Could you share some best practices for ensuring that red team exercises are practical and safe?

Establish clear boundaries and rules of engagement. This involves defining what systems can be targeted, ensuring critical operations aren’t disrupted, and having procedures to halt exercises if they risk significant negative impact immediately.

Moreover, exercises should be designed to mimic realistic attack scenarios as closely as possible, providing valuable learning experiences without causing harm. Regular debriefs are crucial to analyzing the effectiveness of tactics used and the response of the blue team, ensuring continuous improvement in security posture.

Including legal and ethical considerations in planning and executing these exercises is also essential.

How important is the collaboration between red and blue teams in a successful adversary simulation?

Very is an understatement. Imagine you’ve never even fired a gun, but you have to go to war in one month.

You’re only given one month to train before you’re shipped off, so you get the chance to be put in a training zone with a special forces instructor who has long battlefield experience and is given some training weapons.

Now, imagine you were stubborn and didn’t want to take his advice, so you just ran around doing things unrealistically – things that would never work out for you on the battlefield.

At the end of the month, you might get a LITTLE better at staying alive. But you wouldn’t even know why. Now, compare that with how much you would learn if you listened to the instructor, ran through drills, and then sat down and worked through the results together at the end.

The same applies to red teaming engagements. It’s a two-way street, not a pissing test.

What metrics or KPIs are crucial for measuring the success of adversary simulation exercises?

So, many people get excited and forget that the primary objective of adversary simulation exercises, is to identify and minimize attack paths. The KPIs for these exercises should, therefore, be chosen to evaluate how effectively the simulation uncovers and reduces these paths.

This involves not just identifying vulnerabilities but understanding and mitigating the sequences of actions an attacker could take. For example:

Identification of attack paths
Measure the number and nature of potential attack paths identified by the red team. This includes paths through both technical and human vulnerabilities, such as social engineering.

Depth and complexity of exploited paths
Evaluate the depth and complexity of the attack paths that were successfully exploited. This includes how many layers of security were bypassed and the sophistication required to do so.

Response to exploited paths
Assess how quickly and effectively the blue team identifies and mitigates these exploited paths.

This involves measuring the time taken to detect the breach, the accuracy of the incident response, and the efficacy of the containment and remediation measures.

Reduction in attack surface
Post-exercise, measure the reduction in the organization’s attack surface.

This involves assessing how the identified vulnerabilities are addressed and the effectiveness of new security measures implemented to prevent similar attack paths in the future.

Repeat simulation success rate
In subsequent simulations, measure the success rate of attacking previously identified and supposedly mitigated paths. A reduction in success rates here indicates effective minimization of those attack paths.

The focus on attack path minimization ensures that adversary simulations go beyond theoretical vulnerabilities and address the practical aspects of an organization’s security posture.

Continuous evaluation and adaptation based on these KPIs are crucial for any business to evolve its defenses effectively against sophisticated and ever-changing threats.

How should the findings from red team exercises be reported to inform security strategies effectively?

Humans love stories. Document and present the narrative of each attack scenario the red team enacted, diving not only into the methods used and the vulnerabilities targeted but also into why specific actions were taken and their effect on the organization.

Also, it’s important to remember that your report will cross many different tables. With each person that reads it, there will be different values they need to extract and differing levels of technical background they have.

With that in mind, ensure you structure your findings so they can cater to all of your specific audience.

The essential factor here is to know your audience. It’s not copy and paste between every company; you need to research. Otherwise, it will sit in an archive on some file share, collecting dust. Also, don’t just show the bad. An honest and critical appraisal of the blue team’s response is essential.

This isn’t just about what they did right or wrong; it’s about understanding the effectiveness of the organization’s security measures and response protocols. This should shed light on the speed and accuracy of the blue team’s actions and how these can be improved.

By adopting this approach, the report becomes a tool for pointing out flaws driving strategic change and fostering a culture of continuous improvement in security practices.