Unless you’ve been living on a desert island or in a cave for the past several years, you’ve surely heard and read about the transformative power of Artificial Intelligence (AI). From education to entertainment and marketing to medical research (and everything in between), AI’s ability to process tremendous amounts of data, learn patterns, and automate complex and burdensome tasks is radically changing not only how companies conduct business, but how people carry out their jobs.
For those of us in the cybersecurity space, AI — particularly Large Language Models (LLMs) and LLM-powered AI agents — promises to revolutionize the way we work. Perhaps nowhere are the effects more profound than in the field of offensive security, in which security practitioners put themselves in the shoes of malicious threat actors to identify system vulnerabilities and attack vectors so they can proactively address weaknesses and mitigate their company’s risk.
There are many approaches companies can take to offensive security, including vulnerability assessment, pen testing, and red teaming. But as effective as these approaches have proven, they’re losing traction in an increasingly complex landscape. Not only are the bad guys using more sophisticated techniques to infiltrate their targets’ systems, but thanks to a host of new technologies (blockchain and IoT, among them) and a growing remote workforce, it’s getting harder to identify and defend the perimeter.
What’s more, each approach to offensive security, whether it’s vulnerability assessment or pen testing, requires a unique skill set and expertise in specific areas — there’s a considerable difference between testing an IoT device and a mobile banking app — a hurdle which is only exacerbated by a pervasive lack of skilled security professionals. While the list of obstacles today’s offensive security practitioners face is lengthy, much like the Lone Ranger riding Silver to the rescue, help is on the horizon in the form of AI.
A 24/7 workforce
AI-driven tools have a myriad of uses on the security front. For instance, they’re capable of mimicking sophisticated cyberattacks and detecting vulnerabilities in everything from systems to software long before they can be exploited by malicious actors, which, in turn, allows security teams to allocate their efforts more efficiently and across a wider range of attack scenarios. Additionally, they’re able to simultaneously test multiple tools across a system and even across multiple systems concurrently. They’re also capable of dynamically responding to new vulnerabilities, adapting when testing various exploits and their impacts, and continuously improving their performance. (It’s this ability to autocorrect that’s especially useful when scaling out security checks and posture management.)
Perhaps one of AI’s biggest contributions to security is that of alleviating the pressure on already overburdened security teams. AI’s human counterparts can’t compete when it comes to the speed with which AI can analyze and combine massive amounts of data to find commonalities. Unlike people, AI-driven security tools are able to work around the clock, monitoring systems and suggesting courses of action when they encounter problems. Not only can these tools fill roles that previously required several people working across multiple shifts, but AI-driven tools can be used to fast-track future security professionals’ careers as they help those in junior roles see patterns faster and how things work at a quicker rate.
To the future and beyond
AI is already proving its mettle in the field of offensive security, and we’re only getting started. For instance, because of AI’s efficiency at handling time-consuming tasks, security testers are free instead to focus on more strategic and creative aspects of their jobs.
Increased automation will also lead to shorter feedback cycles meaning that these activities can be inserted into the DevSecOps process at even earlier stages than they are today. This shift-left also means that security considerations will be baked in at the beginning of the software development life cycle rather than tacked onto the end, improving an organization’s overall security posture.
Beyond individual tasks, the use of AI has also democratized the field of offensive security to some extent. Its ready availability means that it’s not only the companies with large budgets that can benefit. Today, even smaller companies are able to take advantage of security testing, thanks to tools such as OpenAI’s GPT-4o.
Unfortunately, just as security testers are taking advantage of AI, so too are threat actors. Security practitioners will need to develop new skills and evolve their techniques and tools if they are to stay one step ahead of their adversaries.
It’s no silver bullet
Despite the impressive list of benefits, AI-driven tools are not without their issues. And much like the Lone Ranger faced a litany of threats and tests in his quest to keep the Wild West safe from outlaws, security practitioners are finding that far from being a silver bullet, AI comes with its own set of problems.
For starters, the use of AI in offensive security in particular lends itself to a higher rate of false positives or missed vulnerabilities that a person would be able to catch. There is also the issue of understanding why an AI system came to the decision it did. In other words, can it explain itself? Understanding the reason that certain steps were taken is a critical aspect of offensive security, and the inability to do so undermines the process’s credibility.
It’s also critical for data sets to be continuously updated with the latest vulnerabilities. AI tools are only as good as the data that feeds them — poor data will lead to poor performance. For this, and the reasons mentioned above, it’s essential to maintain some kind of human oversight as a means of checks and balances.
Technical challenges aren’t the only ones in play, either. AI brings with it a host of ethical issues that must be addressed, not the least of which is data bias. Whether a bias stems from data that fails to accurately reflect a given population or scenario, historical bias, incorrect information, or even malicious changes to the data set, the result is the same — degraded performance, errors, and a lack of trust in the system. Data spillage and whether the AI system is using sensitive or personal data from outside the scope of a test is another concern, as is that of accountability.
Mitigating misuse
Despite the issues, AI brings a lot to the table when it comes to offensive security operations. Employing well thought-out mitigation strategies, such as those mentioned below, will go a long way toward preventing misuse and encouraging responsible use.
- Human oversight to validate AI outputs will go a long way toward ensuring greater accuracy and fewer unintended consequences.
- Employing strong Governance, Risk, and Compliance (GRC) frameworks, such as the National Institute of Standards and Technology’s (NIST) AI Risk Management Framework and OWASP’s LLM AI Security and Governance Checklist, will help to ensure safe, secure, and ethical AI use.
- Stringent access controls coupled with anomaly detection will help to protect AI systems from unauthorized access and misuse.
- The use of privacy-preserving techniques and data anonymization methods to protect sensitive information should also be utilized, along with ensuring compliance with data privacy regulations (e.g., GDPR, CCPA).
- Feedback loops along with real-time anomaly detection techniques will both work to reduce false negatives.
- Ethical guidelines should be established to ensure that AI actions don’t overstep agreed-upon boundaries or cause unintended harm.
- Techniques like feature importance scoring and model auditing should be implemented to make AI decision-making processes more understandable and traceable.
AI offers enhanced agency and automation
The use of AI-driven has significantly enhanced offensive security teams’ effectiveness. By automating and scaling tasks which, in turn, leads to increased efficiency, security teams are able to better align their work with their company’s goals, and concentrate their efforts where it matters most.
But as yet, no one AI tool can handle all aspects of offensive security perfectly. It’s important, therefore, that companies take a multifaceted approach, experimenting with various AI tools to see which offer the most effective solutions for their needs. Correspondingly, companies need to ensure that they are promoting a strong culture of ethical and responsible AI use to ensure that AI is working for the people and not the other way around.
By adopting a measured approach to AI, organizations will be able to strengthen their defensive capabilities and gain a leg up on the competition.
About the Author
Sean Heide is the Technical Research Director of the Cloud Security Alliance. Sean spent his early years as an Expeditionary Warfare Navy Intelligence analyst before moving into cyber security. His current research with CSA focuses on Enterprise Architecture and security policy adjustment, Top Threats identifications, and helping lead an initiative for CISOs and security leaders to bridge the gap on critical areas of enterprise. He graduated from Colorado State University with a degree in Information Technology and went on to receive his Masters of Science in Information Technology Management and Cyber Security from the same institution. In his free time, Sean can be found trying to breach his home lab, learning new security concepts, or playing video games. Sean can be reached online at linkedin.com/in/seanheide and at our company website https://cloudsecurityalliance.org/ .
