Artificial intelligence (AI) and machine learning (ML) have become staple technologies in modern business. From customer trend analysis and product design to customer service, the use of AI/ML is everywhere. More and more, AI/ML has become foundational to how modern enterprises run their operations and do business.

But the bad guys know this, and AI/ML systems are becoming targets. Enter adversarial AI (and adversarial ML). These are techniques designed to deceive or exploit AI-based systems with the objective of compromising their effectiveness. Adversarial ML has become so prevalent in recent years that in 2020, MITRE released the ATLAS™ (Adversarial Threat Landscape for Artificial-Intelligence Systems) Threat Matrix, outlining threats to machine learning systems that increase the attack surface of existing systems.

Understanding the inner workings and implications of adversarial AI is critical for any SecOps team. In this article, we’ll explore adversarial AI in depth. We’ll examine why it’s a growing concern, the key techniques adversaries employ, and how you can mitigate the risks adversarial AI introduces.

What is adversarial AI/ML?

Adversarial AI or adversarial machine learning (ML) seeks to inhibit the performance of AI/ML systems by manipulating or misleading them. These attacks on machine learning systems can occur at multiple stages across the model development life cycle, from tampering with training data or poisoning ML models by introducing inaccuracies or biases to crafting deceptive inputs to produce incorrect outputs. Furthermore, these tactics can be combined to magnify the effectiveness of an attack.

Unlike traditional cyber threats — such as malware or phishing — adversarial AI tries to exploit the decision-making logic of an AI system, resulting in malware that can evade a trained and production-ready machine learning model. As a result, adversarial AI/ML is becoming a leading concern for modern SecOps teams.

Critical concerns stemming from adversarial AI and ML

Adversarial AI is a current reality and a trending technique, and its growing sophistication increases its potential to severely compromise an organization’s security. This exposes several critical concerns:

• Increasing complexity of AI and ML models: As AI/ML models become more advanced, they become more attractive as targets for cyberattacks.
• Implicit trust in AI/ML outputs: Organizations are increasingly viewing AI/ML systems as black boxes. This growing, unquestioning trust in a system’s outputs makes these systems more vulnerable to attack, with exploited systems becoming more difficult to detect.
• Widespread presence of AI/ML: AI/ML technologies have made their way into enterprises in every sector. This amplifies the impact of a successful adversarial AI attack.
• Enhanced capabilities of attackers: Technology related to AI/ML continues to advance. As it does, so do the tools, methods, and skills of those seeking to exploit these systems.

Key techniques in adversarial AI and ML

To better understand how adversarial AI attacks are executed, security professionals should become familiar with some key adversarial techniques.

Data poisoning

ML models are trained on vast amounts of data. In predictive AI, an ML model can predict outputs based on the historical data on which it was trained. In generative AI, a model can generate entirely new content based on the training data it has consumed.

Training data is absolutely critical to the performance of an ML model, which is why it is such an attractive target in adversarial AI. Data poisoning involves altering that training data to bring about flawed behavior or decision-making in the resulting model. Systems built on flawed models will experience tremendous detrimental impacts.

Model tampering

Adversarial AI attacks also use model tampering, in which unauthorized modifications are made to an ML model’s parameters or structure. This is unlike data poisoning because the target is the pre-trained model itself. A tampered model may be compromised in its ability to produce accurate outputs.

Because AI/ML models may be used in business or consumer decision-making, a tampered model — especially one that goes undetected for an extended time — can lead to devastating results.

Transferability of attacks

As attackers test out tactics on various AI/ML systems, they can harness tools to apply successful attack techniques to other systems. Scaling their efforts to execute adversarial AI widely takes far fewer computing resources than those used to build and train AI/ML models and systems.

In light of these attack techniques, how can modern enterprises mitigate the risks?

Mitigating the risks

To ensure a robust security posture that includes protection from adversarial AI, enterprises need strategies that begin at the foundational level of cybersecurity.

Monitoring and detection

As with any other system in your network, AI/ML systems should be continuously monitored to bring swift detection and response for an adversarial AI threat. Leverage cybersecurity platforms with continuous monitoring, intrusion detection, and endpoint protection. You can also implement real-time analysis of input and output data for your AI/ML system. By analyzing this data for unexpected changes or abnormal user activity, you can quickly respond with measures to lock down and protect your systems.

Continuous monitoring can also lead to the application of user and entity behavior analytics (UEBA), which you can use to establish a behavioral baseline for your ML model. Based on this, you can more easily detect anomalous patterns of behavior within your models.

User awareness and education

Many of your staff members and stakeholders may be unaware of the concept of adversarial AI, let alone its threats and signs. As a part of your overall cybersecurity defense strategy, raise awareness through training programs and education. Train your teams on how to recognize suspicious activity or outputs related to AI/ML-based systems. You should also ask your security vendor how they harden their technology against adversarial AI. One way CrowdStrike fortifies ML efficacy against such types of attacks is by red teaming our own ML classifiers with automated tools that generate new adversarial samples based on a series of generators with configurable attacks.

When your staff is equipped with this kind of knowledge, you add an extra layer of security and foster a culture of vigilance that enhances your cybersecurity efforts.

Adversarial training

Adversarial training is a defensive algorithm that some organizations adopt to proactively safeguard their models. It involves introducing adversarial examples into a model’s training data to teach the model to correctly classify these inputs as intentionally misleading.

By teaching an ML model to recognize attempts to manipulate its training data, you train the model to see itself as a target and defend against attacks such as model poisoning.

Defend against adversarial AI/ML with CrowdStrike

With the growing dependence of modern enterprises on AI/ML-based systems, the emergence of adversarial AI as a prominent cybersecurity risk should come as no surprise. Seeking to inhibit the performance of AI systems, adversarial AI uses techniques like model poisoning or model tampering to cripple the accuracy and reliability of system outputs.

Defending against adversarial AI requires a hardened cybersecurity posture that uses robust tools within a single platform. The CrowdStrike Falcon^® platform is an advanced suite of cybersecurity tools that offers end-to-end protection against all types of cyber threats. Contact CrowdStrike to learn more.

Sign Up for Free Trial