Learn · Intermediate

Adversarial Examples: Fooling AI With Changes You Can't See

An adversarial example is an input that has been deliberately nudged by a tiny, often imperceptible amount to make an AI model produce a confident wrong answer. Add a precisely computed speckle of noise to a photo of a panda -- noise so faint a person sees no change at all -- and an image classifier that was 58% sure it was a panda becomes 99% sure it is a gibbon. This phenomenon, first documented by Christian Szegedy and colleagues in 2013, is one of the most important and unsettling facts about modern neural networks: they can be fluent and superhuman on normal data while being catastrophically fragile to changes that do not fool a human for a second.

To see why this happens, picture what a classifier actually does. It carves up a very high-dimensional space of possible inputs into regions -- 'this area is pandas, that area is gibbons' -- separated by decision boundaries. On the kinds of images that occur naturally, those boundaries sit in sensible places. But natural images occupy a vanishingly thin sliver of all possible pixel arrangements, and the model has never been forced to behave reasonably everywhere else. An attacker exploits the empty space in between: they compute the exact direction in pixel-space that most rapidly pushes the input across the nearest boundary, and take a small step that way. The image barely changes to your eye, but in the model's coordinate system it has crossed into 'gibbon' territory.

The canonical way to compute that direction is the Fast Gradient Sign Method, introduced by Ian Goodfellow, Jonathon Shlens and Christian Szegedy in 2014. Their key insight was counterintuitive: adversarial examples are not caused by the model being too complex or too nonlinear. They arise partly because models are too linear in high dimensions. If each pixel contributes a little to the output and there are a million pixels, then a tiny per-pixel push -- summed across all of them -- adds up to a large shove on the final score. Their method uses the model's own gradient, the same signal used to train it, but instead of adjusting the weights to fit the image, it adjusts the image to break the weights. Goodfellow's paper framed the takeaway sharply: the ease of generating these examples suggests the models have 'a more superficial understanding' than their accuracy implies.

Adversarial examples come in flavors. A white-box attack assumes the attacker can see the model's internals and gradients, which makes crafting perturbations easy. A black-box attack only lets the attacker query the model and read its outputs -- harder, but often still effective, partly because adversarial examples transfer: an example crafted to fool one model frequently fools a different model trained on similar data. Attacks can be untargeted (just make it wrong) or targeted (make it output this specific wrong label). And they are not confined to images: they apply to audio (commands hidden in noise), to text, and -- as today's research shows -- to AI agents that control robots.

That robotics case is a vivid modern example. The 2026 BadWAM paper attacked 'world-action models' -- systems where a robot both imagines a future and produces actions, with the imagined future meant to serve as a safety check. The attackers used small visual perturbations, bounded so a monitor would not notice, to desynchronize the robot's actions from its imagination: the robot still 'dreamed' a plausible future while its hands drifted toward failure, halving task success on some benchmarks. It is a direct descendant of the panda-gibbon trick, applied where the stakes are physical, and it connects to the same asymmetry seen in the Hugging Face agent breach: AI systems can be manipulated in ways their guardrails and self-checks do not catch.

The strongest defense is adversarial training, formalized by Aleksander Madry and colleagues in 2017. The idea is to fold the attack into the training loop: at each step, generate the worst-case perturbation of each example (they used an iterative method called Projected Gradient Descent), then train the model to classify that perturbed version correctly. This is a min-max game -- the trainer minimizes the loss that the attacker maximizes -- and it produces the most reliably robust models we have. But it is expensive, and it exposes a stubborn tension: more robustness typically means somewhat lower accuracy on clean inputs, because the boundaries the model must learn are less flexible. Other proposed defenses, like detecting or smoothing away perturbations, have a long history of being broken by stronger attacks soon after.

Why does any of this matter beyond a party trick? Because it tells you something deep about how neural networks generalize. A model that leans on regularization and huge data can be accurate without being robust -- it has learned patterns that correlate with the answer rather than patterns that a human would call understanding. That gap has practical consequences anywhere AI meets an adversary: spam and content filters, malware detection, self-driving perception, biometric security, and the prompt injection attacks that plague language models are all, at heart, the adversarial-example problem in different clothing. Understanding that a confident model can be quietly, invisibly wrong is the first step to building systems that fail safely when someone is actively trying to break them.

Ground Truth. (2026, July 17). Adversarial Examples: Fooling AI With Changes You Can't See. Ground Truth. https://groundtruth.day/learn/adversarial-examples.html

@misc{groundtruth:adversarial-examples,
  title  = {Adversarial Examples: Fooling AI With Changes You Can't See},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {jul},
  url    = {https://groundtruth.day/learn/adversarial-examples.html}
}

Topics: adversarial-examples · robustness · security · safety · fundamentals