Researchers attempt to fool AI with magic tricks

What can stage magic reveal about cognitive biases? Quite a lot, as it turns out. Researchers at the Institute of Neuroscience in Spain, Teatro Encantado in Madrid, and University Pompeu Fabra in Barcelona sought to apply AI and machine learning to quantify a professional magician’s skills in “naturalistic conditions.” They say that their trained system — which was designed to follow coins as a magician made them appear and disappear — not only served as a tracking tool but as an “artificial spectator” which could infer their location, paving the way for experiments in a subfield they describe as “artificial illusionism.”

“Magic is not the violation of the natural order of things, but the command of cognitive processes,” wrote the researchers. “The insights that magic offers to psychology and cognitive science are not new, yet attempts for a science of magic are only recently flourishing. [Our system] … offers an unparalleled opportunity to measure human behavior in complex activities outside the lab, such as the magician’s prestidigitation.”

The researchers sourced freely available software optimized for video analysis and pose estimation, dubbed DeepLabCut, which they leveraged to guess the location of coins in a magician’s fingers as they came in and out of view. How? They recorded said magician performing a series of sleight of hand tricks specifically intended to fool a human, none of which involved narrative or verbal instructions, social cues, special effects, or the use of gimmicks. They then conceived the magician’s tricks from the point of view of the AI system, and had a human volunteer label the coins in a video frames such that the system automatically learned to self-annotate.

The aforementioned set consisted of five main brief tricks and two extra ones, all related to cognitive illusions. Each was based on different maneuvers — throwing, dropping, dragging, placing, and grasping movements — that the magician performed on the coins but that humans failed to notice. Interestingly, during tricks where the coin wasn’t visible during most of the sleight of hand, both humans and the AI system inferred it to be in a closed fist when it wasn’t. The researchers say this shows that at least some cognitive tricks can transfer from humans to an AI system. That said, in another trick which tripped up humans even though the coin was visible at the moment of the effect, the AI algorithm had no problem detecting it throughout.

The researchers say that their work, while somewhat inconclusive, shows what might be possible when “magicians and machines merge.” “We have shown that it might be possible to target the development of adversarial cognitive tricks at humans by choosing AI models that match the human cognitive system as closely as possible,” they wrote. “The ‘good’ magic spectator is not the analytic observer who resists to be fooled, but the enchanted mind that voluntarily dwells into the experience of the impossible … Our results are a prelude to further deployments of deep learning in the context of the demanding processes of a magic performance — a perspective to our knowledge not explored so far.”