Artificial intelligence, commonly referred to as AI, represents both a risk and a benefit to the security of society, according to Bruce Schneier, security technologist, researcher, and lecturer at Harvard Kennedy School.
Schneier made his remarks about the risks of AI in an afternoon keynote session at the 2021 RSA Conference on May 17. Hacking for Schneier isn't an action that is evil by definition; rather, it's about subverting a system or a set of rules in a way that is unanticipated or unwanted by a system's designers.
"All systems of rules can be hacked," Schneier said. "Even the best-thought-out sets of rules will be incomplete or inconsistent, you'll have ambiguities and things that designers haven't thought of, and as long as there are people who want to subvert the goals in a system, there will be hacks."
Hacking AI and the Explainability Problem
Schneier highlighted a key challenge with hacking that is conducted by some form of AI: it might be difficult to detect. Even if the hack is detected, it will be difficult to understand what exactly happened.
The so-called explainability problem is one that has been tackled in the popular cult classic science fiction novel The Hitchhiker's Guide to the Galaxy. Schneier recounted that in that novel a race of hyper-intelligent pan-dimensional beings build the universe's most powerful computer, called Deep Thought, to answer the ultimate question of life, the universe, and everything. The answer was 42.
"Deep Thought was unable to explain its answer, or even what the question was, and that is the explainability problem," Schneier said. "Modern AIs are essentially black boxes: data goes one in one end, and the answer comes out the other."
Schneier noted that researchers are working on explainable AI, but he doesn't expect it to yield any short-term results for several reasons. In his view, explanations of how AI works are actually a cognitive shorthand used by humans, suited for the way humans make decisions.
"Forcing an AI to produce a human-understandable explanation is an additional constraint, and it could affect the quality of its decisions," he said. "Certainly in the near term AI is becoming more opaque, less explainable.
No tags.
