...using automated, plausible lies:After training the AI system to play the popular arcade game Frogger, and documenting how human players explained the decisions they made while playing the game, the team of researchers developed the agent to generate language in real time to explain the motivations behind its actions. [...]
When it comes to neural networks -- a kind of AI architecture made up of potentially thousands of computational nodes -- their decisions can be inscrutable, even to the engineers that design them. Such systems are capable of analyzing large swaths of data and identifying patterns and connections, and even if the math that governs them is understandable to humans, their aggregate decisions often aren't.
Upol Ehsan, lead researcher and PhD student in the School of Interactive Computing at Georgia Tech along with Riedl and their colleagues at Georgia Tech, Cornell University, and University of Kentucky, think that if average users can receive easy-to-understand explanations for how an AI software makes decisions, it will build trust in the technology to make sound decisions in a variety of applications.
"If I'm sitting in a self-driving car and it makes a weird decision, like change lanes unexpectedly, I would feel more comfortable getting in that car if I could ask it questions about why it was doing what it was doing," Riedl said.
This is absolutely horrifying. The idea here is: the self-driving car has done something inscrutable. What if there was a chatbot that was good at looking at what just happened, and making up a plausible explanation for it after the fact?
(Side note: that's probably how human consciousness actually works. Your "mind" is the post-facto explainer for what the meat-zombie automaton just happened to do. Sweet dreams.)
But this research is just mechanized "truthiness"! It's not actually explaining what happened inside the neural net, which it still treats as a black box. It's solving the problem of: "If the self-driving murderbox decided to cross 4 lanes and slam on the brakes, let's dig around in our memory and imagination, find a scenario where a human might have done that same thing, and proffer that as the explanation."
Even though the AI and the human might have done those for completely different reasons, that's the explanation that will sound most plausible!
The human's decisions were based on an unbroken straight line chain of 85 million years of "I am the primate who did not get eaten", layered with decades of fine motor control trailing. Whereas the AI based its decisions on whatever ad hoc junk a bunch of low-paid contractors dumped into the training set over a period maybe as long as 18 months.
So they're building an AI whose goal is to lie to you -- to build your confidence in decisions made by AIs.
We are so completely doomed.
Maxwell
Smart:Because, at this very minute, 25 CONTROL agents are converging on this building. Would you believe it? 25 CONTROL agents! Savage: I find that hard to beleive. Max: Would you believe 2 squad cars and a motorcycle cop? Savage: I don't think so. Max: How about a vicious street cleaner and a toothless police dog?
"CNN Drops"