Insight is precisely what Musk's strawberry-picking AI lacks, as do all the other AIs that destroy humanity in similar doomsday scenarios. I used to find it odd that these hypothetical AIs were supposed to be smart enough to solve problems that no human could, yet they were incapable of doing something most every adult has done: taking a step back and asking whether their current course of action is really a good idea. Then I realized that we are already surrounded by machines that demonstrate a complete lack of insight, we just call them corporations. Corporations don't operate autonomously, of course, and the humans in charge of them are presumably capable of insight, but capitalism doesn't reward them for using it. On the contrary, capitalism actively erodes this capacity in people by demanding that they replace their own judgment of what "good" means with "whatever the market decides." [...]

There are industry observers talking about the need for AIs to have a sense of ethics, and some have proposed that we ensure that any superintelligent AIs we create be "friendly," meaning that their goals are aligned with human goals. I find these suggestions ironic given that we as a society have failed to teach corporations a sense of ethics, that we did nothing to ensure that Facebook's and Amazon's goals were aligned with the public good. But I shouldn't be surprised; the question of how to create friendly AI is simply more fun to think about than the problem of industry regulation, just as imagining what you'd do during the zombie apocalypse is more fun than thinking about how to mitigate global warming. [...]

There's a saying, popularized by Fredric Jameson, that it's easier to imagine the end of the world than to imagine the end of capitalism. It's no surprise that Silicon Valley capitalists don't want to think about capitalism ending. What's unexpected is that the way they envision the world ending is through a form of unchecked capitalism, disguised as a superintelligent AI. They have unconsciously created a devil in their own image, a boogeyman whose excesses are precisely their own.

Previously, previously, previously, previously, previously, previously, previously, previously, previously, previously, previously.