Autonomous Murderbots are Going Great

In a series of incidents, Cruise lost contact with its autonomous vehicles, leaving them frozen in traffic and trapping human drivers.

After a few minutes of bemused waiting, Hu says, he resorted to driving over the curbs of the street's median to escape. When he returned on foot a few minutes later to see whether the situation had resolved, the Cruise vehicles hadn't budged. A person who appeared to work for the company had parked in the intersection, Hu says, as if to indicate the street was closed, and was trying to direct traffic away from the immobile self-driving cars. Hu estimates that the robot car blockade, which has not previously been reported, lasted at least 15 minutes.

The Cruise vehicles that trapped Hu weren't the only autonomous cars holding up traffic in San Francisco that night. Internal messages seen by WIRED show that nearly 60 vehicles were disabled across the city over a 90-minute period after they lost touch with a Cruise server. As many as 20 cars, some of them halted in crosswalks, created a jam in the city's downtown. [...]

The June 28 outage wasn't Cruise's first. On the evening of May 18, the company lost touch with its entire fleet for 20 minutes as its cars sat stopped in the street, according to internal documentation viewed by WIRED. Company staff were unable to see where the vehicles were located or communicate with riders inside. Worst of all, the company was unable to access its fallback system, which allows remote operators to safely steer stopped vehicles to the side of the road.

A letter sent anonymously by a Cruise employee to the California Public Utilities Commission that month, which was reviewed by WIRED, alleged that the company loses contact with its driverless vehicles "with regularity," blocking traffic and potentially hindering emergency vehicles. The vehicles can sometimes only be recovered by tow truck, the letter said. [...]

Jeff Bleich, Cruise's chief legal officer [...] warned employees not working on that investigation to try and tune out crashes or related news reports, saying they were unavoidable and would increase in frequency as the company scaled up its operations. [...]

Testo, the Cruise spokesperson, said the company is "proud" of its safety record, "and it speaks for itself."

An Autonomous Car Blocked a Fire Truck Responding to an Emergency:

On an early April morning, around 4 am, a San Francisco Fire Department truck responding to a fire tried to pass a doubled-parked garbage truck by using the opposing lane. But a traveling autonomous vehicle, operated by the General Motors subsidiary Cruise without anyone inside, was blocking its path. While a human might have reversed to clear the lane, the Cruise car stayed put. The fire truck only passed the blockage when the garbage truck driver ran from their work to move their vehicle.

"This incident slowed SFFD response to a fire that resulted in property damage and personal injuries," city officials wrote in a filing submitted to the California Public Utilities Commission.

Previously, previously, previously, previously, previously, previously, previously, previously, previously.

Tags: , , , ,

9 Responses:

  1. Frandroid says:

    What's up with "autonomous" vehicles that just stop when losing their connection to their homebase? Shouldn't they have onboard maps and try to park?

    • Glaurung says:

      The people who programmed them have never had to live in an area with slow or laggy internet, and are incapable of comprehending that what works for them in their little tech bubble might not work out in the real world.  

      Plus, of course, the power of the clown compelled them to use the clown even when it made no sense.  

    • thielges says:

      Stopping is the most basic failsafe approach that has been used in automotive electronics all the way back to at least the 1980s.  Yes, it is a primitive strategy though is better than "full throttle" (*) or other behaviors that could occur with a random fault.  That made sense when you could count the transistors in a controller in under four digits.  But today there are multiple approaches using redundant logic that nearly eliminate those sorts of critical faults.  

      Here it seems that the Cruise AVs freaked out when they lost the connection to the Big Brain.  There's no excuse for using a lame, old "shut down and stop" in that scenario.  The car should limp to the edge of the road and then stop, but even that extra action carries some additional risk.

      (* I experienced a "full throttle" failure mode in a fully analog car once.  Corrosion had built up on an exposed section of the cable linking the gas pedal to the carb.  That section of the cable only entered the tight cable housing when the pedal was floored, something I rarely do but in this situation I was passing on a two lane road and was threading a tight gap.  So the slightly wider corroded cable "corked" into the housing and jammed the car into full throttle.  I killed the engine by twisting the key one notch counterclockwise.  Fortunately I didn't twist the key all the way off and engage the steering wheel lock.  Stuck steering might be worse than a stuck throttle.)

      • Michael says:

        I had car failures before. Not once did my car encounter an error and just stopped. In the old days, the engine may have turned off and then you coast and pull over. In more modern cars I got an error message and was told by the system to pull over and stop where it is safe.

        Not once did my car suddenly slam on the brakes, turn itself off and wished me a nice day.

        “Failing safe” in traffic is to move aside and out of the way, not suddenly come to a complete stop. That’s only happening in a crash and is the exact opposite of “failing safe”.

        • thielges says:

          I meant killing the engine, not hitting the brakes in the middle of the road.  That failsafe mode exists in many (most?) cars with electronic engine controllers.  Such "walk home" failure scenarios are considered serious problems and have mostly been engineered away to the point that people rarely experience them.  But that's old school tech containing 100000X less transistors than what you find in AVs.

          • Michael says:

            You don't want to kill the engine either. Most brakes are hydraulic, so is steering. You need the engine to drive the pump, otherwise brakes and steering won't work without a lot of muscle power, and most people have no idea that this would be the case.

            Safe failing in a car is to alert the driver to pull over and park the car. The car should not suddenly turn the engine off or brake, that's not a "fail safe" scenario in an environment that is as unpredictable and dynamic as a public road.

        • Right, but these things aren't actually bright enough to be trusted to pull that off without access to the clown. So, they become a hazard instead.

  2. Zygo says:

    Good thing they're not in Canada right now...

Leave a Reply

Your email address will not be published. But if you provide a fake email address, I will likely assume that you are a troll, and not publish your comment.

Starting a new top-level thread.

  • Previously