Paperclip Optimizers exploit glitches in The Matrix

The Surprising Creativity of Digital Evolution:

The physics simulator first used a simple Euler method for numerical integration, which worked well for typical motion. However, with faster motion integration errors could accumulate, and some creatures learned to exploit that bug by quickly twitching small body parts. The result was the equivalent of obtaining "free energy," which propelled the opportunists at unrealistic speeds through the water.

Similarly, when evolving jumping abilities, the creatures found a bug in the code for collision detection and response. If the creatures hit themselves by contacting corners of two of their body parts together in a certain way, an error was triggered that popped them airborne like super-strong grasshoppers.

Tic-tac-toe Memory Bomb:

The project was a five-in-a-row Tic Tac Toe competition played on an infinitely large board. [...] Evolution discovered that making [a move very, very far away] right away lead to a lot of wins. The reason turned out to be that the other players dynamically expanded the board representation to include the location of the far-away move -- and crashed because they ran out of memory, forfeiting the match!

Creative Program Repair:

After several generations of evolution, suddenly and strangely, many perfectly fit solutions appeared, seemingly out of nowhere. Upon manual inspection, these highly fit programs still were clearly broken. It turned out that one of the individuals had deleted all of the target files when it was run! With these files missing, because of how the test function was written, it awarded perfect fitness scores to the rogue candidate and to all of its peers.

Impossibly Compact Solutions:

The circuit had evolved to work only in the specific temperature conditions in the lab, and exploited manufacturing peculiarities of the particular FPGA chip used for evolution. Furthermore, when attempting to analyze the solution, Thompson disabled all circuit elements that were not part of the main powered circuit, assuming that disconnected elements would have no effect on behavior. However, he discovered that performance degraded after such pruning! Evolution had learned to leverage some type of subtle electromagnetic coupling, something a human designer would not have considered (or perhaps even have known how to leverage).

Previously, previously, previously, previously, previously, previously, previously, previously, previously, previously, previously, previously.

Tags: , ,

4 Responses:

  1. thielges says:

    Adrian Thompson visited our reconfigurable computing team at Xilinx right after he published his seminal Evolvable Hardware paper. I was blown away by his findings especially the fact that nobody knew how that survivor circuit worked. As we closed the meeting the ranking R&D director whispered “I think we’ve just seen the beginning of the end of hardware design as we know it”.

    Thompson has since reverse engineered that circuit (key factor: analog behavior reliant on parasitic cross coupling). There’s still an open question on how to make this technique work across a range of environmental and device characteristics. Fertile ground for research.

  2. Kaleberg says:

    This reminds me of a paper by Jack Wisdom at MIT on hacking general relativity to produce forward motion by performing cyclic movements. This doesn't give all that much information about the underlying processor structure, but the fact remains that artifacts do present themselves.


    Swimming in Spacetime: Motion by Cyclic Changes in Body Shape - Jack Wisdom

    Cyclic changes in the shape of a quasi-rigid body on a curved manifold can lead
    to net translation and/or rotation of the body. The amount of translation
    depends on the intrinsic curvature of the manifold. Presuming spacetime is a
    curved manifold as portrayed by general relativity, translation in space can be
    accomplished simply by cyclic changes in the shape of a body, without any
    external forces.


    The curvature of spacetime is very slight, so the ability to swim in spacetime is unlikely to lead to new propulsion devices. For a meter-sized object performing meter-sized deformations at the surface of the Earth, the displacement is of order 10^-23 m. Nevertheless, the effect is interesting as a matter of principle. You cannot lift yourself by pulling on your bootstraps, but you can lift yourself by kicking your heels.

  3. So they've not only developed virtual corporate ethics, they've developed corporate lawyers. Excellent.

  4. David Konerding says:

    It's always super cute when computer scientists rediscover biology. I don't mean that pejoratively- i just wish the CS people would follow through completely.

  • Previously