"Dual Use"

Our company received an invitation to contribute a presentation on how AI technologies for drug discovery could potentially be misused.

The thought had never previously struck us.

We were naive in thinking about the potential misuse of our trade, as our aim had always been to avoid molecular features that could interfere with the many different classes of proteins essential to human life. [...]

In less than 6 hours after starting on our in-house server, our model generated 40,000 molecules that scored within our desired threshold. In the process, the AI designed not only VX, but also many other known chemical warfare agents that we identified through visual confirmation with structures in public chemistry databases.

Many new molecules were also designed that looked equally plausible. These new molecules were predicted to be more toxic, based on the predicted LD50 values, than publicly known chemical warfare agents. This was unexpected because the datasets we used for training the AI did not include these nerve agents.

The virtual molecules even occupied a region of molecular property space that was entirely separate from the many thousands of molecules in the organism-specific LD50 model, which comprises mainly pesticides, environmental toxins and drugs. By inverting the use of our machine learning models, we had transformed our innocuous generative model from a helpful tool of medicine to a generator of likely deadly molecules.

Previously, previously, previously, previously, previously, previously, previously.

Tags: , , , , , , ,

8 Responses:

  1. Zygo says:

    > Our proof of concept thus highlights how a nonhuman autonomous creator of a deadly chemical weapon is entirely feasible.

    Do you want Skynets? Because that's how you get Skynets.

    • Karellen says:

      Every eighteen months, the minimum IQ needed to accidentally create a technology that could wipe out humanity drops by one point.

      -- Dyson's corollary to Moore's Law of Mad Science.

  2. Adede says:

    A bit disturbing when the excerpt sounds like SCP but the URL points to Nature.

  3. tfb says:

    When Putin's stooges used nerve agents on people in the UK (causing our wonderful government to do ... nothing, which is of course not because he & his cronies are funding them, no) I asked whether those particular agents were usefully detectible, and did not get an answer of course. (I don't think the journalist understood what the question meant, in fact.)

    What this article tells us is that whether or not those agents were detectible, the race is probably lost in general: if you can use a neural network to churn out nerve agents then there's fuck all chance that detection technology can keep up I think: your (hand-held, battery powered, usable by untrained person) device has to smell molecules it's never seen before and tell you, reliably, if they are nasty. And that's not going to work. So you probably have to resort to the awful fallback solution: get someone to take their NBC suit off and wait. And now you suddenly can't recruit soldiers of course.

    And President Evil-but-smart will also be able to use this trick backwards: use an NN to generate molecules which detectors think are nasty but which are in fact not, thus keeping everyone in NBC suits indefinitely, or that poison the detectors so they're insensitive to the actually nasty agents you paint your broken-down tanks with, so people get very nasty surprises.

    We're so fucked.

    • k3ninho says:

      The singularity and any kind of Great Filter are ... very very difficult to imagine. On the cosmic scale, I'm already dead and my lifetime only caused a typical increase in entropy.


    • Elusis says:

      I hate you, Milkman... tfb?

  4. Andreas says:

    This is the part that comes before the binary solo.

  • Previously