Whoops! AI might have invented 40000 new toxic nerve agents…but probably not
TL:DR – Software used to predict which chemicals in a database have potential as new medicines has been flipped as a warning demonstrating that the same system could be used to predict whether they are highly toxic. Artificial intelligence, AI, machine learning, language models, neural networks, generative models, algorithms trained on big data. A lovely … Continue reading "Whoops! AI might have invented 40000 new toxic nerve agents…but probably not"
TL:DR – Software used to predict which chemicals in a database have potential as new medicines has been flipped as a warning demonstrating that the same system could be used to predict whether they are highly toxic.
Artificial intelligence, AI, machine learning, language models, neural networks, generative models, algorithms trained on big data. A lovely collection of buzzwords that have been littering grant proposals for years and are now emerging into the real world as apps and websites and citizen science projects. For chemists and pharmaceutical scientists, AI has been a tool to experiment with for a long time. Creating molecular diversity, simulating physiology and screening for drug-like activity have been high on the laboratory agenda for quite some time. They have also used it to test for toxicity in silico and so reduce animal testing and problems once a drug gets into early. clinical trials.
Scientists at Collaborations Pharmaceuticals Inc., which focuses on drug discovery for rare diseases, commonly use trained AI to help them determine the likely toxicity of various novel molecular structures. Molecules with physiological activity are a prerequisite for finding new pharmaceuticals but it is also necessary to avoid those compounds that have a toxic effect on the patient rather than the disease.
A thought experiment flipped this work for good on its head and had team leader Fabio Urbina and colleagues wondering about the implications of an AI that seeks out life-taking toxic compounds rather life-saving pharmaceuticals. The team was asked to carry out simulations and to present the findings and implications to an international security conference.
The Convergence Conference is convened every two years by the Swiss Federal Institute for NBC (nuclear, biological and chemical) Protection – Spiez Laboratory – on behalf of the Swiss government to identify developments in chemistry, biology and enabling technologies that could have implications for the Chemical and Biological Weapons Conventions. The conference seeks to spot the next VX or Novochok.
Urbina and his colleagues have been careful not to disclose the details of their simulations, but by setting their AI in reverse so that it seeks out structures that are likely to be toxic rather than safe pharmaceuticals they were able to generate a whole molecular library of potential toxic agents on a par with VX and Novochok. Indeed, six hours on the computer produced a database of 40000 such molecules any one of which might theoretically be synthesised in a rogue state laboratory and many of which were predicted to be more toxic than VX. Of course, it could be that the AI has got it wrong (it happens!) and that none of these compounds are at all toxic. The details the team provides were published in the journal Nature Machine Intelligence.
It is rather worrying. But perhaps also inevitable and better that we know this is now a possibility rather than our ignoring the risks. It’s worth remembering, that the invention of dynamite had a notable dual use, noble and not so noble, you might say. This is in some way just a modern take of how an invention for good might so easily be manipulated for bad.
To quote from the paper: “We can take inspiration from examples such as The Hague Ethical Guidelines, which promote a culture of responsible conduct in the chemical sciences and guard against the misuse of chemistry, in order to have AI-focused drug discovery, pharmaceutical and possibly other companies agree to a code of conduct to train employees, secure their technology, and prevent access and potential misuse.”
That is unlikely to be enough in the long-term and it is perhaps inevitable that some rogue team somewhere is already building its own database of toxic agents using AI. We just have to hope that the worst in any such database are false positives or that they all prove to be too to tough to crack even for the best, bad chemists. But, given that we already have AI tools for devising chemical reaction schemes and have done for many years, it feels unlikely that this particular toxic genie is going to stay in its reaction flask for long.