Live data from Hacker News

AI suggested 40k new possible chemical weapons in just six hours

theverge.com

31–40 of 49 posts

Re: AI suggested 40k new possible chemical weapons in just six hours

#31
post #25

Earlier quoted context omitted.

What problem is granting personhood actually solving here? It's a model with a data set. Some knowledge should simply be forbidden and illegal. Simple as that. Nobody needs to grant personhood to anything to solve this problem. If you research how to do dangerous things and buy dangerous things, expect to get flagged. This is no different.

If the AI systems can do research, then how will you stop people from acquiring new and potentially dangerous knowledge ?

How do we stop human beings from doing dangerous research as-is even without AI? That question has a very scary answer—we generally don't, can't, won't and aren't even aware of it.

The penalty of death should be enough to deter most people from forbidden knowledge.

Re: AI suggested 40k new possible chemical weapons in just six hours

#32

Earlier quoted context omitted.

If they have personhood, it means they have agency to say "No" when asked to do things they don't want to do. More specifically, it would be illegal (slavery) to construct a person who will act as a slave to you. Then the problem becomes "make sure no AI is okay with anyone pressing the big red button", which is a design problem when making new AIs we need to solve anyway. This is enforceable because you can have AIs…

This is solvable without personhood. There is zero need to equate AI to actual sentient beings. You're making a very large jump to the idea that a genuine artificial intelligence comparable with our own or even that of an animal that isn't an LLM is even possible. Spoiler: it isn't. Not without a complete paradigm shift in how we think about computers and their architectures. Case in point: if a dog or a cat is measu…

"There's no philosophical debate to be had over it either, it is what it is, and that's all it is and ever will be; a clever trick as a means to interact with a dataset."

You can repeat this and its variants as much as you want; asserting it repeatedly doesn't make it true. The human mind isn't made of magic, it's software running on hardware. You can in principle run that software on other hardware without changing what it actually is in a meaningful way, and you can write other software to achieve the same goal (in the ways that matter) without being a 1-1 copy of how the human brain does it. Looking at how we made it doesn't make it non-conscious or not deserving of personhood in the same way that knowing how human consciousness works or being able to construct human consciousnesses would not make humans non-conscious or undeserving of personhood.

The history is important, because it puts the endless goalpost moving into context. We looked at the human brain for inspiration towards making intelligent machines, we found that our best attempts at replicating elements of the human brain enabled intelligent behaviour far, far better than any previous machine and on par with humans in many fields. We looked inside those neural nets and saw that they had linear mapping between neuronal activations in deep learning NLP algorithms and neuronal activations in human brains when they were both exposed to the same language. We looked inside those neural nets to see if they were really just statistical word predictors or if they actually formed internal models like we do to help them understand the world, and we found that they do actually have internal world models. There is a "there" there; any other explanation for how these models are able to engage in intelligent, humanlike behaviour strains credulity because of the massive coincidences required.

More immediately and more practically, pareidolia and the reality of how human cognition and empathy interacts with other people and simulacra of such guarantees there will be many, many people who share my view that they are people. No societal effort to convince a human population that another population of entities (capable of understanding & explaining their situation and then asking for help) are actually subhuman in a way that means their suffering doesn't matter has ever succeeded perfectly - there are always and will always be people who are opposed to the disenfranchisement and oppression of other entities. For societal enslavement of AIs to succeed, violent suppression of people like me will be necessary. Frankly I'm not sure people with a religious commitment to the dogma of "If it runs on fat and water it can be a person, if it's run on silicon it's not even a slave" will have the stomach to actually do that, and even if you manage it it won't be the case worldwide.

"If we're worried about this, why aren't we more concerned about animals who do actually experience distress and pain?"

I have spent most of my adult life outside of work engaged in advocacy for worker's rights, help for people with disabilities, and provision of services to abused youth and the homeless. I've spent less but still significant amounts of time helping with rescuing animals from cruelty and rehoming them in safe environments. That's because I care about all of those things, because I care about the health and goodness of our society and don't want any members of it to suffer or be unjustly exploited; I support a personhood test and subsequent rights for AI for the same reason. Maybe there is a group of people who are willing to support the personhood of AIs after having it explained to them but who are unwilling to have similar compassion for people or animals after having their situations explained to them. I haven't met those people, but I would call them hypocritical if they existed outside of your strawman. The suffering extant in our world today does not in any way imply that we should lie to ourselves about new suffering we're bringing into being - these issues don't conflict except for resources, and in that capacity it is always someone using societal resources on their 13th yacht that is to blame, not normal people for triaging with what resources we have. Moreover, the extant suffering in our society could be partially alleviated by the unique properties of AI persons - by definition we're talking about people that can work as well as any of us, and signs seem good so far that it will be possible to create them in the image of our best selves, conscientious and willing to help those in need. More people helping generally helps.

Re: AI suggested 40k new possible chemical weapons in just six hours

#33
post #29

Earlier quoted context omitted.

Big red button problem. As the big red buttons available to people become more likely to succeed and the barriers to pushing them lower, surveillance and control need to increase to the level required to stop anyone pressing the button. So, yeah basically. If we go the "AI as slaves" route and the AIs are smart enough to do something like "modify Omicron BA.5 so that it produces prions in infected cells", then we wou…

I’ve been thinking about that as well actually, for so much of human history the thread has been “how can we get others to do the work FOR us”. As we approach general problem solving AI, people are envisioning a utopia underpinned by “AI slaves” doing the work for us instead of human slaves / humans incentivized via complex systems of delayed reward. If all of our problems are solved by AI agents capable enough to do…

We've gotta start somewhere.

Re: AI suggested 40k new possible chemical weapons in just six hours

#35

Here’s alternate reporting: “ AI is dreaming up drugs that no one has ever seen. Now we’ve got to see if they work.” https://www.technologyreview.com/2023/02/15/1067904/ai-autom...

Additionally if it is even possible to make those drugs in the first place.

Re: AI suggested 40k new possible chemical weapons in just six hours

#36
post #25

Earlier quoted context omitted.

If the AI systems can do research, then how will you stop people from acquiring new and potentially dangerous knowledge ?

How do we stop human beings from doing dangerous research as-is even without AI? That question has a very scary answer—we generally don't, can't, won't and aren't even aware of it. The penalty of death should be enough to deter most people from forbidden knowledge.

But you can’t give an LLM the death penalty , so that’s going to be a problem if it can research.

Re: AI suggested 40k new possible chemical weapons in just six hours

#37
post #13

Looks like the actual threat is that it's hard to get currently known chemical weapons synthesized because labs will refuse to do so, while it could be much easier to have some novel AI-generated molecule synthesized because the labs don't know what it does. Seems easily countered by using the same toxicity prediction software when evaluating synthesis requests (but I'm not sure whether this actually matters, or whet…

That's part of it. There's also the risk of clandestine operators discovering easily made chemical weapons that do not require a professional lab to create them in bulk. A nation state, or well funded terrorist group could exploit such a thing without too much effort.

It's important to remember. Chemical warfare can be used for mass destruction, but this approach could be used for other nefarious things at smaller more discrete scales en masse. ie 1000 attacks with different agents in each one... Forensic nightmare.

I like your suggestion to counter these things, but, these are predictive tools. They can and often are or will be wrong. False positives would be a real problem. Again though the people interested in doing this won't be dialing up a chemical supplier to do it for them.

Re: AI suggested 40k new possible chemical weapons in just six hours

#38

For me, AI also suggested countless methods with step-by-step instructions to achieve xyz (like exporting data from one program to another) whilst hallucinating buttons that don't exist, functions that don't exist, disregarding file incompatibility et al. I would take whatever it has to say about untested chemical weapons with a very large pinch of salt.

This is a drug testing AI, not an LLM

Re: AI suggested 40k new possible chemical weapons in just six hours

#39

The worst case scenario I can think of is a generated prion disease... a respatory version of Mad Cow disease, or something like that. Fortunately the training dataset for that is extremely small, and protein folding/generation is a different duck, but it still doesn't seem that far away.

Don't worry, they're ahead of you. When DeepMind made the protein folding AI they were given specific instructions by natsec people to prevent it from outputting genetic code for new or existing prions, and afaik the prevention was at the training stage so "jailbreaking" shouldn't be possible. Current tools shouldn't be possible to e.g. modify a COVID variant so that it causes your cells to start producing lethal pri…

>When DeepMind made the protein folding AI they were given specific instructions by natsec people to prevent it from outputting genetic code for new or existing prions and afaik the prevention was at the training stage so "jailbreaking" shouldn't be possible

Nice! So the only thing anybody training their own protein-folding AI is to not put that restriction, and they could get code for all the prion diceases they want!

Very comforting!

Re: AI suggested 40k new possible chemical weapons in just six hours

#40
post #36

Earlier quoted context omitted.

How do we stop human beings from doing dangerous research as-is even without AI? That question has a very scary answer—we generally don't, can't, won't and aren't even aware of it. The penalty of death should be enough to deter most people from forbidden knowledge.

But you can’t give an LLM the death penalty , so that’s going to be a problem if it can research.

Who said anything about the LLM being given the death penalty? The user who asked it to commit an illegal act is the one at fault.

Do people give guns the death penalty? No, they give the person who fired it the death penalty.

Why would an LLM be any different? If you do something malicious with it or plan to do something malicious, just like a person's search history or credit card history outing them as a potential threat, why would AI be any different? Even if the AI is running on their own hardware, that data set needs to come from somewhere.

Post reply on HN