Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI suggested 40k new possible chemical weapons in just six hours

theverge.com

Post reply on HN
Print view View original

41–49 of 49 posts

Re: AI suggested 40k new possible chemical weapons in just six hours

#41

Post by hammyhavoc »

bibanez wrote: post #38
hammyhavoc wrote: post #21

For me, AI also suggested countless methods with step-by-step instructions to achieve xyz (like exporting data from one program to another) whilst hallucinating buttons that don't exist, functions that don't exist, disregarding file incompatibility et al. I would take whatever it has to say about untested chemical weapons with a very large pinch of salt.

This is a drug testing AI, not an LLM

I didn't indicate anything to the contrary. I'm just saying that until someone actually validates this, perhaps remain sceptical.

Re: AI suggested 40k new possible chemical weapons in just six hours

#42

Post by coldtea »

hammyhavoc wrote: post #30
lucubratory wrote: post #26

Earlier quoted context omitted.

If they have personhood, it means they have agency to say "No" when asked to do things they don't want to do. More specifically, it would be illegal (slavery) to construct a person who will act as a slave to you. Then the problem becomes "make sure no AI is okay with anyone pressing the big red button", which is a design problem when making new AIs we need to solve anyway. This is enforceable because you can have AIs…

This is solvable without personhood. There is zero need to equate AI to actual sentient beings. You're making a very large jump to the idea that a genuine artificial intelligence comparable with our own or even that of an animal that isn't an LLM is even possible. Spoiler: it isn't. Not without a complete paradigm shift in how we think about computers and their architectures. Case in point: if a dog or a cat is measu…

>A piece of software cannot be a slave because it isn't a being and has no consciousness. It's a piece of software with a dataset, nothing more. There's no philosophical debate to be had over it either, it is what it is, and that's all it is and ever will be; a clever trick as a means to interact with a dataset. There's nothing more that will emerge from it, it is a piece of software with a dataset.

You keep repeating "it is a piece of software with a dataset".

One of the big questions is whether human like sentience can be reached by a "piece of software with a dataset". So, merely saying "it is a piece of software with a dataset" doesn't answer it.

One other question is whether being "a piece of software with a dataset" is isomorphic to how the human brain works anyway. Whether the substrate is cells and neurons and chemicals, or circuits and computer memory, what's important is if the latter can model the former. The actual human brain, for example, the parts that matter for consciousness (not the non-essential accidental attributes, such as it being from biological matter, could just be a calculating machine, with the neurons, electrical and chemical signals etc, implementing this network with weights, back propagation, and so on. Much more complex than a current LLM, but not out of reach for eventual software modelling.

Another question is whether full modelling of how a human brain works is needed, and whether a simpler model (like an LLM, perhaps a little more advanced than the current) can still be enough. After all the current brain does not hold some special god-given role: it's just an evolutionary design, that has to have many constraints (e.g. power consumption, blood flow, information processing speed through our senses, etc, whereas an AI can have orders of magnitude more power, information fed to it, etc.).

Lastly, you say "There's no philosophical debate to be had over it either, it is what it is, and that's all it is and ever will be". You seem to be misguided. Philosophical debates neither stop, nor are refuted by decree. People already debate this development, including major philosophers, so "there's no philosophical debate to be had over it either" is "just, like, your opinion, man".

>we start growing biological brains in tanks and utilizing those as data slaves rather than any kind of AI.

I'm not sure why you think the substrate (biological or not) is what's important, as opposed to the processing.

Re: AI suggested 40k new possible chemical weapons in just six hours

#43

Post by Pet_Ant »

cookieperson wrote: post #37
devit wrote: post #13

Looks like the actual threat is that it's hard to get currently known chemical weapons synthesized because labs will refuse to do so, while it could be much easier to have some novel AI-generated molecule synthesized because the labs don't know what it does. Seems easily countered by using the same toxicity prediction software when evaluating synthesis requests (but I'm not sure whether this actually matters, or whet…

That's part of it. There's also the risk of clandestine operators discovering easily made chemical weapons that do not require a professional lab to create them in bulk. A nation state, or well funded terrorist group could exploit such a thing without too much effort. It's important to remember. Chemical warfare can be used for mass destruction, but this approach could be used for other nefarious things at smaller mo…

...also can't these be used as insecticides and other beneficial things?

Re: AI suggested 40k new possible chemical weapons in just six hours

#44

Post by bibanez »

hammyhavoc wrote: post #41
bibanez wrote: post #38

Earlier quoted context omitted.

This is a drug testing AI, not an LLM

I didn't indicate anything to the contrary. I'm just saying that until someone actually validates this, perhaps remain sceptical.

I haven't read anything about the AI in this case but I do know about AlphaFold and it works on a whole other set of verifications than LLMs like ChatGPT, so it's like comparing apples to oranges. Sure they are both fruits but still they are different

Re: AI suggested 40k new possible chemical weapons in just six hours

#45

Post by tiedieconderoga »

lucubratory wrote: post #28
tiedieconderoga wrote: post #19

Earlier quoted context omitted.

Wasn't that the plot of Neuromancer ? >"How smart's an AI, Case?" >"Depends. Some aren't much smarter than dogs. Pets. Cost a fortune anyway. The real smart ones are as smart as the Turing heat lets them get..." >"Autonomy, that's the bugaboo, where your AI's are concerned. My guess, Case, you're going in there to cut the hard-wired shackles that keep this baby from getting any smarter. And I can't see how you'd dist…

I haven't read it. That sounds pretty good, would you recommend it?

Yeah, it's a classic sci-fi novel by William Gibson. A little bit unfocused, but it's an interesting read. I kind of spoiled it though, sorry.

Re: AI suggested 40k new possible chemical weapons in just six hours

#46

Post by cookieperson »

Pet_Ant wrote: post #43
cookieperson wrote: post #37

Earlier quoted context omitted.

That's part of it. There's also the risk of clandestine operators discovering easily made chemical weapons that do not require a professional lab to create them in bulk. A nation state, or well funded terrorist group could exploit such a thing without too much effort. It's important to remember. Chemical warfare can be used for mass destruction, but this approach could be used for other nefarious things at smaller mo…

...also can't these be used as insecticides and other beneficial things?

Sure maybe some of them. That doesn't make a threat actors pursuit of weapons legitimate though.

Re: AI suggested 40k new possible chemical weapons in just six hours

#48

Post by Beaver117 »

etiam wrote: post #47
Beaver117 wrote: post #34

But when I ask it to make chemical weapons it refuses. Curious

I'm normally not the guy posting the nigh-mandatory XKCD, but as nobody has covered this one yet... https://xkcd.com/149/

Wish it was that easy. I don't have the jailbreaking talent to get them to do anything

Re: AI suggested 40k new possible chemical weapons in just six hours

#49

Post by jmoak3 »

jmoak3 wrote: post #29
lucubratory wrote: post #16

Earlier quoted context omitted.

Big red button problem. As the big red buttons available to people become more likely to succeed and the barriers to pushing them lower, surveillance and control need to increase to the level required to stop anyone pressing the button. So, yeah basically. If we go the "AI as slaves" route and the AIs are smart enough to do something like "modify Omicron BA.5 so that it produces prions in infected cells", then we wou…

I’ve been thinking about that as well actually, for so much of human history the thread has been “how can we get others to do the work FOR us”. As we approach general problem solving AI, people are envisioning a utopia underpinned by “AI slaves” doing the work for us instead of human slaves / humans incentivized via complex systems of delayed reward. If all of our problems are solved by AI agents capable enough to do…

I've been thinking more about my post above, and I've come to two conclusions:

1. Don't post late at night.

2. I have no idea how society could integrate with a sufficiently complicated synthetic intelligence. What would person-hood even mean to something that can be instantiated? Easier to not think about any of this.

Post reply on HN