Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Making Claude a Chemist

anthropic.com

Post reply on HN
Print view View original

41–50 of 97 posts

Re: Making Claude a Chemist

#41

Post by bouncycastle »

UltraSane wrote: post #38
thefounder wrote: post #2

Let’s ban this before it gets too powerful !

I don't really want the average LLM being able to tell anyone exactly how to syntheize lethal nerve agents.

you can already get that information by reading books in the library.

The biggest barrier is not information, it's the ability to secure enough of the materials and equipment.

For example, information for how to make a nuclear weapon is already there in the library. However, mining enough yellow cake and then purifying it is an industrial scale operation, out of reach unless you are a nation state, and have good mountain tunnels, etc. To a lesser extent, this is also true for producing chemical weapons. The theory is there, but actual production extremely out of reach. No LLM can help you there. (You can verify by reading up on Aum Shinrikyo to get an idea of the staggering scale required)

Re: Making Claude a Chemist

#42

Post by defrost »

bouncycastle wrote: post #41
UltraSane wrote: post #38

Earlier quoted context omitted.

I don't really want the average LLM being able to tell anyone exactly how to syntheize lethal nerve agents.

you can already get that information by reading books in the library. The biggest barrier is not information, it's the ability to secure enough of the materials and equipment. For example, information for how to make a nuclear weapon is already there in the library. However, mining enough yellow cake and then purifying it is an industrial scale operation, out of reach unless you are a nation state, and have good moun…

> mining enough yellow cake and then purifying it is an industrial scale operation, out of reach unless you are a nation state

or a transnational (or even national scale) energy and or minerals company.

Might be hard to slip past the shareholders, but dark projects have flown under the annual reports of several large players.

Re: Making Claude a Chemist

#43

Post by orphea »

simulator5g wrote: post #14
tearwear wrote: post #10

Earlier quoted context omitted.

not really, though. you don't see automated intelligence in the hands of junkies et al. ... and you don't see it coming, either ...

Junkies et al can log into chatgpt right now and use it to create a phishing email to steal drug money from your Grandma. Many people saw this coming.

And they don't need castrated Fable for that.

Re: Making Claude a Chemist

#46

Post by dmd »

NewsaHackO wrote: post #24
andai wrote: post #21

If they succeed here, won't they have to gate access to this feature, too? For the same reasons as, "if I so much as mention mitochondria, it downgrades me to Opus." Step 1. Make it so Claude can do anything — the whole point of AGI Step 2. Wait, if the user can do Anything, that would be Very Bad! Step 3. Err on the safe side with blanket bans of entire fields The latter actually seems to me a sensible reaction to e…

I think one time I asked opus about copyfail when it just came out and it did treat me like some sort of criminal, but are there really people that run into this on a regular basis other than cybersecurity experts (which cannot be a big enough group to generate all of this criticism)?

I asked it to clip a video onto segments based on location of spoken words (using separately done audio transcription) and it flagged that as cyber.

Re: Making Claude a Chemist

#47

Post by squidbeak »

echelon_musk wrote: post #30
MiracleRabbit wrote: post #23

Earlier quoted context omitted.

I just wait until a Opensource model inhaled all the chemistry books and papers. Lol. They are following closely and the best offer 80-90% of the performance and come with a very small fraction of the costs.

Why wouldn't the US gov. outlaw the open source models?

Without international treaty and regulation restricting frontier capabilities globally, any attempt to outlaw open source models will only be as effective as King Canute ordering the tide to turn. Unless the USA fancies bombing those who refuse.

Re: Making Claude a Chemist

#48

Post by dmd »

bouncycastle wrote: post #41
UltraSane wrote: post #38

Earlier quoted context omitted.

I don't really want the average LLM being able to tell anyone exactly how to syntheize lethal nerve agents.

you can already get that information by reading books in the library. The biggest barrier is not information, it's the ability to secure enough of the materials and equipment. For example, information for how to make a nuclear weapon is already there in the library. However, mining enough yellow cake and then purifying it is an industrial scale operation, out of reach unless you are a nation state, and have good moun…

Sure, information on all sorts of things is in “the library”. But (a) most people in the world don’t have access to a decent library (if you’re on HN you’re statistically much more likely to), (b) most people have no idea how to use one (same), and most importantly (c) AI will distill the knowledge from ten books you might need to read into step by step simple instructions if you ask.

Re: Making Claude a Chemist

#49

Post by snovv_crash »

smj-edison wrote: post #18

I feel like chemistry is one thing that current models will struggle with for the next while, because it's inherently 3D. In the micro world, shape = function. Maybe enough textual patterns will let it under chemistry, but like how do you describe a hydrogen shift without showing how it moves positions and rebalances bonds?

I work a lot with the geometry side of computer vision (camera calibration, 3d reconstruction etc.) and LLMs have been really bad in this space. They throw stuff at the problem until a minor improvement is made by over fitting on a testset, then gaslight that this is the best thing possible. Then I go do something like working with raster-based data, or some JavaScript based visualization, and it goes super smoothly.
Post reply on HN