Live data from Hacker News

A conversation about AI for science with Jason Pruet

lanl.gov

111–120 of 163 posts

Re: A conversation about AI for science with Jason Pruet

#111
post #60

When will we get AI CEOs? I am tired of seeing corporate decisions driven by quarterly bonuses.

How is this related to TFA which focuses on national (public) labs? In any case, if you want to take a stab at it, feel free to go ahead and start a company whereby you as the owner would assign the CEO responsibilities to an AI.

No they probably meant existing companies... when are AI companies going to get rid of their CEOs?

Re: A conversation about AI for science with Jason Pruet

#112

Earlier quoted context omitted.

Which is actually a problem AI is perfect for.

How so? LLMs will just guess at what sounds like a likely human response to the problem.

Far more to AI than just LLMs. I am talking about more traditional models.

Re: A conversation about AI for science with Jason Pruet

#113
> If you’ve played with the most recent AI tools, you know: They’re very good coders, very good legal analysts, very good first drafters of writing, very good image generators. They’re only going to get better.

Most of the bullshitters will tip their hand pretty early that they're just hype men for AI. Right off the bat, the fact that AI is disruptive and transforming society is apparently self-evident because they never cite a single premise or event to back this up. In the quote above, the phrase "if you've played" stuck out to me. Yes if you play around with them a little it's easy to believe they're really good at so many things. When you stringently evaluate them, you begin to see they make a lot of mistakes and perform inconsistently on even trivial tasks.

Re: A conversation about AI for science with Jason Pruet

#114

> If you’ve played with the most recent AI tools, you know: They’re very good coders, very good legal analysts, very good first drafters of writing, very good image generators. They’re only going to get better. Most of the bullshitters will tip their hand pretty early that they're just hype men for AI. Right off the bat, the fact that AI is disruptive and transforming society is apparently self-evident because they n…

You sound more prejudiced against AI than they seem to be hyping AI in that sentence.

Yes, they’re very good coders; no, they aren’t perfect. Sometimes they make trivial mistakes or hallucinate, and other times they have unique insights about difficult problems or one-shot a lot of complex tasks.

If you don’t see the value of AI even at this point, you’re either lacking imagination and/or just too stuck in your own ways.

Re: A conversation about AI for science with Jason Pruet

#115

> If you’ve played with the most recent AI tools, you know: They’re very good coders, very good legal analysts, very good first drafters of writing, very good image generators. They’re only going to get better. Most of the bullshitters will tip their hand pretty early that they're just hype men for AI. Right off the bat, the fact that AI is disruptive and transforming society is apparently self-evident because they n…

You sound more prejudiced against AI than they seem to be hyping AI in that sentence. Yes, they’re very good coders; no, they aren’t perfect. Sometimes they make trivial mistakes or hallucinate, and other times they have unique insights about difficult problems or one-shot a lot of complex tasks. If you don’t see the value of AI even at this point, you’re either lacking imagination and/or just too stuck in your own w…

I appreciate the value of AI in my own work, but "they’re very good coders" and "they make trivial mistakes or hallucinate" seems incongruous.

For me, AI tools shine when I know enough about a topic to quickly error check, but not enough where I can code fluently without documentation. I'm sure it will get more useful over time, but that's where it's been for me for the last year or so.

Re: A conversation about AI for science with Jason Pruet

#116

Earlier quoted context omitted.

How so? LLMs will just guess at what sounds like a likely human response to the problem.

Far more to AI than just LLMs. I am talking about more traditional models.

Which ones? Machine learning is a larger field, but artificial intelligence is different entirely.

Re: A conversation about AI for science with Jason Pruet

#117

Earlier quoted context omitted.

You sound more prejudiced against AI than they seem to be hyping AI in that sentence. Yes, they’re very good coders; no, they aren’t perfect. Sometimes they make trivial mistakes or hallucinate, and other times they have unique insights about difficult problems or one-shot a lot of complex tasks. If you don’t see the value of AI even at this point, you’re either lacking imagination and/or just too stuck in your own w…

I appreciate the value of AI in my own work, but "they’re very good coders" and "they make trivial mistakes or hallucinate" seems incongruous. For me, AI tools shine when I know enough about a topic to quickly error check, but not enough where I can code fluently without documentation. I'm sure it will get more useful over time, but that's where it's been for me for the last year or so.

> "they’re very good coders" and "they make trivial mistakes or hallucinate" seems incongruous.

It seems that way if you judge them the same way you would human coders, but they’re different. They might be able to do things that veteran coders can’t without spending days on it, and fail at things that beginners can do in half an hour.

A car might not be able to traverse difficult terrain as well as a horse, but it doesn’t mean the car is not a good mode of transportation.

Re: A conversation about AI for science with Jason Pruet

#118

Earlier quoted context omitted.

I appreciate the value of AI in my own work, but "they’re very good coders" and "they make trivial mistakes or hallucinate" seems incongruous. For me, AI tools shine when I know enough about a topic to quickly error check, but not enough where I can code fluently without documentation. I'm sure it will get more useful over time, but that's where it's been for me for the last year or so.

> "they’re very good coders" and "they make trivial mistakes or hallucinate" seems incongruous. It seems that way if you judge them the same way you would human coders, but they’re different. They might be able to do things that veteran coders can’t without spending days on it, and fail at things that beginners can do in half an hour. A car might not be able to traverse difficult terrain as well as a horse, but it do…

Yes, human coders are predictably 'bad' - it scales roughly with the complexity of the task and the years of their experience, along with some randomness ('brain farts'). LLMs are completely unpredictably 'bad' - sometimes a task that a high schooler can solve sends an LLM into a spin, sometimes an LLM solves a task that would require a PhD in the field. The unpredictability is what makes these tools so unfit for purpose.

Re: A conversation about AI for science with Jason Pruet

#119

> If you’ve played with the most recent AI tools, you know: They’re very good coders, very good legal analysts, very good first drafters of writing, very good image generators. They’re only going to get better. Most of the bullshitters will tip their hand pretty early that they're just hype men for AI. Right off the bat, the fact that AI is disruptive and transforming society is apparently self-evident because they n…

I don’t understand this viewpoint at all. Even if AI tools are nothing more than complex autocomplete systems - and not some kind of new consciousness - that alone is enough to dramatically shift entire industries. And it already is doing so…this isn’t theoretical anymore.

Re: A conversation about AI for science with Jason Pruet

#120

Earlier quoted context omitted.

I appreciate the value of AI in my own work, but "they’re very good coders" and "they make trivial mistakes or hallucinate" seems incongruous. For me, AI tools shine when I know enough about a topic to quickly error check, but not enough where I can code fluently without documentation. I'm sure it will get more useful over time, but that's where it's been for me for the last year or so.

> "they’re very good coders" and "they make trivial mistakes or hallucinate" seems incongruous. It seems that way if you judge them the same way you would human coders, but they’re different. They might be able to do things that veteran coders can’t without spending days on it, and fail at things that beginners can do in half an hour. A car might not be able to traverse difficult terrain as well as a horse, but it do…

I'm not sure if the analogy fits. I would say AI is more like a fast car that turns the wrong way every couple of blocks, or a car that randomly refuses to go on certain roads. Is it still a useful mode of transportation? Sometimes, but, it's not reliable.

The sibling comment says it better - it's about predicability of mistakes, and the effort and knowledge necessary to correct them.

Maybe the next generation of models will solve these problems, who knows.

Post reply on HN