Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are the ultimate demoware

blog.charliemeyer.co

Post reply on HN
Print view View original

121–128 of 128 posts

Re: LLMs are the ultimate demoware

#121

Post by Jensson »

rsynnott wrote: post #113
lm28469 wrote: post #70

Earlier quoted context omitted.

They can "feel it", like people "felt" we'd have commercial space flight "soon" after we put people on the moon, it's all delusion and wishful thinking.

It's worse than that, really, because there was at least a fairly obvious _path_ there, even if the economics were, to say the least, shaky. For AGI... not so much.

Yeah, if energy continued to get exponentially more plentiful like it used to then a casual trip to the moon or flying cars today wouldn't have been out of the question.

People imagined a future where everyone had their own personal fusion reactor to power their devices with infinite energy, that world didn't happen but the exponential rate energy technologies used to develop in made that seem feasible.

Re: LLMs are the ultimate demoware

#122

Post by Jensson »

tptacek wrote: post #81
pjc50 wrote: post #34

Earlier quoted context omitted.

Is it always correct?

In my experience, it's 100%. Not 95%, not 99%. Unless GPT5 (and O4-mini) were colluding with Math Academy behind the scenes specifically to be wrong about something, it just doesn't get any of this content wrong. And keep in mind, what it's getting right is trickier than just answering Calc I questions: it's taking an answer I give it, calculating the correct answer itself, selecting its answer over mine, and then sp…

> In my experience, it's 100%. Not 95%, not 99%.

Yeah, they seem to be there on high school math problems today, there aren't that many variations on them and there are billions of examples of data on them so LLM can saturate those.

Just don't assume they are this reliable on solving real world math tasks yet, those are more varied still and stump models.

Re: LLMs are the ultimate demoware

#123

Post by benterix »

tptacek wrote: post #89
benterix wrote: post #88

Earlier quoted context omitted.

While I agree, on an unrelated note - I knew I know your nick from somewhere... And then I realized[0]. [0] https://ludic.mataroa.blog/blog/contra-ptaceks-terrible-arti...

I had a conversation with that person a couple weeks ago. They're nice. I think we both would tweak (if just a little bit) how we presented our articles with the benefit of hindsight. For the record, I'm a systems programmer and a security person and I don't work for an AI company (you can Six Degrees of Sam Altman any startup to AI now if you want to make the claim, but if you try I'm just going to say "Sir, This Is…

> I think we both would tweak (if just a little bit) how we presented our articles with the benefit of hindsight.

Maybe you could present a joint statement of some kind, that would be interesting. I enjoy listening to the arguments of both camps and constantly comparing them to the actual state of things - and my conclusion is, sorry for the cliche, the only constant is change.

Re: LLMs are the ultimate demoware

#124

Post by lm28469 »

Jensson wrote: post #121
rsynnott wrote: post #113

Earlier quoted context omitted.

It's worse than that, really, because there was at least a fairly obvious _path_ there, even if the economics were, to say the least, shaky. For AGI... not so much.

Yeah, if energy continued to get exponentially more plentiful like it used to then a casual trip to the moon or flying cars today wouldn't have been out of the question. People imagined a future where everyone had their own personal fusion reactor to power their devices with infinite energy, that world didn't happen but the exponential rate energy technologies used to develop in made that seem feasible.

It's not an energy question, for the moon it's a "why the fuck would we even do this" question and for flying car it's a thing about bald monkeys being on average quite stupid and not responsible.

Re: LLMs are the ultimate demoware

#125

Post by simonw »

Jensson wrote: post #122
tptacek wrote: post #81

Earlier quoted context omitted.

In my experience, it's 100%. Not 95%, not 99%. Unless GPT5 (and O4-mini) were colluding with Math Academy behind the scenes specifically to be wrong about something, it just doesn't get any of this content wrong. And keep in mind, what it's getting right is trickier than just answering Calc I questions: it's taking an answer I give it, calculating the correct answer itself, selecting its answer over mine, and then sp…

> In my experience, it's 100%. Not 95%, not 99%. Yeah, they seem to be there on high school math problems today, there aren't that many variations on them and there are billions of examples of data on them so LLM can saturate those. Just don't assume they are this reliable on solving real world math tasks yet, those are more varied still and stump models.

They did well at the International Mathematical Olympiad this year.

Re: LLMs are the ultimate demoware

#126

Post by rsynnott »

lm28469 wrote: post #124
Jensson wrote: post #121

Earlier quoted context omitted.

Yeah, if energy continued to get exponentially more plentiful like it used to then a casual trip to the moon or flying cars today wouldn't have been out of the question. People imagined a future where everyone had their own personal fusion reactor to power their devices with infinite energy, that world didn't happen but the exponential rate energy technologies used to develop in made that seem feasible.

It's not an energy question, for the moon it's a "why the fuck would we even do this" question and for flying car it's a thing about bald monkeys being on average quite stupid and not responsible.

Yeah, more or less from the start, commercial space was always more of a “could, but won’t” thing than anything else.

Re: LLMs are the ultimate demoware

#127

Post by kiitos »

simonw wrote: post #115
rsynnott wrote: post #110

Earlier quoted context omitted.

On that basis you'll also be adopting TCM, and homeopathy, and dowsing, and all major religions simultaneously, and all major fad diets simultaneously? Like, "lots of people like this thing and think it is helping them" is not terribly strong evidence that it is actually helping them. It's not a good argument.

How about "lots of people like this thing" where many of those people are credible professionals who I have respected from LONG before they started using LLMs?

there are just as many equally-qualified experts who have been consistently giving exactly the opposite feedback/signal, which you never seem to acknowledge or incorporate into your comments

Re: LLMs are the ultimate demoware

#128

Post by [deleted] »

kiitos wrote: post #127
simonw wrote: post #115

Earlier quoted context omitted.

How about "lots of people like this thing" where many of those people are credible professionals who I have respected from LONG before they started using LLMs?

there are just as many equally-qualified experts who have been consistently giving exactly the opposite feedback/signal, which you never seem to acknowledge or incorporate into your comments

[deleted]
Post reply on HN