Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are the ultimate demoware

blog.charliemeyer.co

Post reply on HN
Print view View original

31–40 of 128 posts

Re: LLMs are the ultimate demoware

#31

Post by bgwalter »

The initial ChatGPT release in 2022 was the product of 7 years of private research that in turn built on decades of public research.

Rumors say that Google wasn't far behind at the time, but didn't push releases. Perhaps because they were not that impressed by the applications or did not want "AI" to cannibalize their other products.

So it seems very likely that everything has been squeezed out of the decades of research and we have plateaued.

Desperate measures like Nvidia buying its own graphics cards through circular investment schemes do not inspire confidence either. Or Microsoft now doing CoPilot product placement ads in teenager YouTube channels. When Google launched, people just used it because it was good. This all fits very well with the demoware angle of the article.

Re: LLMs are the ultimate demoware

#32

Post by simonw »

soperj wrote: post #27
book_mike wrote: post #24

LLMs are useful if you use them properly and they are getting better everyday. Arguing against LLMs is like arguing against a shovel. Just use it right.

I haven't noticed them getting any better in the last year.

You absolutely have not been paying attention then. The difference in quality between September 2025 LLMs (GPT-5, Claude 4/4.5) and September 2024 (we were still on GPT-4o) is huge.

For one thing, last year's LLMs were nowhere near winning gold on collegiate math and programming competitions. That's because the "reasoning" thing hadn't kicked off yet - the first model to demonstrate that trick was o1 in ... OK that was September 12th 2024 so it just makes it to a year old now.

Re: LLMs are the ultimate demoware

#33

Post by worldsayshi »

getnormality wrote: post #21
tptacek wrote: post #7

It's wild to me that, of all the things to call LLMs out for, this piece has chosen to include math tutoring. I've been doing Math Academy for a bit over 6 months now, going from (essentially) Algebra II through Calc II (integration by parts, arc lengths, Taylor expansions) and LLMs have been a huge part of what has made that effective: * Clear explanation of concepts that respond to questions and reformulate when th…

It's nice that you think it's clear and responsive, but I think it [1] needs to be validated by an expert in both the material and education. Or we need some way to show that people have actually learned the topic. People sometimes prefer explanations that are intuitive and familiar but not accurate. Meanwhile, there are math education resources like iXL that maybe cost a little money but the lessons and practice pro…

I agree that LLM output need to be validated to be valuable but math (unless it's on a quite high level I suppose) seems like one of the areas with the most potential for doing validations, without requiring an expert to validate everything.

If you're working on educational math problems with solutions you can validate against the solutions. If you're working with proofs you can evaluate the proofs in a proof checker. Or you can run the resulting math expressions through a calculator.

Re: LLMs are the ultimate demoware

#34

Post by pjc50 »

tptacek wrote: post #7

It's wild to me that, of all the things to call LLMs out for, this piece has chosen to include math tutoring. I've been doing Math Academy for a bit over 6 months now, going from (essentially) Algebra II through Calc II (integration by parts, arc lengths, Taylor expansions) and LLMs have been a huge part of what has made that effective: * Clear explanation of concepts that respond to questions and reformulate when th…

Is it always correct?

Re: LLMs are the ultimate demoware

#35

Post by pjc50 »

simonw wrote: post #26
jdiff wrote: post #18

Earlier quoted context omitted.

Countless people in comments say this, but other people fail to see evidence of that in the wild. As has been said in response to this point many times in the past: Where's the open source renaissance that should be happening right now? Where are the actual, in-use dependencies and libraries that are being developed by AI? The only times I've personally seen LLMs engaged in repos has been handling issues, and they ma…

Let's look at every PR on GitHub in public repos (many of which are likely to be under open source licenses) that may have been created with LLM tools, using GitHub Search for various clues: GitHub Copilot: 247,000 https://github.com/search?q=is%3Apr+author%3Acopilot-swe-age... - is:pr author:copilot-swe-agent[bot] Claude: 147,000 https://github.com/search?q=is%3Apr+in%3Abody+%28%22Generate... - is:pr in:body ("Gener…

What's the acceptance rate on such PRs?

Re: LLMs are the ultimate demoware

#36

Post by pizlonator »

simonw wrote: post #26
jdiff wrote: post #18

Earlier quoted context omitted.

Countless people in comments say this, but other people fail to see evidence of that in the wild. As has been said in response to this point many times in the past: Where's the open source renaissance that should be happening right now? Where are the actual, in-use dependencies and libraries that are being developed by AI? The only times I've personally seen LLMs engaged in repos has been handling issues, and they ma…

Let's look at every PR on GitHub in public repos (many of which are likely to be under open source licenses) that may have been created with LLM tools, using GitHub Search for various clues: GitHub Copilot: 247,000 https://github.com/search?q=is%3Apr+author%3Acopilot-swe-age... - is:pr author:copilot-swe-agent[bot] Claude: 147,000 https://github.com/search?q=is%3Apr+in%3Abody+%28%22Generate... - is:pr in:body ("Gener…

The main problem with your search methodology is that maybe AI is good at generating a high volume of slop commits.

Slop commits are not unique to AI. Every project I’ve worked on had that person who has high commit count and when you peek at the commits they are just noise.

I’m not saying you’re wrong btw. Just saying this is a possible hole in the methodology

Re: LLMs are the ultimate demoware

#38

Post by getnormality »

fourside wrote: post #30
getnormality wrote: post #21

Earlier quoted context omitted.

It's nice that you think it's clear and responsive, but I think it [1] needs to be validated by an expert in both the material and education. Or we need some way to show that people have actually learned the topic. People sometimes prefer explanations that are intuitive and familiar but not accurate. Meanwhile, there are math education resources like iXL that maybe cost a little money but the lessons and practice pro…

Not sure the condescending tone is really necessary. I’d agree with you if the parent comment was saying they asked an LLM to create a math curriculum and problems for them. But they’re using an established app created by a math major and then using LLMs to ask questions. It’s easier to validate the responses you get back in those cases.

I think students are not a reliable source of information about the effectiveness of LLM tutoring. There is no 100% nice way to say this, but I did my best. You're free to disagree, but I think the tone criticism is off-base.

Re: LLMs are the ultimate demoware

#39

Post by voidhorse »

tptacek wrote: post #7

It's wild to me that, of all the things to call LLMs out for, this piece has chosen to include math tutoring. I've been doing Math Academy for a bit over 6 months now, going from (essentially) Algebra II through Calc II (integration by parts, arc lengths, Taylor expansions) and LLMs have been a huge part of what has made that effective: * Clear explanation of concepts that respond to questions and reformulate when th…

Generating problems is fantastic, but I'd caution on overreliance in the other two cases.

Basically all of the cognitive science literature on learning that I am aware of says that the more you do directly and the less hand holding you are given, the better your acquisition and long term retention. In particular, having the LLM elaborate concepts for you is probably one of the worst things you can do when it comes to learning. Struggling through that elaboration process yourself is going to make the learning stick much more strongly, at least if all of the research is to be believed.

Re: LLMs are the ultimate demoware

#40

Post by pizlonator »

jdiff wrote: post #18
eitland wrote: post #4

Again and again people keep saying this while many of us keep using LLMs to create value.

Countless people in comments say this, but other people fail to see evidence of that in the wild. As has been said in response to this point many times in the past: Where's the open source renaissance that should be happening right now? Where are the actual, in-use dependencies and libraries that are being developed by AI? The only times I've personally seen LLMs engaged in repos has been handling issues, and they ma…

What would it mean to see it in the wild?

I think that highly productive people who have incorporated LLMs into their workflows are enjoying a productivity multiplier.

I don’t think it’s 2x but it’s greater than 1x, if I had to guess. It’s just one of those things that’s impossible to measure beyond reasonable doubt

Post reply on HN