Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are the ultimate demoware

blog.charliemeyer.co

Post reply on HN
Print view View original

101–110 of 128 posts

Re: LLMs are the ultimate demoware

#101

Post by clueless »

slaterbug wrote: post #5
clueless wrote: post #3

"then fails to consistently help in completing tasks when deployed for daily use." This article seems to be baitware trying to push some outdated perspective. LLMs have only gotten more powerful over the last 3 years (being able to do more things), and so far not much has stopped them from becoming even more powerful (with the help of reasoning, other external methods, etc) in the future. "daily use" is so subjective…

What evidence is there that AGI will come “soon”?

What evidence did we have that LLMs would be such transformative techs before they were suddenly introduced, and have such surprising behaviors? Not sure we need to always be looking for evidence for potentially surprising and disruptive tech

Re: LLMs are the ultimate demoware

#102

Post by thegrim33 »

tptacek wrote: post #7

It's wild to me that, of all the things to call LLMs out for, this piece has chosen to include math tutoring. I've been doing Math Academy for a bit over 6 months now, going from (essentially) Algebra II through Calc II (integration by parts, arc lengths, Taylor expansions) and LLMs have been a huge part of what has made that effective: * Clear explanation of concepts that respond to questions and reformulate when th…

So .. a person who doesn't know X, is using LLMs to learn X, yet is able to judge that LLMs are doing a good job at teaching X, even though the person doesn't know X?

Re: LLMs are the ultimate demoware

#103

Post by tptacek »

thegrim33 wrote: post #102
tptacek wrote: post #7

It's wild to me that, of all the things to call LLMs out for, this piece has chosen to include math tutoring. I've been doing Math Academy for a bit over 6 months now, going from (essentially) Algebra II through Calc II (integration by parts, arc lengths, Taylor expansions) and LLMs have been a huge part of what has made that effective: * Clear explanation of concepts that respond to questions and reformulate when th…

So .. a person who doesn't know X, is using LLMs to learn X, yet is able to judge that LLMs are doing a good job at teaching X, even though the person doesn't know X?

You're confused. Math Academy is not an LLM.

Re: LLMs are the ultimate demoware

#104

Post by simonw »

lm28469 wrote: post #75
simonw wrote: post #26

Earlier quoted context omitted.

Let's look at every PR on GitHub in public repos (many of which are likely to be under open source licenses) that may have been created with LLM tools, using GitHub Search for various clues: GitHub Copilot: 247,000 https://github.com/search?q=is%3Apr+author%3Acopilot-swe-age... - is:pr author:copilot-swe-agent[bot] Claude: 147,000 https://github.com/search?q=is%3Apr+in%3Abody+%28%22Generate... - is:pr in:body ("Gener…

HN people: lines of code and numbers of PRs are irrelevant to determine the capabilities of a developer. Also HN people: look at the magic slop machine, it made all these lines of codes and PRs, it is irrefutable proof that it's good and AGI

Both of these things can be true at the same time:

1. Counting lines of code is a bad way to measure developer productivity.

2. The number of merged PRs on GitHub overall that were created with LLM assistance is an interesting metric for evaluating how widely these tools are being used.

Re: LLMs are the ultimate demoware

#105

Post by Ancapistani »

skydhash wrote: post #16
turnsout wrote: post #8

This is such a weak take to read while I have Claude Code running in the background creating a new database migration for a feature we're building

How much time to create a new database migration, like for actually typing it?

I'd estimate that using AI to implement a well-defined feature typically takes about 1.5x the time for me - and yet I still use AI extensively.

The key difference is that I can context switch. Once the AI has context and is doing its thing, I can move on to another task that's not working in the same area or project. I can post on HN. I can catch up on my Slack inbounds, or my email.

Having two tasks running at once nets a small but nice improvement in velocity. Having any tasks running while I'm doing other things effectively doubles my output.

Re: LLMs are the ultimate demoware

#106

Post by tptacek »

voidhorse wrote: post #39
tptacek wrote: post #7

It's wild to me that, of all the things to call LLMs out for, this piece has chosen to include math tutoring. I've been doing Math Academy for a bit over 6 months now, going from (essentially) Algebra II through Calc II (integration by parts, arc lengths, Taylor expansions) and LLMs have been a huge part of what has made that effective: * Clear explanation of concepts that respond to questions and reformulate when th…

Generating problems is fantastic, but I'd caution on overreliance in the other two cases. Basically all of the cognitive science literature on learning that I am aware of says that the more you do directly and the less hand holding you are given, the better your acquisition and long term retention. In particular, having the LLM elaborate concepts for you is probably one of the worst things you can do when it comes to…

I understand that. The core of the pedagogical approach here is Math Academy, not LLMs. (Math Academy isn't an LLM; it's a spaced-repetition accelerated curriculum centered on graded problem set submissions). But the LLM functions exactly the way a tutor would in a math course, and for that application, LLMs have become extremely effective; arguably more effective than human tutors.

Re: LLMs are the ultimate demoware

#107

Post by Blackthorn »

baklavaEmperor wrote: post #85
Blackthorn wrote: post #51

Earlier quoted context omitted.

Wanting an actual check on the device that is notorious for making things up is gatekeeping now?

You’re projecting a bad faith use case that the original commenter never described. they’re using it in a exploratory and iterative way, not deferential.

If you're using it for education it is by definition deferential.

Re: LLMs are the ultimate demoware

#108

Post by simonw »

thegrim33 wrote: post #102
tptacek wrote: post #7

It's wild to me that, of all the things to call LLMs out for, this piece has chosen to include math tutoring. I've been doing Math Academy for a bit over 6 months now, going from (essentially) Algebra II through Calc II (integration by parts, arc lengths, Taylor expansions) and LLMs have been a huge part of what has made that effective: * Clear explanation of concepts that respond to questions and reformulate when th…

So .. a person who doesn't know X, is using LLMs to learn X, yet is able to judge that LLMs are doing a good job at teaching X, even though the person doesn't know X?

There are many, many things in life where you can evaluate if you are learning the thing despite not having access to an expert guide who can verify what you are learning.

Cooking: does the food taste better as you learn more?

Programming: are you able to build functioning software that does what you want it to do, better than you could earlier on in your path?

Fixing a broken dishwasher: does the dishwasher work again now?

The idea that learning only works if you have an expert on hand to verify that you are learning is one of those things that seems obviously true until you think harder about it.

Re: LLMs are the ultimate demoware

#109

Post by tptacek »

Blackthorn wrote: post #107
baklavaEmperor wrote: post #85

Earlier quoted context omitted.

You’re projecting a bad faith use case that the original commenter never described. they’re using it in a exploratory and iterative way, not deferential.

If you're using it for education it is by definition deferential.

No it isn't. Again, what's happening here I think is that this thread doesn't understand what Math Academy is. It's not an LLM. I'm using the LLM alongside it.

Re: LLMs are the ultimate demoware

#110

Post by rsynnott »

empath75 wrote: post #10
tptacek wrote: post #7

It's wild to me that, of all the things to call LLMs out for, this piece has chosen to include math tutoring. I've been doing Math Academy for a bit over 6 months now, going from (essentially) Algebra II through Calc II (integration by parts, arc lengths, Taylor expansions) and LLMs have been a huge part of what has made that effective: * Clear explanation of concepts that respond to questions and reformulate when th…

It seems very hard to maintain the belief that LLMs are useless in the face of the fact that millions of people are using them. It's very much "nobody goes there anymore, it's too crowded"

On that basis you'll also be adopting TCM, and homeopathy, and dowsing, and all major religions simultaneously, and all major fad diets simultaneously? Like, "lots of people like this thing and think it is helping them" is not terribly strong evidence that it is actually helping them. It's not a good argument.
Post reply on HN