Live data from Hacker News

LLMs are the ultimate demoware

blog.charliemeyer.co

71–80 of 128 posts

Re: LLMs are the ultimate demoware

#71
post #53

Earlier quoted context omitted.

Can't your stack do that for you? The one I use creates the migrations, locally, for free and deterministically in about 30 seconds.

I'm using raw Postgres. I don't like a lot of dependencies or "stacks" where they don't need to be. Claude does this extremely efficiently.

You say you don't like stacks, and that you don't need them.

But you pay a subscription to run a stack of software in anothers people computer.

btw wtf is raw postgres?

Re: LLMs are the ultimate demoware

#72
post #4

Again and again people keep saying this while many of us keep using LLMs to create value.

If the value produced was so great I think we've be able to measure it by now, or at least see something. If you remove the hype around AI the economy is actually on the way down, productivity wasn't measured to have increased since llm became mainstream either.

Lot's of vibes and feelings but 0 measurable impact

Re: LLMs are the ultimate demoware

#73
post #29

Earlier quoted context omitted.

What makes you think https://www.mathacademy.com/faq hadn't been evaluated by experts? That appears to be their whole thing, and they've been in business for longer than LLMs have been around.

I think before that question is useful to ask, we have to know if that FAQ even says anything about LLM-based tutoring. After a few minutes of research, I can't find any evidence that Math Academy offers LLM-based tutoring.

You're confused. Math Academy isn't LLM-based. I use an LLM alongside it.

Re: LLMs are the ultimate demoware

#74

Earlier quoted context omitted.

Not sure the condescending tone is really necessary. I’d agree with you if the parent comment was saying they asked an LLM to create a math curriculum and problems for them. But they’re using an established app created by a math major and then using LLMs to ask questions. It’s easier to validate the responses you get back in those cases.

I think students are not a reliable source of information about the effectiveness of LLM tutoring. There is no 100% nice way to say this, but I did my best. You're free to disagree, but I think the tone criticism is off-base.

We found our way to "No True Math Student". I love it!

Re: LLMs are the ultimate demoware

#75
post #26
post #18

Earlier quoted context omitted.

Countless people in comments say this, but other people fail to see evidence of that in the wild. As has been said in response to this point many times in the past: Where's the open source renaissance that should be happening right now? Where are the actual, in-use dependencies and libraries that are being developed by AI? The only times I've personally seen LLMs engaged in repos has been handling issues, and they ma…

Let's look at every PR on GitHub in public repos (many of which are likely to be under open source licenses) that may have been created with LLM tools, using GitHub Search for various clues: GitHub Copilot: 247,000 https://github.com/search?q=is%3Apr+author%3Acopilot-swe-age... - is:pr author:copilot-swe-agent[bot] Claude: 147,000 https://github.com/search?q=is%3Apr+in%3Abody+%28%22Generate... - is:pr in:body ("Gener…

HN people: lines of code and numbers of PRs are irrelevant to determine the capabilities of a developer.

Also HN people: look at the magic slop machine, it made all these lines of codes and PRs, it is irrefutable proof that it's good and AGI

Re: LLMs are the ultimate demoware

#76
post #17
post #7

It's wild to me that, of all the things to call LLMs out for, this piece has chosen to include math tutoring. I've been doing Math Academy for a bit over 6 months now, going from (essentially) Algebra II through Calc II (integration by parts, arc lengths, Taylor expansions) and LLMs have been a huge part of what has made that effective: * Clear explanation of concepts that respond to questions and reformulate when th…

This https://www.mathacademy.com/ ? Interesting, hadn't seen that before. I've been thinking I'd like to brush up on a bunch of those topics.

Wholeheartedly recommend it, just remember we're not the core market for it (that's high school students, though the curricula goes all the way through the normal college math sequence).

Minutes later

In case I've spooked anyone, they have an adult course series (Foundations I, II, and III) that's accelerated by trimming out all the material their authors believe are important only for things like school placement exams; the modal adult Math Academy person is doing I, II, and III as a leadup to their Math for Machine Learning course, which is linear algebra and multivariable calc.

I think it's one of the three most mindblowing learning resources I've ever used. One of the other three: Lingua Latina Familia Romana. In both cases, I have the uncanny certainty that I am operating at the limit of my ability to acquire and retain new information, which is a fun place to be.

Re: LLMs are the ultimate demoware

#77
post #63

Earlier quoted context omitted.

I've used LLMs to try to help digest some advanced maths. Eg. "Explain the number field seive with lots of numeric examples". Yes the numeric examples often don't work. The consequences of this though are similar to a failed web search. As in it's not a big deal and when it does work it's very helpful. Maths is one of those things with so much objectivity that even the LLM usually realizes it has failed to create a n…

A LLM can't "realize" anything. Unless you are saying that LLMs are aware.

It's a term i used to explain that in 'thinking' mode LLMs will read their own output and call out things like incorrect math statements before posting to the user.

Now you probably want a debate about the term 'thinking' mode but i cbf with that. It's pretty clear what was meant and semantic arguments suck. Don't do that.

Re: LLMs are the ultimate demoware

#78

LLMs are useful if you use them properly and they are getting better everyday. Arguing against LLMs is like arguing against a shovel. Just use it right.

A lot of arguing "against LLMs" is not arguing "shovels aren't useful," it's arguing "maybe shovels aren't actually going to replace all human labor, and sinking so much capital into it we're starting to conceptualize it in terms of 'percent of global GDP' might not be such a great idea."

Re: LLMs are the ultimate demoware

#79
post #63

Earlier quoted context omitted.

A LLM can't "realize" anything. Unless you are saying that LLMs are aware.

It's a term i used to explain that in 'thinking' mode LLMs will read their own output and call out things like incorrect math statements before posting to the user. Now you probably want a debate about the term 'thinking' mode but i cbf with that. It's pretty clear what was meant and semantic arguments suck. Don't do that.

I want people to use correct terms, i don't think that is unreasonable.

Re: LLMs are the ultimate demoware

#80
post #18
post #4

Again and again people keep saying this while many of us keep using LLMs to create value.

Countless people in comments say this, but other people fail to see evidence of that in the wild. As has been said in response to this point many times in the past: Where's the open source renaissance that should be happening right now? Where are the actual, in-use dependencies and libraries that are being developed by AI? The only times I've personally seen LLMs engaged in repos has been handling issues, and they ma…

> Countless people in comments say this, but other people fail to see evidence of that in the wild. As has been said in response to this point many times in the past: Where's the open source renaissance that should be happening right now? Where are the actual, in-use dependencies and libraries that are being developed by AI?

The thing that this comment misses, imo, is that LLMs are not always enabling people who previously couldn't create value to create value. In fact i think they are likely to cause some people who created value previously to create even less value!

However that's not mutually exclusive with enabling others to create more value than they did previously. Is it a net gain for society? Currently I'd bet not, by a large margin. However is it a net gain for some individual users of LLMs? I suspect yes.

LLMs are a powerful tool for the right job, and as time goes on the "right job" keeps expanding to more territory. The problem is it's a tool that takes a keen eye to analyze and train on. It's not easy to use for reliable output. It's currently a multiplier for those willing to use it on the right jobs and with the right training (reviews, suspicion, etc).

Post reply on HN