Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let's talk about LLMs

b-list.org

Post reply on HN
Print view View original

161–170 of 201 posts

Re: Let's talk about LLMs

#161

Post by aspenmartin »

slopinthebag wrote: post #150
aspenmartin wrote: post #115

Earlier quoted context omitted.

See chinchilla scaling laws, we have the functional form of the curve and know the constants (though they change and are domain and model specific): L(N,D) ~= 1.69 + 406 / N^0.339 + 411 / D^0.285 L is loss (pre training test loss) D is the scale of the data N is the number of model parameters

You need to touch grass dude, seriously.

Why deflect from the conversation and attempt to insult someone? What I’m saying is literally canonical and extremely well known literature.

Re: Let's talk about LLMs

#162

Post by aspenmartin »

nothinkjustai wrote: post #131
aspenmartin wrote: post #99

Earlier quoted context omitted.

I would argue LLMs are possibly the largest paradigm shift the world has ever seen, and we are only at the beginning. The entire scaffolding and structure of programming is in the process of changing — coding has moved to orchestration and testing and governance of how to manage and productionalize code that has surpassed the capacity of human review. If this sounds melodramatic it’s likely that it hasn’t fully taken…

No offense but this reads like AI psychosis

Well I’m not offended but it sounds like you may not be paying attention? Do you know the capital outlay that has gone into infra buildouts? several people here have described “6 months” of AI mania—-the fact that people are saying 6 months is exactly the point. Development has been going on since 2010s. All of the “boosters” as HN likes to say have been saying “hey this thing is huge and the performance trends are startling, get ready” and people then say “that’s psychotic I can’t even get Siri to understand my name”. Sure enough, 6 months ago we hit a performance inflection point where “madness” has begun. That’s just when you started paying attention, the rate of change has not stopped. Pretty easy to predict what happens next…

Re: Let's talk about LLMs

#163

Post by aspenmartin »

jatora wrote: post #140
aspenmartin wrote: post #107

Earlier quoted context omitted.

I’m going to parrot back what you’re saying and you tell me if I’m getting close - AI coding is a disappointing fad (“fever dream?”). - that has not made meaningful progress in…6 months? - coding harness is improving - model improvements are lies: it’s just businesses “benchmaxxing” and misleading people. Real performance has not meaningfully improved - “opus 4.7 is a dud” - 5.5 suffering from “system collapse” (I’ve…

I'm pretty sure their mindset is pure cope. All top AI labs are agentically coding 100% now. There's a reason for that. Anyone not on that paradigm yet is either slow acting or purposefully resistant. (excluding workplace policies that hamstring you of course)

Yea that’s what I just can’t wrap my mind around. It’s a cacophony of engineers with authoritative sounding blog posts explaining a subject they seem to have barely a tenuous grasp on. It’s hard to watch a population of tech people I used to really revere getting things so wrong. I thought “surely once we’re no one with any self respect would still claim AI is a useless fad or that it shouldn’t be used” and yet to my disappointment that’s where we seem to be.

Re: Let's talk about LLMs

#164

Post by taneq »

aspenmartin wrote: post #162
nothinkjustai wrote: post #131

Earlier quoted context omitted.

No offense but this reads like AI psychosis

Well I’m not offended but it sounds like you may not be paying attention? Do you know the capital outlay that has gone into infra buildouts? several people here have described “6 months” of AI mania—-the fact that people are saying 6 months is exactly the point. Development has been going on since 2010s. All of the “boosters” as HN likes to say have been saying “hey this thing is huge and the performance trends are s…

Maybe it’s time to ask Siri again, “hey are you smart yet or are you still just a script?”

If she says “I’m sorry, I don’t know how to are you still just a script” then I have my answer. :P

LLMs are remarkable these days but they’re still missing a some essential insight. I’m far less confident now, though, that this will require another big breakthrough and not just a combination of tweaks.

Re: Let's talk about LLMs

#165

Post by dijksterhuis »

eiekeww wrote: post #51
dijksterhuis wrote: post #9

Earlier quoted context omitted.

it’s felt like the new crypto to me for about 2-3 years now. i was doing an ML Sec phd a year or two before all this hype took off. i took one of the OG transformer papers along to present at our official little phd reading group when the paper was only a few months old (the details of this might be a bit sketchy here, was years ago now). now i want nothing to do with the field in any way shape or form. i’m just done…

Sadly investing is all about making money… you should be more pissed at the naive people who have contributed to the effort and in particular those who don’t care about truth, but about cash flow potential.

everyone involved is responsible, just to different degrees.

Re: Let's talk about LLMs

#166

Post by weakfish »

marcus_holmes wrote: post #94

I was waiting for the "so I tried coding something with an LLM myself, and I found..." paragraph. But apparently the author never did try it, or at least if they did, they didn't write about it. This is a very academic approach to the subject - read what other people have written about it without ever doing it yourself. Study what someone said about LLM coding 50 years ago, before they were even invented, to see what…

Are you asking, essentially, to move past the data and evidence and get anecdotes? That seems the opposite of useful, and tbh LLM coding has wayyyy too much anecdotal 'evidence' going on.

Re: Let's talk about LLMs

#167

Post by weakfish »

jatora wrote: post #78
strange_quark wrote: post #69

Earlier quoted context omitted.

What rapid improvement has occurred, because in this six month AI coding fever dream we've been living in, I really haven't seen anything new in awhile, both in terms of new ideas for AI coding or in new consumer products or services. I'll give you the coding harnesses themselves are better because that was a new product category with a lot of low-hanging fruit, but have the models actually improved in a way that isn…

Uninformed opinion of someone who clearly doesnt consistently use AI coding tools, clearly. And why are you limiting it to 6 months? Whats wrong with you?

Why does this _always_ happen in agentic coding convos?

> I don't find $MODEL useful

> CLEARLY you're doing it wrong

It's so dumb.

(I write code w/ agents btw, I'm just also skeptical)

Re: Let's talk about LLMs

#168

Post by aspenmartin »

taneq wrote: post #164
aspenmartin wrote: post #162

Earlier quoted context omitted.

Well I’m not offended but it sounds like you may not be paying attention? Do you know the capital outlay that has gone into infra buildouts? several people here have described “6 months” of AI mania—-the fact that people are saying 6 months is exactly the point. Development has been going on since 2010s. All of the “boosters” as HN likes to say have been saying “hey this thing is huge and the performance trends are s…

Maybe it’s time to ask Siri again, “hey are you smart yet or are you still just a script?” If she says “I’m sorry, I don’t know how to are you still just a script” then I have my answer. :P LLMs are remarkable these days but they’re still missing a some essential insight. I’m far less confident now, though, that this will require another big breakthrough and not just a combination of tweaks.

I accept your skepticism is all I can say but just consider we’re not talking about the most important numbers and topics in this conversation. We have a lot of mileage left in the current stack. Nothing is plateauing though you wouldn’t know it if you read HN.

Re: Let's talk about LLMs

#169

Post by brazukadev »

aspenmartin wrote: post #160
brazukadev wrote: post #128

Earlier quoted context omitted.

> Every major tech company - the ones poised to get the first best rewards, have already gotten good incremental revenue from AI via ads ranking/recommendations (Google, Meta, etc.) That's just software evolving. It happened before LLMs, it would happen without LLMs. > good productivity increases due to scale of workforce and advanced in house tooling. Exactly same case.

But I don’t really understand: the ask is for evidence AI is generating meaningful returns and it demonstrably is, even while we have integrated these tools only partially. “Just software evolving” um yes, I agree, just that now this happens faster and more efficiently. It is also more than that: models that power advertising and content recommendation at TikTok, Google, Facebook, Instagram, etc are not just “softwar…

Yes, it is meaningful improvements but AI changes working profitable software. It is more difficult to create exponential value from that compared to new platforms like the internet or mobile. Git and then github, for example, have a much bigger impact on increasing software development productivity than AI, with a fraction of the investment.

Re: Let's talk about LLMs

#170

Post by hombre_fatal »

prmph wrote: post #145
hombre_fatal wrote: post #72

Earlier quoted context omitted.

I read that section but I disagree with it. 1. You don't have to be an LLM expert to get good, consistent results with LLMs. My best vibe-code process after years of using LLMs is to have Claude Code create a plan file and then cycle it through Codex until Codex finds nothing more to review, then have an agent implement it. This process is trivial yet produces amazing results. It's solved by better and better harness…

This sounds like someone who have never had to write serious software. > 1. You don't have to be an LLM expert to get good, consistent results with LLMs. You don't get good consistent results with LLMs, expert or not > 2. You don't have to write technical specs. The LLM does that for you. You just tell it "I want the next-tab button to wrap back to the first one" and it generates a technical plan. Natural language is…

> This sounds like someone who have never had to write serious software.

Why the insult? You never know who you're talking to on HN.

Your points have to do with process failure, not intractable LLM limitations. Most of which already apply to human-conceived software.

Your "Performance Optimizations" bit exemplifies this since you baked in the assumption that it will have no connection with your project. Well, why not? You need to figure out how to use your source code and relevant data as ground truth when working with LLMs.

A markdown viewer is on the simpler side of things I've built with LLMs, so this too suggests that you have a weak process. A common mistake is to expect LLMs to one-shot everything (the spec, the plan, or the actual impl). Instead you should use LLMs to review-revise-cycle one of those until it's refined, ideally the spec/plan since impl is derived from it. You will have much better and consistent results.

I recommend finding an engineer you respect/trust that has found a way to build good software with LLMs, and then tap them for their process.

Post reply on HN