Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLM Critics Are Right. I Use LLMs Anyway

theocharis.dev

Post reply on HN
Print view View original

121–130 of 330 posts

Re: The LLM Critics Are Right. I Use LLMs Anyway

#121

Post by jszymborski »

> I tend not to actually read most LLM output anymore; I skim it, to check if I vibe with it. But a problem statement of three sentences, that I will fact-check really hard. It is like code review: a review with 1,000 lines of code gets an “LGTM”. A review with 100 lines gets 15 comments.

This is where I take issue. I'm in a similar boat to the author. In the last couple of months, I've been experimenting with increasing the use of local and cloud LLMs for my research code. I'll create a prototype, maybe port it to a language I don't use very much like Rust, run some tests... but at the very end when I'm very happy with it, I _need_ to go line by line and understand _everything_ that is happening. Sometimes that means using an LLM to understand it, but even when I do and there is a concept I don't get, I try to read primary resources written by experts.

The least bad thing I've found LLMs good for is ideation because it's super easy to take the good nuggets and leave the bad, but even that carries risks of shaping thought and making everyone reach for and ignore the same ideas in the way the Spotify radio or YouTube autoplay has been shaping/flattening tastes for the worse.

I'm not sure what I'll rule at the end of my experiments with LLMs, but right now I'm enjoying the rush of having prototypes that run quickly. I've always been a top-down learner, being motivated by hacking a cool demo I half understand and progressively tearing it apart.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#122

Post by breadzeppelin__ »

Diogenesian wrote: post #109
breadzeppelin__ wrote: post #100

Earlier quoted context omitted.

I am both a career developer and experienced team manager. from first hand experience the 1.5x im getting from AI is not flowing down to my team / org because why would i output 50% more when the pay environment and leadership are already underwhelming. That additional 50% productivity goes completely to side projects built on my second computer between 9-5 tasks

Actually, the METR report speculates that some of the overreported productivity uplift comes from grabbing unnecessary low-hanging fruit, things like "oh I'll make a web dashboard to keep track of this stuff // wow that would have taken all day without Claude!" But in the olden days they would have just used a notepad. Yet psychologically they built a real thing and saved a lot of time.

True I have made a few low tier apps that just hit apis that were previously obfuscated deep in menus that have had an outsized impact

Re: The LLM Critics Are Right. I Use LLMs Anyway

#123

Post by cj »

quantummagic wrote: post #103
dd8601fn wrote: post #101

Earlier quoted context omitted.

And someone else always cries luddite if anyone says, “Hey guys, this isn’t what the salesman said.” What matters is the reality of the thing… which is exactly what everyone is discussing. As they should. We need more conversations like this, early. It could have saved us decades of bad practices with impossible expectations (see: agile).

But people can't separate their emotions from the "reality". The reality is, this technology has strengths and limitations. It has benefits and it has negative consequences. We can and will discuss all of that. But many people aren't operating from a detached analytical viewpoint. They're operating from an emotional, self-interested, defensive stance. And granted, there are just as many people operating from a utopia…

> They're not wrong, they're just not helpful or accomplishing anything.

I think what it accomplishes is developing a shared understanding of the pros/cons of the tech, the risks associated, how we should mitigate those risks, where the tech is appropriate vs. not appropriate, etc.

It feels very dismissive to say that these conversations aren't worth having. Or that we're thinking about these things too much.

I get that it's tiring, especially in the HN echo chamber. But the conversations are still worth having IMO.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#124

Post by Rumudiez »

matsemann wrote: post #48

My use of AI is correct. My coworkers use is not. I'm saying it in jest, but it's also a bit true. Not necessarily because we use it any differently. But because my use of AI saves me time. But their use of AI adds more to my plate, no matter if it's slop or not.

I noticed very recently I actually prefer reviewing other people’s generated code over handwritten. There’s no ego involved and it usually does a better, more thorough job. We have enough infrastructure that by the time another human lays eyes on it for the first time each pull request has already been through 3+ rounds of automated review. Being the only specialist in my area on the team used to mean more coaching and more involvement in code review. Now my coworkers are sending me idiomatic solutions with modern approaches that I approve right away or with hardly any comments

Re: The LLM Critics Are Right. I Use LLMs Anyway

#125

Post by skydhash »

skinfaxi wrote: post #46
throw10920 wrote: post #33

Earlier quoted context omitted.

I think they didn't phrase it precisely, but my guess is the underlying idea is actually "high-level software architecture doesn't have a clear abstraction layer you can use to separate it from low-level coding (unlike logic gates, the CPU's ISA, the kernel API, etc), and so delegating the latter leads to delegating the former".

That makes sense but I'm still not so sure—we have things like software architecture patterns that can be discussed at a high level without knowing the intricacies. Like you can be aware of load balancing and even use it but be unaware of how load balancing might work algorithmically. Let's consider even the original example. > You cannot remove facts like 'paris is capital of france' from llm and have it just retain…

> we have things like software architecture patterns that can be discussed at a high level without knowing the intricacies. Like you can be aware of load balancing and even use it but be unaware of how load balancing might work algorithmically.

It can be fine as a “user”, but not really as a “designer”. Because discussion about possible solutions is a matter of tradeoffs and tweaking of parameters, not slinging words around. Abstractions are not appliances that are plug and play. They’re often full of parameters that dictate their usefulness and costs, and not understanding those parameters is just roleplaying.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#126

Post by MisterTea »

msdz wrote: post #2

> LLM’s amplify what you already have: opinions, structure, frameworks. So far, so agreeable, but… > If you have thoughts, they come out sharper and faster. I can’t help but wonder whether constant use of “agent” harnesses will lead to an atrophy of the software engineering (or really any field) muscles. Actual muscles need exercise to stay in shape (let alone grow), so does the brain. Can we really be sure that thou…

> but risking being left in the dust productivity-wise?

What's the risk here? Left behind by who or what?

> Or taking an “agent first” approach and trying to learn and improve more only on the side, as more of an afterthought?

This reads like anxiety resulting from FOMO.

Here's my take: I don't care about LLMs or AI in the sense that I don't feel any need or want to use them. I've only ever tinkered with the free ChatGPT. Never opened an account with any LLM vendor and never even considered it. I program by hand for the joy of it and sometimes for work. Still by hand as I have been doing. MY work gives me that luxury. For now.

Am I obsolete? Am I no longer of any value to society? Of course not. That thinking is just implanted by a group of money hungry individuals who don't give a fuck about me, you or society as a whole. So why would or should I care about LLMs?

Re: The LLM Critics Are Right. I Use LLMs Anyway

#127

Post by ambicapter »

rmarshallATD wrote: post #107
Levitz wrote: post #24

Earlier quoted context omitted.

>Conversely, I also am a user of LLMs (true shocker these days, I know), and am noticing a speedup in areas I was already familiar with, and a quicker introduction to new ones. The obvious benefit cannot be denied, and doing so regardless makes you look uninformed. My largest concern comes from something tangential to this: I'm not sure we're all that good at deciding what should be learned and sticking to it. Silly…

> But then, you just don't learn regex. So where does the intuition for what regex can do come from? The training data is there for regex and it's unlikely that regex will experience massive changes but your concern makes sense. I had to learn before LLMs handled that part, the "when to use this" intuition. My guess would be that the logical conclusion is dependence on LLMs to make that determination increasing over…

We're talking about validating the regex, which you have to do with your own skills, while LLMs are still wrong, some of the time.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#128

Post by parineum »

jmartrican wrote: post #59
xnorswap wrote: post #31

Earlier quoted context omitted.

I've been using regex decades, but it never really stuck to do anything too complex, it was the perfect intersection of difficult and infrequent. ( And also variable - PCRE vs others customisations / non-regular parts, etc ). I am very glad that I can now just ask claude for a regex to achieve my intent. Does it mean I'll never master regex? Yes it does, but decades has shown that was unlikely to ever happen anyway.

Regex came up so infrequent that I found myself referring to documentation whenever I needed to use it. But I always wondered, what are the jobs or roles that use it so often that they have mastered it.

anything with large amounts of user text input.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#129

Post by snarfy »

msdz wrote: post #2

> LLM’s amplify what you already have: opinions, structure, frameworks. So far, so agreeable, but… > If you have thoughts, they come out sharper and faster. I can’t help but wonder whether constant use of “agent” harnesses will lead to an atrophy of the software engineering (or really any field) muscles. Actual muscles need exercise to stay in shape (let alone grow), so does the brain. Can we really be sure that thou…

I have some hobby projects I write for fun without an LLM, just to learn. And also have hobby projects that use LLM extensively, also just to learn.

Get some hobby projects.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#130

Post by nicce »

swiftcoder wrote: post #117

> I am a little bit scared to say this too: last month I spent almost 10k USD on tokens. It sounds so insane. What are people actually doing with all these tokens? I use LLMs pretty heavily for development, and I'm rarely spending all the tokens that come with a $10/month OpenCode Go subscription...

If you are reviewing the changes and keep giving input, it is not that token consuming. But many people go with full yolo with agent mode.
Post reply on HN