Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs reward expertise

seangoedecke.com

Post reply on HN
Print view View original

351–360 of 607 posts

Re: LLMs reward expertise

#351

Post by roncesvalles »

downloadram wrote: post #338
krisoft wrote: post #191

I did a test a few months ago. A friend of mine wanted to develop what i understood to be a simple single page web app. But since she didn’t have any software engineering experience she asked me to help. Around that time everyone was talking about how literally anyone can develop software with LLMs i asked her if she could give it a try first, and if I could watch the attempt. I was fully expecting that writing the c…

>she didn’t find a way to tip the AI into “just do it, write it now” mode worse, the longer an LLM conversation goes on, but especially with constricted/free models (yes the simple chat interface they are likely using) the harder it is to get an LLM into this mode even *IF* you know the right words to say at that point the best way forward is to terminate the exchange entirely, and to start off with the right initial…

Also, the correct way to LLM is to constantly trial-and-error in new/branched contexts.

Remember the LLM is not a human employee. You don't have to say "yes and" to whatever crap they produced so as to not hurt their feelings or infringe upon their creative autonomy, nor do you have to defend the correctness of your original instructions so that they don't think less of you for asking them to chase the wrong goose.

I probably generate 20-50 lines of code for every 1 line that I keep.

This is also why I think harnesses and things like Claude Code and OpenCode are false efficiency. The only way I can maintain my pace of branched trial-and-error is by using claude.ai/chat and manually extricating code fragments to and from my codebase. The human is still the best harness for production-level code.

Re: LLMs reward expertise

#352

Post by willtemperley »

HoneySpoons wrote: post #217
abixb wrote: post #73

The amplifying mirror analogy works best here. LLMs are ultimately a reflection of your own interactions with its weights, the tone you use, the structure with which you construct your prompt, aspects of an issue you tend to focus on, your breadth of vocabulary and world knowledge and whatnot. People who (carefully) use it as an extension of their own mind and senses will very likely thrive, and those who use it as a…

ELI5 is the way to go. I have Claude break down high level physics "as if I'm a farmer standing in a field" - works beautifully.

I tried that but Claude thought I was already outstanding in my field.

Re: LLMs reward expertise

#353

Post by sans_souse »

sam_lowry_ wrote: post #305
friendzis wrote: post #299

Earlier quoted context omitted.

> They didn’t even get to that point. Because my friend didn’t have the vocabulary to ask the AI to write code. What harness did you use? In e.g. claude, there are two modes: 1. Spit out code 2. Draft a plan, ask questions, GOTO 1 You literally have to go out of your way to get it NOT to write code. I keep mine on a tight-ish leash because it modify code way too happily even when there's no intention or instruction t…

Harness? I am an avid HN reader but even I don't yet fully understand how this word is used in the AI context. Which is exactly the point of the OP. It all boils down to naming things and cache invalidation, /s

My question is do we harness a bootstrap or bootstrap a harness?

Re: LLMs reward expertise

#354

Post by didntknowyou »

krisoft wrote: post #191

I did a test a few months ago. A friend of mine wanted to develop what i understood to be a simple single page web app. But since she didn’t have any software engineering experience she asked me to help. Around that time everyone was talking about how literally anyone can develop software with LLMs i asked her if she could give it a try first, and if I could watch the attempt. I was fully expecting that writing the c…

same thing as people just entering a question prompt and copying and pasting the response as gospel. e.g. politicians using it to write speeches, or lawyers for testimonials.

the output is programmed to look correct so unless you have some sort of background you won't actually know what errors to look for.

Re: LLMs reward expertise

#355

Post by Bootvis »

annzabelle wrote: post #283
lopsotronic wrote: post #253

Earlier quoted context omitted.

As an unapologetic generalist[1] this has also been my experience. Many tools that would have been "eh maybe if I get bored over Thanksgiving holiday" have become "hold on, gimme fifteen minutes". Tiny, isolated, but awesomely useful CLI scriptlets, for me, seem to be the sweet spot. Little shining rays spreading out from the veins of my own familiarity. The downside, the Achilles Heel of LLMs, so far as I can tell,…

I've found Claude Code absolutely amazing for the sorts of 100-500 line data cleaning/analysis/visualization tasks that used to take me a couple hours to knock out. They're often self contained (boss wants a graphic for a slide or some numbers), and I tell it which packages I would prefer it to use. On the other hand, I've been using it to make small changes to a ~4000 line codebase, and it takes a lot of wrangling t…

I went to VB.NET first using a previous generation of LLM's (that was quite manual back then) and then from VB.NET to C# or just keeping the VB.NET around worked very well. The code was not highly complex but more than just CRUD. The porting from VB6 to VB.NET included building tests which helped.

Re: LLMs reward expertise

#357

Post by b112 »

didntknowyou wrote: post #354
krisoft wrote: post #191

I did a test a few months ago. A friend of mine wanted to develop what i understood to be a simple single page web app. But since she didn’t have any software engineering experience she asked me to help. Around that time everyone was talking about how literally anyone can develop software with LLMs i asked her if she could give it a try first, and if I could watch the attempt. I was fully expecting that writing the c…

same thing as people just entering a question prompt and copying and pasting the response as gospel. e.g. politicians using it to write speeches, or lawyers for testimonials. the output is programmed to look correct so unless you have some sort of background you won't actually know what errors to look for.

And look correct is accurate.

Not only does it look correct, it looks correct with an extremely Subject Matter Expert degree of authority. Often I'll work with an LLM, and it simply just misses so many things. I've worked in all sorts of different domains, software, chemistry, material design, everything from power generation through to physics, and in each and every case I see it missing incredibly important things. Any true subject matter expert would immediately bring up and prompt concerns, but not the LLM.

This makes sense, of course, because these are language models. They were trained on language. Their first and foremost capability is language.

An LLM's true expertise, true subject matter expertness is language.

And so anyone working with LLMs who isn't already highly skilled in the field they're asking questions about, will invariably be led astray and miss extremely important parts of a puzzle that need to be solved.

Re: LLMs reward expertise

#358

Post by nlpnerd »

This why those RL env startups are able to charge frontier labs so much for their work. LLMs still generalize poorly outside of self-verifiable tasks like coding and math.

Labs have to compensate with post-training in RL env that embeds these expertise well, which is non-trivial both in terms of domain knowledge and technical expertise.

Re: LLMs reward expertise

#359

Post by Maxion »

raincole wrote: post #345
harvey9 wrote: post #323

Earlier quoted context omitted.

Knowing what Claude Code is and why you might want it actually is domain knowledge which the op's friend does not have. Your idea of the average person might be biased if you work and socialise with people who have this kind of expertise.

> Your idea of the average person might be biased if you work and socialise with people who have this kind of expertise. I just said I've seen multiple people struggle installing Steam... The point is that it's something objectively easy. Once they find (in this case, given by me) the correct instructions and follow through, they can easily do it by themselves again. It's quite different from what are traditionally c…

> The point is that it's something objectively easy. Once they find (in this case, given by me) the correct instructions and follow through, they can easily do it by themselves again.

This is true of most things in life. It is very easy to make compost, it is very easy to grow carrots, it is very easy to graft an apple tree onto rootstock, it's very easy to hang a door and it's also very easy to replace the break pads on your car.

Once you've done it, that is. And once you know what tools you need. And how to use those tools. And that you actually have those tools.

Codex, Zed, the like are all tools that you need to know exist and you need to have and you need to know how to use. It's the same thing as a wrench, a break bleeding kit, or some graft tape.

Re: LLMs reward expertise

#360

Post by Maxion »

vector_spaces wrote: post #330
harvey9 wrote: post #323

Earlier quoted context omitted.

Knowing what Claude Code is and why you might want it actually is domain knowledge which the op's friend does not have. Your idea of the average person might be biased if you work and socialise with people who have this kind of expertise.

I'm genuinely a bit floored reading the comments here, but I guess my idea of the average HN commenter's ability to talk to non-technical folks about technical topics is biased because I work and socialize with lots of people who don't have technical expertise (thankfully along with other technical folks who also have lots of experience talking to the former group). Many of them don't even know what Claude is, let al…

The average person does not even know what ChatGPT is and has not interacted with an LLM ever.
Post reply on HN