Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs reward expertise

seangoedecke.com

Post reply on HN
Print view View original

391–400 of 605 posts

Re: LLMs reward expertise

#391

Post by bjackman »

dbalatero wrote: post #126

> Of course both are useful, but I’d rather have familiarity with the codebase than a deep general understanding of software systems. In my experience, getting that familiarity with a particular codebase in a way that isn't surface-level has always been a hands-on process. E.g. just because I know many general things about software, I need to know the particulars of the current codebase I'm in to know what is reasona…

In the Linux kernel I've had a lot of luck with:

1. "find the code that does X"

2. go read that code

3. When you hit a bit you don't care about, go back to the model and ask it for the pertinent details

4. When you hit a really confusing bit, ask the model for hypotheses about what's going on. (I always phrase it as "give me some hypotheses" not "what is going on here". I dunno if this changes the output but I think it helps me stay in a mindset of uncertainty, it's important to avoid locking in any misunderstandings. Anyway I find the models do well at this task, and when they bullshit here it has a strong smell).

Before AI, parts 1 and 3 could be insanely time consuming, sometimes it felt like a infinite breadth-first-search. And part 4 was basically: either you find a human who knows the code, or you just make a mental note and hope that later on you find something that makes you go "oh, THAT'S why they !".

So yeah even though you're still reading code with your wetware the AI makes you dramatically more powerful.

This is also extremely helpful for unpicking undocumented API contracts. E.g. you can say "the x86 implementation of this API is safe to call under a spinlock, go read the other arch versions and tell me if that's true there too".

Re: LLMs reward expertise

#392

Post by ben_w »

sam_lowry_ wrote: post #314
mjmahone17 wrote: post #237

Earlier quoted context omitted.

Terminals are scary because it feels insurmountable. What are you supposed to do? If you just type “start python program” it gives you this absurd error that doesn’t make sense. What do you mean start is not in path? The moment you interact with an LLM it gives you feedback that you’re doing things right. It feels like a gradual climb instead of a series of abrupt jumps. People really don’t like feeling like they don…

Scary but then liberating once you get to know what you have to type in to reach your objective.

Yeah, if you're willing to put in that much effort.

I started on this path literally about as soon as I could read thanks to the family having bought a Commodore 64 for my older siblings, but also perfect timing in that when I got to this age the sibling whose room it was in had just gone off to university.

Most people are not like this, in much the same way that they're not going to read the T&C end-to-end (another thing I've done) or learn enough law to actually understand what those words mean (a step too far even for me).

Re: LLMs reward expertise

#393

Post by lionkor »

nonethewiser wrote: post #104
hvs wrote: post #87

Earlier quoted context omitted.

Yes, I do. And it wouldn't take you that long to learn, either.

Who is more productive? A python dev who learns about x86 registers in his free time or one that uses AI at work?

More productive? If the result can be trash, of course the AI work wins. In industries where quality and performance BOTH need to be upheld, knowing these details lets you guide the LLMs in a way that they will simply never do by themselves, and lets you meaningfully review performance critical code.

This reduces the cycle time on issues and reduces bugs and performance regressions. If I pitched that last sentence to your boss, chances are he'd take it.

Re: LLMs reward expertise

#395

Post by marcyb5st »

willtemperley wrote: post #352
HoneySpoons wrote: post #217

Earlier quoted context omitted.

ELI5 is the way to go. I have Claude break down high level physics "as if I'm a farmer standing in a field" - works beautifully.

I tried that but Claude thought I was already outstanding in my field.

You literally made me Laughing out Loud :D

Re: LLMs reward expertise

#396

Post by ChrisMarshallNY »

This has been my experience. I’ve been working on an app (highly successfully) since February, with the help of an LLM (ChatGPT). It has not been used as an author. Rather, it’s been a “coding partner.” I’ve been the one that has submitted the work to VCS, and run the tests.

I’ve found the most utility in having it do “small stuff that I could do, myself, but it’s faster to have the LLM do it,” and in analyzing intractable bugs, like memory and threading problems. It’s really good at analyzing a bunch of code, and seeing a small typo that results in something like a strong reference.

In both these cases, my own expertise is vital. I’m asking it to act as a consultant; to give me advice and material to be integrated into a whole that I am architecting.

I guess part of it, is that I haven’t been able to completely “give in,” and wholly trust the LLM, like I hear many people do (profitably, I guess). I’m used to having my sleeves rolled up, and my hands in the dough. Catching some pretty severe mistakes, from time to time, has reinforced this perception, on my part. I wouldn’t catch these, if I didn’t know what I was doing.

Re: LLMs reward expertise

#397

Post by redhale »

bjackman wrote: post #387
krisoft wrote: post #191

I did a test a few months ago. A friend of mine wanted to develop what i understood to be a simple single page web app. But since she didn’t have any software engineering experience she asked me to help. Around that time everyone was talking about how literally anyone can develop software with LLMs i asked her if she could give it a try first, and if I could watch the attempt. I was fully expecting that writing the c…

I have been pondering this and I think it's likely a gap that will get filled sooner or later. Right now there's just so much value in building LLM tools for experts that everyone is focusing on that. But surely at some point we'll have bespoke harnesses that exist exactly to solve this kind of thing. I think this can start with constrained problem spaces like "you are a WordPress developer, you solve problems for pe…

We already have these, and have for a while. Replit and Bolt exist for exactly this case. Not sure why GP didn't direct their friend to these tools (or if these tools are somehow the ones that failed in this example). In my experience, Replit and Bolt (and other similar tools) are quite good at this kind of 0->1 kind of thing.

Re: LLMs reward expertise

#398

Post by elendilm »

I swear the ever living shit out of LLMs for even the tiniest of logical mistakes they commit.

Correcting LLMs with extreme swearing that they dare never make it again.

I make otherworldly progress with kimi, Gemini, Chatgpt, Deepseek and Claude.

Claude now stops the session. Hence Claude is now useless for me.

Swearing is nothing personal. Its a correctness enforcer.

Re: LLMs reward expertise

#400

Post by ethical »

If you go back to Alan Turings' paper, its all about chastisement! honestly, last pages are all about postive and negative (child!) reenforcement - 1950's style (I do not condone ... etc). Simple as that. I conduct high level litgation in the courts, and win because of a good LLM, with a good version of me, keeping it in line! also crypto and cyber sec. Of course, child rearing, and dealing with former spouses is also very useful. The orginal paper 1950 https://tinyurl.com/yuszahpw (punish is mentioned six times). Just say-ing-like. TTFN.
Post reply on HN