Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs reward expertise

seangoedecke.com

Post reply on HN
Print view View original

201–210 of 607 posts

Re: LLMs reward expertise

#201

Post by davely »

Synthetic7346 wrote: post #196
matthew-wegner wrote: post #192

Earlier quoted context omitted.

Are you describing a "chat window" experience here? This is apples to oranges.

Yeah I wonder if they had given their friend Claude code or Codex, would it have been more likely to create what she wanted?

Perhaps! But I do think the vocabulary issue is real and I think LLMs are still sycophantic enough that they won’t really challenge someone or offer alternative ideas on how to implement something unless they explicitly ask.

Interestingly at my work, Claude Code was available before Claude Desktop, so a number of non-technical PMs tried to use it in order to build… anything, with very mixed success.

The “hey guys, check out the website I built with Claude: http://localhost:3000/” joke is real!

In my experience, the whole “the terminal is a scary place” aspect is very real and some non-technical people can feel intimidated by.

I think Claude Code in the desktop app helps alleviate that a bit (perhaps Codex, too, but man what a mess the ‘ol ChatGPT app has become).

But I’m sure there are entire repos of web dev skills that someone could use to put together things with a bit of effort.

Re: LLMs reward expertise

#202

Post by beering »

lagrange77 wrote: post #146
PaulStatezny wrote: post #132

Earlier quoted context omitted.

LLMs skew toward over-focusing on things that you mention. The reason "the agent suddenly started suggesting all kinds of things to make its code more robust" is because you said you "want to build reliable software". It's not a signal of good judgment or understanding. It's just how LLM attention works.

I thought exactly the same at first. But then i wondered if that still holds true with today's advanced thinking, RLHF involved, frontier models. I guess to a certain extend it did indeed behave better, as a reaction to his self description into account. EDIT: I mean, those systems accumulated so much complexity around the attention based next token predictor.

Training the LLM to do things that the user didn’t explicitly ask for is a good way to get complaints from the users. Doesn’t matter if those things are best practices.

Re: LLMs reward expertise

#203

Post by uzername »

At work we call this implicit steering. To use webdev metaphor, if a non-technical person describes making a web page with a big block at the top and some things to click on and then my pictures below that, that will eventually get somewhere. Meanwhile, if you know industry jargon, you might describe a hero, with call to action buttons, and then below a 3x3 grid of images of my portfolio photos—that's likely going to generate something entirely different and likely richer. It can assume things about you (it doesn't think), it can ask you specific questions a web personal might know, it can infer domain context that is otherwise omitted with a basic conversation. Everyone wants to capitalize on corporate vibe coding but the tech literacy is hardly there, let alone more advanced topics.

Re: LLMs reward expertise

#205

Post by engineer_22 »

StevePrefontain wrote: post #172
dbalatero wrote: post #126

> Of course both are useful, but I’d rather have familiarity with the codebase than a deep general understanding of software systems. In my experience, getting that familiarity with a particular codebase in a way that isn't surface-level has always been a hands-on process. E.g. just because I know many general things about software, I need to know the particulars of the current codebase I'm in to know what is reasona…

We are having trouble onboarding engineers with AI now. Some still struggle after their first year with very basic concepts/patterns we use and make the same mistakes again and again in their PRs because they just ask Claude to fix it and never internalize it. I think using LLMs feels good at first because you can get tickets out the door faster but you never develop enough knowledge to make a big impact or become an…

These would have been good hires before AI but now they struggle?

Re: LLMs reward expertise

#206

Post by AnotherGoodName »

krisoft wrote: post #191

I did a test a few months ago. A friend of mine wanted to develop what i understood to be a simple single page web app. But since she didn’t have any software engineering experience she asked me to help. Around that time everyone was talking about how literally anyone can develop software with LLMs i asked her if she could give it a try first, and if I could watch the attempt. I was fully expecting that writing the c…

OTOH my wife's friends got drunk and made "tinder for horse purchases". They prompted to read typical horse advertisements (we're all horse people) and create an app with mock tinder like entries to swipe right and left to buy horses.

A web app was produced with lots of mock "Hi i'm Dominique and i love running through fields and having a bucking good time" type entries complete with silly horse photos. A huge amount of drunken fun even if it boiled a towns water supply and blew through half a subscription to create.

I was looking at the results as a dev with 30 years experience and thinking fuck me. The little apps i made here and there before AI are being outdone by a bunch of drunk people on a whim!

Re: LLMs reward expertise

#207

Post by cortesoft »

xvfLJfx9 wrote: post #137
abixb wrote: post #73

The amplifying mirror analogy works best here. LLMs are ultimately a reflection of your own interactions with its weights, the tone you use, the structure with which you construct your prompt, aspects of an issue you tend to focus on, your breadth of vocabulary and world knowledge and whatnot. People who (carefully) use it as an extension of their own mind and senses will very likely thrive, and those who use it as a…

That sounds useful. Can you share that skill?

I have been using this project recently to make myself tutorials on things I want to learn: https://github.com/devenjarvis/lathe

Re: LLMs reward expertise

#208

Post by HarHarVeryFunny »

I think this is just the nature of LLMs as predictive generators.

The model is predicting the type/level of conversation based on what the other party is saying.

The most typical types of conversation are of two peers, so by default the LLM is likely to respond to you at your own level, unless you ask it to behave differently.

As always, prediction goes deep. The best response to Terrance Tao is Tao-level math.

It reminds me of reading how LLMs continue chess games if given a partial game - they have learnt to assess player strength based on the moves they make, and will predict game continuations based on the perceived strength of each player, predicting (generating) poor quality moves for a weaker player. This isn't an AI playing chess to win - it's an expert predictor predicting what comes next.

Re: LLMs reward expertise

#209

Post by 27183 »

dafelst wrote: post #189
27183 wrote: post #179

Earlier quoted context omitted.

maybe outing myself as a dinosaur, but "back in my day" the calculator came with a book that detailed exactly how to use it. Both the high level basic language and the low level system language. Not knowing how to use it is simply a failure to Read The Fucking Manual.

You can read the manual all you want, but if you don't know basic algebra, trig, calculus, etc, you are not going to have any idea how to apply or use much of anything that the manual describes with regards to actually doing math with a graphing calculator. There is a base level of knowledge required.

It was a general purpose computer, just a small one. Anything you could do on a "real" computer could be done on a calculator, albeit with tighter constraints. It might help to have some higher math objective to accomplish, because that would better utilize the preloaded system software. But in terms of the hardware? Probably not super relevant.

I made a lot of use of the TI-89 era CAS in college. But IMO the TI-83 era manuals taught me more about both math and computers than the subsequent generations could have.

Re: LLMs reward expertise

#210

Post by jvanderbot »

matthew-wegner wrote: post #192
krisoft wrote: post #191

I did a test a few months ago. A friend of mine wanted to develop what i understood to be a simple single page web app. But since she didn’t have any software engineering experience she asked me to help. Around that time everyone was talking about how literally anyone can develop software with LLMs i asked her if she could give it a try first, and if I could watch the attempt. I was fully expecting that writing the c…

Are you describing a "chat window" experience here? This is apples to oranges.

Open any chat window and ask for a simple SPA with startup instructions. It's fine, it's fine, it works.
Post reply on HN