Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs reward expertise

seangoedecke.com

Post reply on HN
Print view View original

361–370 of 607 posts

Re: LLMs reward expertise

#361

Post by hintymad »

> Because my friend didn’t have the vocabulary to ask the AI to write code

Is it possible that the effectiveness of an LLM user with respect to the expertise of the user is like a sigmoid function or at least a step function in that shape? That is, one has to know something like the basic concepts and the vocabulary to bootstrap a programming project, but one does not have to know too much to do lots of meaningful work, and then again one needs to be en expert to build something extraordinary. Since most of the work is somewhere middle, most of us mere mortals are still concerned or stressed out for the possibility that LLMs will squeeze out too many job opportunities.

Re: LLMs reward expertise

#362

Post by friendzis »

harvey9 wrote: post #323
raincole wrote: post #313

Earlier quoted context omitted.

Do your friend at least know what Claude Code (or any harness) is? Of course she wouldn't be able to make a website if she doesn't even know the right tools to use. But I don't think it prove anything. Knowing and installing Claude Code might not be a common sense, but nor is it "expertise" or "skill." I've seen in first hand that people struggle installing Steam . Yes, "people" in the plural. But just because some p…

Knowing what Claude Code is and why you might want it actually is domain knowledge which the op's friend does not have. Your idea of the average person might be biased if you work and socialise with people who have this kind of expertise.

You are assuming that the hypothetical friend has been isolated from any kind of technology from the last few decades. The friend can, presumably, use search engines and the very same chip chipities to get the very rudimentary domain knowledge going.

Re: LLMs reward expertise

#363

Post by MrScruff »

sothatsit wrote: post #259
dmazzoni wrote: post #222

Earlier quoted context omitted.

As opposed to what? If you ask a non-programmer to install Claude Code, just installing it will be a challenge, then opening the shell and interacting with it. Things as simple as copying and pasting can present roadblocks if you've never used a shell before, and things intuitive to programmers like using up-arrow to go back to a previous prompt would never occur to someone in the field. Claude Code seems so simple a…

Claude Cowork is the application aimed at non-developers that gives them a lot of the same functionality. My girlfriend uses it and has gotten quite far in producing her own software.

I got my girlfriend to install Claude Code and she was happily able to create software with it completely independently of me.

Re: LLMs reward expertise

#364

Post by wei_b0 »

I've experienced this firsthand and 100% agree.

The more cracked you are in a domain, the more you can squeeze out of an LLM. If you already know what "good" looks like, you can steer it, call out its BS, and iterate way faster than someone who's using it to learn the domain itself.

Re: LLMs reward expertise

#365

Post by lazyasciiart »

krisoft wrote: post #191

I did a test a few months ago. A friend of mine wanted to develop what i understood to be a simple single page web app. But since she didn’t have any software engineering experience she asked me to help. Around that time everyone was talking about how literally anyone can develop software with LLMs i asked her if she could give it a try first, and if I could watch the attempt. I was fully expecting that writing the c…

did she try saying "I want to make a webpage"? Even ChatGPT will just build, deploy and host a webpage for you with that request. I don't really understand what system she must have been using.

Re: LLMs reward expertise

#366

Post by sensanaty »

jvanderbot wrote: post #210
matthew-wegner wrote: post #192

Earlier quoted context omitted.

Are you describing a "chat window" experience here? This is apples to oranges.

Open any chat window and ask for a simple SPA with startup instructions. It's fine, it's fine, it works.

And you think a random John/Jane Doe knows what a SPA is?

Re: LLMs reward expertise

#367

Post by lazyasciiart »

baxtr wrote: post #348
friendzis wrote: post #346

Earlier quoted context omitted.

Having an experienced friend looking over your shoulder and taking notes of every step might help a bit, I reckon

I thought the idea was to test if she gets along without any help

I'm pretty sure the idea of the question "what harness did she use" was to ask the expert recounting this anecdote to include more detail, not suggest that he relay the question to his example friend.

Re: LLMs reward expertise

#368

Post by wongarsu »

roncesvalles wrote: post #276
coder-3 wrote: post #273

Earlier quoted context omitted.

We've seen plenty of examples of apps successfully vibe coded by non technical people, including apps making real revenue. Your friend could start with telling the LLM that they are a non technical person who wants to make an app and it will explain all the successive steps.

>We've seen plenty of examples of apps successfully vibe coded by non technical people, including apps making real revenue. Have we? Or is this just something that people say now, without citation?

Maybe people don't cite specific apps because they like their jobs, and outing apps as vibe-coded is still seen as negative

I personally know of two completely vibe-coded large apps in my professional environment. One by a non-technical manager, made to solve his needs, then sold to customers. Initial development went along great, but by now velocity has greatly slowed down. Also took a lot of engineering hours (of actual software developers) to get permission management from "chaotic and ineffective" to passable. It's still worse than what you would have gotten by just using a couple sentences of the right technical language at the start. Deployment is also a bit of a nightmare. All in all, anything beyond the first rollout phase was delayed by months. Honestly it should have stayed as a prototype that then gets rebuilt from the ground up. But still, it is a real app, making real revenue

The other example was vibe-coded by a software engineer in his free time. Works pretty well, doesn't have too many bugs. Makes some revenue, but a lot less. Solving manager problems just sells better.

Re: LLMs reward expertise

#369

Post by lazyasciiart »

b112 wrote: post #357
didntknowyou wrote: post #354

Earlier quoted context omitted.

same thing as people just entering a question prompt and copying and pasting the response as gospel. e.g. politicians using it to write speeches, or lawyers for testimonials. the output is programmed to look correct so unless you have some sort of background you won't actually know what errors to look for.

And look correct is accurate. Not only does it look correct, it looks correct with an extremely Subject Matter Expert degree of authority. Often I'll work with an LLM, and it simply just misses so many things. I've worked in all sorts of different domains, software, chemistry, material design, everything from power generation through to physics, and in each and every case I see it missing incredibly important things.…

It's just like working with engineers.

Re: LLMs reward expertise

#370

Post by inigyou »

HoneySpoons wrote: post #217
abixb wrote: post #73

The amplifying mirror analogy works best here. LLMs are ultimately a reflection of your own interactions with its weights, the tone you use, the structure with which you construct your prompt, aspects of an issue you tend to focus on, your breadth of vocabulary and world knowledge and whatnot. People who (carefully) use it as an extension of their own mind and senses will very likely thrive, and those who use it as a…

ELI5 is the way to go. I have Claude break down high level physics "as if I'm a farmer standing in a field" - works beautifully.

Does it? ChatGPT had no problem telling me that dark energy is the Higgs VEV.
Post reply on HN