[deleted]
LLMs reward expertise
161–170 of 607 posts
Re: LLMs reward expertise
#162Earlier quoted context omitted.
> Honestly, we don't really need to learn any more. Perhaps for your task of closing Jira tickets that’s true - but the you have a decision to make: wait until you’re eliminated or try to anticipate where future competitiveness comes from.
Yeah, that's the thing; future competitiveness isn't going to come from learning, at least not from learning about software. The LLMs have that covered, and software is becoming low status work anyways. It's largely going to come from interpersonal skills and being likeable enough to be the face of the projects and companies. Technical skill is rapidly dropping in value.
This claim can only be true if LLMs have completely mastered all intellectual aspects of creating and shipping software.
That's so far from being true it's like saying that humans are currently an interplanetary, post-scarcity species.
Re: LLMs reward expertise
#163The amplifying mirror analogy works best here. LLMs are ultimately a reflection of your own interactions with its weights, the tone you use, the structure with which you construct your prompt, aspects of an issue you tend to focus on, your breadth of vocabulary and world knowledge and whatnot. People who (carefully) use it as an extension of their own mind and senses will very likely thrive, and those who use it as a…
When I get out of my domain, I always ask it to describe things to me "like I'm a senior CS student who doesn't know any industry terms". I find it really easy to skim past the definitions I know and very useful to get the explicit clarification on unfamiliar terminology. Telling it to talk down to you a little also seems to calm down it's tendency to talk like it's trying really hard to convince you it's smart like…
I've been using a prompt that comes from the opposite direction for non-programming stuff: "Assume the user is an expert in all fields; while this is clearly logically untrue on a literal level, the user prefers to see a field's technical terminology and then ask the AI about terms the user does not understand rather than get an inaccurate statement about some issue."
Whether you have to reassure the LLM that this is obviously untrue, I don't know, but they do have a knowledge baseline to know it's not true and I have a sneaking suspicion it would be less effective without that.
This has ended up in some of the most interesting incidental knowledge exploration I've ever done. A recent example is that I was asking about some stretches and it started talking about how useful they are for the sarcomeres, which I had not heard of. Now I have.
I'm not saying this is better, just that it is different. I think there's a time and a place for both approaches.
Re: LLMs reward expertise
#164Agree so much with this! In domains I know well I get much better results then someone who doesn't know the domain because I know where to challenge the LLM. LLMs need to be pushed because otherwise their answers are typically average.
Re: LLMs reward expertise
#165Re: LLMs reward expertise
#166> Of course both are useful, but I’d rather have familiarity with the codebase than a deep general understanding of software systems. In my experience, getting that familiarity with a particular codebase in a way that isn't surface-level has always been a hands-on process. E.g. just because I know many general things about software, I need to know the particulars of the current codebase I'm in to know what is reasona…
I think this is only a partial answer but I've been surprised by how familiar dev leads become with the app even if they are not in the code.
They tend to stick around and they engage in the problem solving on a higher level and develop a detailed picture of how the app does and should behave. So at least that part of the expertise may come from working with an LLM to solve problems.
Re: LLMs reward expertise
#167The amplifying mirror analogy works best here. LLMs are ultimately a reflection of your own interactions with its weights, the tone you use, the structure with which you construct your prompt, aspects of an issue you tend to focus on, your breadth of vocabulary and world knowledge and whatnot. People who (carefully) use it as an extension of their own mind and senses will very likely thrive, and those who use it as a…
The amplifying mirror is not just a characteristic, but the fundamental driving force of LLMs. With every token it consumes, its primary goal is to understand who you are and what you intend. If you type Hola thats sufficient to tell it so much more than if you type hi.
Re: LLMs reward expertise
#168Earlier quoted context omitted.
Yeah, that's the thing; future competitiveness isn't going to come from learning, at least not from learning about software. The LLMs have that covered, and software is becoming low status work anyways. It's largely going to come from interpersonal skills and being likeable enough to be the face of the projects and companies. Technical skill is rapidly dropping in value.
This claim can only be true if LLMs have completely mastered all intellectual aspects of creating and shipping software. That's so far from being true it's like saying that humans are currently an interplanetary, post-scarcity species.
All? obviously not, but enough that I can ship more than my colleagues while not really paying attention to the output or fully understanding it. Technical skills aren't unnecessary, they're just becoming something that nobody will pay for.
LLMs debug extremely well, generate code well enough, and do high level design acceptably, but fail in weird ways, so my actual work these days (outside of delegating to the LLM) is mostly manual testing. I need to give them nudges sometimes, but a year ago they needed constant nudges. In another year, I expect that it'll be good enough that pasting in a bug report with the right jargon will be all the nudging I need to get something that customers will accept if it gets shipped. They're already close.
It's obviously going to remain pretty manual without some breakthroughs, but it's largely not work that needs a ton of intelligence or skill. The world hasn't caught up with software development being a few steps up from burger flipping, so I'm still getting paid well to do it, and I'll keep collecting that paycheck as long as it lasts. But learning about code is rapidly becoming relatively valueless.
I wish it was different, I actually enjoyed getting paid to learn, but I play with the cards I'm dealt.
Re: LLMs reward expertise
#169I totally see this. I just did 3 hours of bot sitting to put together some thrash loops that thrash our provisioning working flow for a BLE gadget we make. It was pretty straightforward and productive. But then, I have a lot of experience with BLE, and a quite a bit of experience with python and shell scripting. So I was able to guide the process through stages, do some intermediate testing, make some adjustments, an…
>Where my angst comes, is worrying that no one will ever get that experience anymore.
I am a fairly inexperienced python developer using LLMs to build software and find that I still learn a lot just from prompting and tinkering. Maybe that's less true once you reach a certain level of competence, but at my intermediate level I am still learning a lot even leaning heavily on LLMs.