Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding is the new bottleneck

geoffreylitt.com

Post reply on HN
Print view View original

251–259 of 259 posts

Re: Understanding is the new bottleneck

#251

Post by drooby »

mrheosuper wrote: post #142
drooby wrote: post #61

Earlier quoted context omitted.

My team solved this by creating a PR draft skill that clamps the length of the description to 3-5 sentences max. Those 3-5 sentences must only say WHAT is changing and WHY. I find it to be far more useful than when humans wrote PR descriptions. Many engineers didn't write one, and those that did were poorly written... this problem is mostly solved for us.. it still has LLMism speak.. but it's useful enough for me to…

Your team can't spend effort to write 3-5 sentences by yourself ?

well... _of course_ the skill does more than write 3-5 sentences. We have a lot more that it automates into the description that makes it worth using.

Re: Understanding is the new bottleneck

#252

Post by drooby »

AlotOfReading wrote: post #77
drooby wrote: post #61

Earlier quoted context omitted.

My team solved this by creating a PR draft skill that clamps the length of the description to 3-5 sentences max. Those 3-5 sentences must only say WHAT is changing and WHY. I find it to be far more useful than when humans wrote PR descriptions. Many engineers didn't write one, and those that did were poorly written... this problem is mostly solved for us.. it still has LLMism speak.. but it's useful enough for me to…

The challenge of hard rules like this is that they're always overly restrictive. I've made multi-thousand file PRs that needed two lines of description including the title, and 5 line PRs that needed a 1hr presentation to fully explain them. My personal guideline is that writing for humans should be done by humans.

I mean.. human judgment still applies. But as an automated first pass the clamp works in 90% of cases. I scan for correctness and make small edits here and there.

Re: Understanding is the new bottleneck

#253

Post by iLemming »

Forgeties79 wrote: post #199
jfyi wrote: post #188

Earlier quoted context omitted.

I disagree, both of those are skills. Neither are skills that a large portion of users of those services pursue to any meaningful extent, I'll grant you that. They also certainly are not synonymous with the term "soft skills" as I know it. So I think I am on your side of the fence on that part any way.

It is impossible to duplicate results with an LLM. This strikes me as a serious barrier to calling it a proper skill. If you can’t even somewhat replicate the results you can’t really improve the input consistently. You can’t become “skilled” at it if you can’t even reproduce what you did. If you enter the same prompt 3 times the results are of pretty significantly different quality. ChatGPT literally has you A/B tes…

> It is impossible to duplicate results with an LLM

But that's the fundamental property of it - it is stochastic by nature. The skill is to learn how to sandwich deterministic logic between layers of randomness. Determinism doesn't live in the model. It lives in the harness you build around it. You can't make the model deterministic (it simply cannot be), so you make the system deterministic instead. Validation before, validation after, the randomness stays contained in the middle.

Talking to an LLM is not a skill, having a meaningful conversation leading to practical outcomes is.

Re: Understanding is the new bottleneck

#254

Post by mildeukaryote »

MinimalAction wrote: post #91

I am surprised by the title and the story. Understanding has always been the bottleneck; there is nothing new about it. The argument goes like we humans should understand so we can verify and participate. How bold! Maybe we should have been doing that all along...?

A good chunk of the non-tech-savvy management equates coding with typing. They're rediscovering Amdahl's law in the agentic LLM era.

Re: Understanding is the new bottleneck

#255

Post by 1659447091 »

Forgeties79 wrote: post #219
gilrain wrote: post #211

Earlier quoted context omitted.

> It is impossible to duplicate results with an LLM. Humans are called experts in many domains whereby they cannot achieve duplicate results. In fact, in what domain can an expert do that? Nobody crosses the same river twice.

If you ask me to write an email, all 3 variations will be more or less the same, except it will almost always improve with each iteration. 3 cracks at a prompt is 3 separate, unrelated attempts. None of them informs the other. In fact, you run the risk of making it worse if you include previous versions.

> 3 cracks at a prompt is 3 separate, unrelated attempts. None of them informs the other.

Thats where the "skill" part comes in. Like your 3 attempts at email that `almost always improve`. This is where you input the "previous versions", not directly back into the model. It's the "soft skill" of being flexible and adjusting based on how an entity responds to the input. Not learning how to adjust the input (by using previous attempts to inform the next) to more optimally direct the output given the state of the llm (chat? agent? model, effort lvl etc) it will seem chaotic.

The skills wont make an llm* deterministic, same as applying these soft skills to people. Give the same input to a person in `3 times separate, unrelated attempts` you are likely to get 3 different seemly chaotic outcomes. The "skill" is in being able to take what you received as output the first time, and make adjustments based on previous attempts while accounting for the state of the entity for the next attempt.

* In no way am I saying they are conscious beings or whatever nonsense by using people in the analogy. There are, however, parallels in how a set of soft skills (and this is why they are "soft" skills) can be used to get more optimal results from an entity that should never be expected to act as a pure function.

Re: Understanding is the new bottleneck

#257

Post by odshoifsdhfs »

alecbz wrote: post #246
odshoifsdhfs wrote: post #75

Earlier quoted context omitted.

I don't really care about the dev future as a few years ago I started to divert from it and most of my income is in other things. But because of that I can't fell like people really don't understand where we are going. I have a conspiracy theory that even VCs are on it. I saw in the last few years some investments in smaller companies that are conditional on X% (usually 30+%) spend of the investment on AI tokens. I a…

OOC what's your other income from?

investment rental properties, just normal investments with boring returns, carpentry, electrical installations, handy-man jobs.

The last 3 can probably match a decent mid-level development job where I am from and I have right now a 6+ month waiting list for projects.

Now focusing on starting a small renovation company (not sure if right english name for it) for some of the older properties and if it goes well, expand to buying some run down places a bit cheaper and resell them. (Had limited success with this before, but was subcontracting most of the work, now want to bring it in-house) (ps: not buy for 100k and sell for 500k, but something like buy for 100k, spend 40-60k and sell for 180k)

Re: Understanding is the new bottleneck

#258

Post by sscaryterry »

bmurphy1976 wrote: post #125
361994752 wrote: post #117

Earlier quoted context omitted.

I’ve worked on both sides, so I know they’re actually very different. As a manager, the first thing you do is get to know your people. Some of them will be very strong and trustworthy. You give them the hardest work, and you ask them the least. That’s how you scale your team’s scope without getting overburdened. And that’s why teams have key people. But when you work with LLMs, you still need to understand most of th…

This doesn't change the fact that those soft skills are necessary to use LLMs effectively. You need a mixture of hard and soft skills. Engineers who have avoided learning the soft skills are going to have a harder time adjusting.

> avoided learning the soft skills

LLMs don't have emotions?

Re: Understanding is the new bottleneck

#259

Post by deterministic »

madrox wrote: post #82

I think it funny how much average engineers are beginning to discover the challenges of engineering leadership and program management. This has always been the bottleneck. It's why managers and PMs want to be in standup. It's why slack exists and engineers are constantly being poked on it. It's why execs always talk about not getting too far away from the work. It's how seagull management happens. It's why program ma…

I’ve managed large software teams for 10+ years and also worked as an individual developer.

In my experience, most middle managers, probably 90%, have no clue how to manage software developers. It’s not that hard to manage teams IMHO, but you need to focus on the right things.

Post reply on HN