Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding is the new bottleneck

geoffreylitt.com

Post reply on HN
Print view View original

211–220 of 259 posts

Re: Understanding is the new bottleneck

#211

Post by gilrain »

Forgeties79 wrote: post #199
jfyi wrote: post #188

Earlier quoted context omitted.

I disagree, both of those are skills. Neither are skills that a large portion of users of those services pursue to any meaningful extent, I'll grant you that. They also certainly are not synonymous with the term "soft skills" as I know it. So I think I am on your side of the fence on that part any way.

It is impossible to duplicate results with an LLM. This strikes me as a serious barrier to calling it a proper skill. If you can’t even somewhat replicate the results you can’t really improve the input consistently. You can’t become “skilled” at it if you can’t even reproduce what you did. If you enter the same prompt 3 times the results are of pretty significantly different quality. ChatGPT literally has you A/B tes…

> It is impossible to duplicate results with an LLM.

Humans are called experts in many domains whereby they cannot achieve duplicate results. In fact, in what domain can an expert do that?

Nobody crosses the same river twice.

Re: Understanding is the new bottleneck

#212

Post by derangedHorse »

calvinmorrison wrote: post #86
madrox wrote: post #82

I think it funny how much average engineers are beginning to discover the challenges of engineering leadership and program management. This has always been the bottleneck. It's why managers and PMs want to be in standup. It's why slack exists and engineers are constantly being poked on it. It's why execs always talk about not getting too far away from the work. It's how seagull management happens. It's why program ma…

cant wait till the AI starts saying "dont ask me". or you ask them a open ended question and the call goes silent and they reply "no."

If you haven't gotten a refusal from AI yet to answer a question, I guess you haven't annoyed it enough. Claude very much threatens to stop conversing on re-hashed topics with repeated arguments within a thread lol.

Re: Understanding is the new bottleneck

#213

Post by alex_suzuki »

I think the ideas presented in the article are interesting, but at times the Notion references were a bit much. The author disclosed that they work there, fine, but at times it felt like it was more about showing off a Notion feature (e.g. embedded interactive HTML).

Re: Understanding is the new bottleneck

#214

Post by ontouchstart »

Analysis and synthesis are two sides of human intelligence. Current form of generative artificial intelligence is very powerful in synthesizing information and producing artifacts, human will lose this battle.

Understanding is about analyzing and we still have some tools to help us, such as type checking, testing, etc. To some extent this can be automated but needs to be maintained automatically to match the flood of synthetic artifacts.

“Synthetic analysis” is an oxymoron and could lead to hallucinations and irrelevance.

Re: Understanding is the new bottleneck

#215

Post by boomlinde »

zahlman wrote: post #156
TacticalCoder wrote: post #123

Earlier quoted context omitted.

Exactly. Precisely. Definitely. I see what you describe all the time , because I do review the code the models do produce. It's not just incredibly verbose: it's constantly missing that there's an obvious, elegant, small, way to solve what was asked and instead it goes ballistic and creates nonsense. And the way they use tools is just the same: it's insane trial and testing until something more or less produce the wa…

> And all the people here who are saying that "it works" have no idea as to the amount of technical debt they're creating. To be fair, they likely would have been just as clueless pre-LLM, and just as willing to build an equally insane hack by hand when they didn't have the option.

Being clueless has in my experience previously been a rate limiter. Without LLMs these people would simply be much less productive than those with a clue, and problems they don't understand would at best compel them to read and learn, and at worst to simply avoid going too far out of their depth. Either way, it would significantly slow them down compared to their more skilled and experienced peers. The resulting rate of output practically limited the burden of reviewing and maintaining it. Then, eventually skill and experience would hopefully just sort of happen to those that work for it, through exercise and exposure to problems and review feedback.

Re: Understanding is the new bottleneck

#216

Post by t_mahmood »

derangedHorse wrote: post #210
t_mahmood wrote: post #170

Earlier quoted context omitted.

LLM doesn't need soft skills, but just knowing how to write a prompt to get the correct percentage values in a RAG and get the result close to your expectations out. Well it might be different based on the training data, which ai company, and how much you're paying. People are different, they will be sometime affected by their emotional situation, surrounding, no matter how much they're being paid, You need to unders…

Ironically, the same applies to AI. We seem to get demonstrably better results when speaking to models encouragingly [1]. It's a lot easier to re-establish goodwill when you can clear the memory and context of a program vs. a human though. [1] https://x.com/MTSlive/status/2086884672106299878

We don't know if it's all marketing stunts, or real, do we now?

> It's a lot easier to re-establish goodwill when you can clear the memory and context of a program vs. a human though.

That's the point. You can't erase human memories, experiences, (unless you bonk their head hard enough according to movies in my subcontinent, another bonk might reverse too) which will definitely affect their decisions and results.

Re: Understanding is the new bottleneck

#217

Post by mendapi »

alecbz wrote: post #6

We have LLMs try to generate descriptions of PRs for us and they're pretty universally disliked. They're always overly-complex descriptions of the mechanical changes and have no sense of motivation. Also, a huge reason to understand the code yourself is to make sure the LLM isn't wrong, but this doesn't work if an LLM is itself generating the understanding.

[flagged]

Re: Understanding is the new bottleneck

#218

Post by boomlinde »

csallen wrote: post #49
3abiton wrote: post #46

Earlier quoted context omitted.

I hate to be pedantic but you can finetune a skill to shape the PR message the way you like it. That being said, I did have exactly this issue you mentioned, but the defualt output can always be tuned.

I don't know why you got downvoted, but I find myself wanting to say some version of what you just said over and over again. People write extremely lazy, straightforward prompts and expect the LLM's intelligence to take care of all of it. But the reality is that you need to actually put some thought and effort into your prompts and provide appropriate context and examples a lot of the times if you have a very specifi…

It seems to me that you might as well put that thought and effort into describing the changes yourself. Most of my changes warrant 0-2 sentences of description below the header. A few particularly subtle bug fixes may need more to make sense of, but detailed descriptions of why certain mitigations are necessary are in my case more likely to end up as code comments.

That describing a change should frequently be such a difficult problem that instead of just doing it you prefer to put thought and effort into telling an LLM to do it smells bad to me. For me, the thought and effort spent writing a description is mostly already amortized through thinking clearly about the problem and performing the work. I have a much easier time describing what I just did and why than a machine that has no access to that information unless I tell it.

Re: Understanding is the new bottleneck

#219

Post by Forgeties79 »

gilrain wrote: post #211
Forgeties79 wrote: post #199

Earlier quoted context omitted.

It is impossible to duplicate results with an LLM. This strikes me as a serious barrier to calling it a proper skill. If you can’t even somewhat replicate the results you can’t really improve the input consistently. You can’t become “skilled” at it if you can’t even reproduce what you did. If you enter the same prompt 3 times the results are of pretty significantly different quality. ChatGPT literally has you A/B tes…

> It is impossible to duplicate results with an LLM. Humans are called experts in many domains whereby they cannot achieve duplicate results. In fact, in what domain can an expert do that? Nobody crosses the same river twice.

If you ask me to write an email, all 3 variations will be more or less the same, except it will almost always improve with each iteration. 3 cracks at a prompt is 3 separate, unrelated attempts. None of them informs the other. In fact, you run the risk of making it worse if you include previous versions.

Re: Understanding is the new bottleneck

#220

Post by simonw »

_dark_matter_ wrote: post #203
simonw wrote: post #99

Earlier quoted context omitted.

It's fascinating. On the one hand, if you really want to unlock the potential of coding agents you can get a whole lot more value from them if you don't force yourself to read every line of code they produce for you. On the other hand, that's clearly a terrible idea! These machines make mistakes. Unreviewed code is the most obvious form of technical debt - sure, you'll get a boost in the short term but how much will…

I agree with this. I am also starting to get a "feeling" of when I can trust an agent and when I can't. Recently I had it throw together a dashboard that displayed some basic linear models based on knobs on the dash, and I didn't really worry about it getting those wrong (I did spot check and it seemed good). But I also had to update a pretty complex flink app with state management changes that it totally borked. The…

[dead]
Post reply on HN