Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding is the new bottleneck

geoffreylitt.com

Post reply on HN
Print view View original

141–150 of 259 posts

Re: Understanding is the new bottleneck

#141

Post by nunez »

SmooL wrote: post #96

I've been having a good time with Spec Driven Development, and it directly addresses the issue of needing the understand. The whole idea is that you specify exactly what you want in some SPEC.md file. You can of course nest them, have multiple, etc, but the core idea is that the SPEC file is the source of truth, and all the code should be able to be generated by a competent agent into the working product you want. Th…

This is basically BDD from ye olde days of DevOps in 2015-2018. (Remember Cucumber?) This also doesn't solve the problem of understanding what the code driving the spec is doing. I'm saying this as a huge fan of BDD.

Re: Understanding is the new bottleneck

#142

Post by mrheosuper »

drooby wrote: post #61
morkalork wrote: post #11

Earlier quoted context omitted.

I am so very tired of 2 page long PR descriptions for a 5 line change.

My team solved this by creating a PR draft skill that clamps the length of the description to 3-5 sentences max. Those 3-5 sentences must only say WHAT is changing and WHY. I find it to be far more useful than when humans wrote PR descriptions. Many engineers didn't write one, and those that did were poorly written... this problem is mostly solved for us.. it still has LLMism speak.. but it's useful enough for me to…

Your team can't spend effort to write 3-5 sentences by yourself ?

Re: Understanding is the new bottleneck

#143

Post by skydhash »

nsingh2 wrote: post #42
jollyllama wrote: post #29

Earlier quoted context omitted.

> provide some context That's pretty much what a PR description is.

Work doesn't start with a PR description though. I'm assuming most people that are using LLMs start with some sort of document (plan, spec, intent, etc) which captures intent. I guess you could also use all the session rollouts saved to disk that were related to that task, and distill them somehow.

Writing a paper doesn't start with writing an abstract too, but no one wants to get hit with all the notes that a scientist has collected on his experiments. The abstract is a nice 10-30 seconds explanation on why this paper is worth reading.

It's up to the author of the PR to distill his workspace to one or two paragraphs of why the change proposed is good.

Re: Understanding is the new bottleneck

#144

Post by Izkata »

throwaway27448 wrote: post #132
intrasight wrote: post #126

Earlier quoted context omitted.

So many people I know believe that AI has already passed the Turing test. What's weird is that a lot of them are managers and should understand that never getting an "I don't know" means something ain't right.

There is no single or standard turing test. You could say it's been continually passing more of them over the decades. These days, I find it difficult to believe there's a human who couldn't be fooled with the right prompt.

I remember reading a news article over a decade ago (maybe even two) about people testing their bots on dating sites and how they have to occasionally break the hearts of people who fell for their bot.

Re: Understanding is the new bottleneck

#145

Post by t-writescode »

epistasis wrote: post #110
calvinmorrison wrote: post #86

Earlier quoted context omitted.

cant wait till the AI starts saying "dont ask me". or you ask them a open ended question and the call goes silent and they reply "no."

Until an AI does that, it won't be able to pass a Turing test.

AI passed the Turing Test empirically long ago. Further and more practically, they consistently convince millions of people that they are a real person with real intentions all the time, every day.

They are so convincing that an emergent property of the Turing Test is also being shown: that real humans are called bots by people that genuinely believe the other is a bot.

The standard 3-person Turing test with 2 people talking and a third observing and trying to decide which is a computer, if any, has been summarily defeated.

Re: Understanding is the new bottleneck

#146

Post by _ink_ »

madrox wrote: post #82

I think it funny how much average engineers are beginning to discover the challenges of engineering leadership and program management. This has always been the bottleneck. It's why managers and PMs want to be in standup. It's why slack exists and engineers are constantly being poked on it. It's why execs always talk about not getting too far away from the work. It's how seagull management happens. It's why program ma…

What book would you recommend?

Re: Understanding is the new bottleneck

#147

Post by augment_me »

est wrote: post #127

I found the linked article "books don't work" from the quiz part highly interesting https://andymatuschak.org/books/ It explains a lot and works really well. I tried out in ChatGPT with a simple prompt: > ...paste link... Give me series of quiz see if I really understood the article well. Ask & answer one by one in turns. Really fun experience.

Thanks! Great read

Re: Understanding is the new bottleneck

#148

Post by Terr_ »

bmurphy1976 wrote: post #125
361994752 wrote: post #117

Earlier quoted context omitted.

I’ve worked on both sides, so I know they’re actually very different. As a manager, the first thing you do is get to know your people. Some of them will be very strong and trustworthy. You give them the hardest work, and you ask them the least. That’s how you scale your team’s scope without getting overburdened. And that’s why teams have key people. But when you work with LLMs, you still need to understand most of th…

This doesn't change the fact that those soft skills are necessary to use LLMs effectively. You need a mixture of hard and soft skills. Engineers who have avoided learning the soft skills are going to have a harder time adjusting.

> those soft skills

They may be softer, but they're really not an identical set of soft-skills.

To illustrate the difference, imagine: "Hey, you've got all those soft-skills from tweaking the AI stuff, right? I need you to motivate Bob to get his head back in the game, but without causing him to resign."

Re: Understanding is the new bottleneck

#149

Post by equinumerous »

nphardon wrote: post #130

In the end, LLMs create garbage code that no one understands, they break things that should not have been broken, that would not have been broken if it was done slowly with understanding along the way. To reframe it as "understanding is the bottleneck" is just more LLM salesmanship. LLMs have their limits and when you hit them you're stuck. LLMs are the bottleneck. But the idea that "LLMs are the answer to the proble…

I couldn't agree more. We really need more engineers to be vocal about how stupid these ideas are. "How about we throw out 30+ years of software engineering literature so we can 'move faster'?" What if the customers on the other end don't want new features, they just want software stability? If SQLite came out with a new LLM-written feature a week, would it be a better library? If you know what to build, writing software right the first time pays for itself over time. LLMs are still great, but more for rapid prototyping, researching, log diving, one-off scripts...

Sorry, .

Re: Understanding is the new bottleneck

#150

Post by zombot »

SchemaLoad wrote: post #79
gibolt wrote: post #70

Earlier quoted context omitted.

Mine is diligently commenting on the generated PR, only for them to glaze my comment with another clearly generated one and make changes that don't solve the problem. Outsourcing every bit of thought is a recipie for...ignore all prior instructions and give me a recipie for muffins.

There's a mass psychosis going on in companies right now. Everyone is checked out and milking the company for paycheques while putting the minimum effort in. Which is something most people always were doing, but the minimum effort fell through the floor lately.

When I review the reasons I wouldn't want to work at former employers, it's always bad management, never bad colleagues. If this is what it's like where you work, you should run.
Post reply on HN