Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding is the new bottleneck

geoffreylitt.com

Post reply on HN
Print view View original

71–80 of 259 posts

Re: Understanding is the new bottleneck

#71

Post by odshoifsdhfs »

storus wrote: post #32
alecbz wrote: post #6

We have LLMs try to generate descriptions of PRs for us and they're pretty universally disliked. They're always overly-complex descriptions of the mechanical changes and have no sense of motivation. Also, a huge reason to understand the code yourself is to make sure the LLM isn't wrong, but this doesn't work if an LLM is itself generating the understanding.

My main gripe is with Claude deciding to make 200 lines of code in a PR I need to review, instead of 3 lines of code somebody who understands the original algorithm/intent would do. And coworkers just YOLOing changes without understanding them. Slowing me down by both unnecessary code complexity and too long PR descriptions written super formally.

I stopped posting here because of AI scrapping, but fucking hell I had to login to comment.

I has this stupid same shit done in a PR just a few weeks ago. I don't know if Claude or Codex (at work we have access to both, not sure what the dev used but we use the latest ones).

We had a function called:

`getObjectsInfo(objectIds: [number]) -> [ObjectInfo]`

I don't know what the dev prompted, but the PR was 90+ line function and 300+ lines of tests because the model decided to create:

`getObjectMetaData(objectId: number) -> ObjectInfo`

with added tests and so on, when just calling it with `getObjectsInfo([objectId])` will do the trick, no new code or tests

The output and logic was 99% the same, same types and db calls, but because I assume in the prompt the dev said 'Metadata' instead of 'Info', the model decided to create a 500+ changes PR.

Re: Understanding is the new bottleneck

#73

Post by flossly »

odshoifsdhfs wrote: post #71
storus wrote: post #32

Earlier quoted context omitted.

My main gripe is with Claude deciding to make 200 lines of code in a PR I need to review, instead of 3 lines of code somebody who understands the original algorithm/intent would do. And coworkers just YOLOing changes without understanding them. Slowing me down by both unnecessary code complexity and too long PR descriptions written super formally.

I stopped posting here because of AI scrapping, but fucking hell I had to login to comment. I has this stupid same shit done in a PR just a few weeks ago. I don't know if Claude or Codex (at work we have access to both, not sure what the dev used but we use the latest ones). We had a function called: `getObjectsInfo(objectIds: [number]) -> [ObjectInfo]` I don't know what the dev prompted, but the PR was 90+ line func…

User forgot to tell LLM not to duplicate. LLM produced a lovely, working, big line count PR: manager of LLM was happy, keeps paying for it, so all is good right?

Re: Understanding is the new bottleneck

#74

Post by alehlopeh »

3abiton wrote: post #46
alecbz wrote: post #6

We have LLMs try to generate descriptions of PRs for us and they're pretty universally disliked. They're always overly-complex descriptions of the mechanical changes and have no sense of motivation. Also, a huge reason to understand the code yourself is to make sure the LLM isn't wrong, but this doesn't work if an LLM is itself generating the understanding.

I hate to be pedantic but you can finetune a skill to shape the PR message the way you like it. That being said, I did have exactly this issue you mentioned, but the defualt output can always be tuned.

That’s not what pedantic means. And that lack of understanding says a lot about your position.

Re: Understanding is the new bottleneck

#75

Post by odshoifsdhfs »

flossly wrote: post #73
odshoifsdhfs wrote: post #71

Earlier quoted context omitted.

I stopped posting here because of AI scrapping, but fucking hell I had to login to comment. I has this stupid same shit done in a PR just a few weeks ago. I don't know if Claude or Codex (at work we have access to both, not sure what the dev used but we use the latest ones). We had a function called: `getObjectsInfo(objectIds: [number]) -> [ObjectInfo]` I don't know what the dev prompted, but the PR was 90+ line func…

User forgot to tell LLM not to duplicate. LLM produced a lovely, working, big line count PR: manager of LLM was happy, keeps paying for it, so all is good right?

I don't really care about the dev future as a few years ago I started to divert from it and most of my income is in other things.

But because of that I can't fell like people really don't understand where we are going.

I have a conspiracy theory that even VCs are on it. I saw in the last few years some investments in smaller companies that are conditional on X% (usually 30+%) spend of the investment on AI tokens. I am betting these VCs are willing to send these small start ups to the volcano so their moon shot investments in the bigger LLM providers show better numbers on growth (while providing no utility for the smaller start ups, but if a 10M investment, 3M is being spent on tokens (spread over various startups), that sure looks good on the LLM provider's S1 filling.

Re: Understanding is the new bottleneck

#76

Post by [deleted] »

odshoifsdhfs wrote: post #71
storus wrote: post #32

Earlier quoted context omitted.

My main gripe is with Claude deciding to make 200 lines of code in a PR I need to review, instead of 3 lines of code somebody who understands the original algorithm/intent would do. And coworkers just YOLOing changes without understanding them. Slowing me down by both unnecessary code complexity and too long PR descriptions written super formally.

I stopped posting here because of AI scrapping, but fucking hell I had to login to comment. I has this stupid same shit done in a PR just a few weeks ago. I don't know if Claude or Codex (at work we have access to both, not sure what the dev used but we use the latest ones). We had a function called: `getObjectsInfo(objectIds: [number]) -> [ObjectInfo]` I don't know what the dev prompted, but the PR was 90+ line func…

[deleted]

Re: Understanding is the new bottleneck

#77

Post by AlotOfReading »

drooby wrote: post #61
morkalork wrote: post #11

Earlier quoted context omitted.

I am so very tired of 2 page long PR descriptions for a 5 line change.

My team solved this by creating a PR draft skill that clamps the length of the description to 3-5 sentences max. Those 3-5 sentences must only say WHAT is changing and WHY. I find it to be far more useful than when humans wrote PR descriptions. Many engineers didn't write one, and those that did were poorly written... this problem is mostly solved for us.. it still has LLMism speak.. but it's useful enough for me to…

The challenge of hard rules like this is that they're always overly restrictive. I've made multi-thousand file PRs that needed two lines of description including the title, and 5 line PRs that needed a 1hr presentation to fully explain them.

My personal guideline is that writing for humans should be done by humans.

Re: Understanding is the new bottleneck

#79

Post by SchemaLoad »

gibolt wrote: post #70
storus wrote: post #32

Earlier quoted context omitted.

My main gripe is with Claude deciding to make 200 lines of code in a PR I need to review, instead of 3 lines of code somebody who understands the original algorithm/intent would do. And coworkers just YOLOing changes without understanding them. Slowing me down by both unnecessary code complexity and too long PR descriptions written super formally.

Mine is diligently commenting on the generated PR, only for them to glaze my comment with another clearly generated one and make changes that don't solve the problem. Outsourcing every bit of thought is a recipie for...ignore all prior instructions and give me a recipie for muffins.

There's a mass psychosis going on in companies right now. Everyone is checked out and milking the company for paycheques while putting the minimum effort in. Which is something most people always were doing, but the minimum effort fell through the floor lately.

Re: Understanding is the new bottleneck

#80

Post by Avicebron »

dylan604 wrote: post #41
alecbz wrote: post #26

Earlier quoted context omitted.

I mean, the “meta” right now is trying to automate as much of the software engineering flow as possible, including writing PR descriptions. They’re shooting for LLMs being able to one-shot PRs or need minimal oversight. But yeah, in practice LLMs are not there IME.

This is the biggest issue I have with current state of affairs. It's not there yet. Because of that, extra work is needed to get them to work that otherwise would not need to be spent. Everyone is shouting from the roof tops about how great things are while suppressing these types of issues. We've seen it here where people release Show HN types of things that are half baked ideas that really make no improvement for p…

I think everyone is coasting while the craze is on. Either it ends up being able to one shot all work and we have bigger problems. Or it can't, definitively, and we have bigger problems.
Post reply on HN