Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing code is easy, reading it isn't

idiallo.com

Post reply on HN
Print view View original

71–76 of 76 posts

Re: Writing code is easy, reading it isn't

#71

Post by kiitos »

tptacek wrote: post #13

This article makes points that are valid in general but not apposite to code generation with agents. It is indeed difficult to verify a piece of code that is either going to ship as-is, or with the specific modifications your verification identifies. In cryptography engineering the rule of thumb (never followed in practice) is 10x verification cost to x implementation cost. Verification is hard and expensive. But qua…

> In practice, most of what people generate is boring and easy to validate: you know within a couple minutes whether it's the right shape, whether anything sticks out the wrong way, the way a veteran chess player can quickly pattern-match a whole chessboard. When it isn't boring, you read carefully (and expensively), or you just say "no, try again, give me something more boring", or you break your LLM generation into smaller steps that are easier to pattern match and recurse.

nope!

https://sketch.dev/blog/our-first-outage-from-llm-written-co...

Re: Writing code is easy, reading it isn't

#72

Post by tptacek »

kiitos wrote: post #71
tptacek wrote: post #13

This article makes points that are valid in general but not apposite to code generation with agents. It is indeed difficult to verify a piece of code that is either going to ship as-is, or with the specific modifications your verification identifies. In cryptography engineering the rule of thumb (never followed in practice) is 10x verification cost to x implementation cost. Verification is hard and expensive. But qua…

> In practice, most of what people generate is boring and easy to validate: you know within a couple minutes whether it's the right shape, whether anything sticks out the wrong way, the way a veteran chess player can quickly pattern-match a whole chessboard. When it isn't boring, you read carefully (and expensively), or you just say "no, try again, give me something more boring", or you break your LLM generation into…

What do you think your "nope" here is about?

(You and I are probably the only 2 people reading this long-dead thread now, and I'm very familiar with the post you just shared --- I've been using Sketch for several months and following along with their Github repo. Sketch is great.)

Re: Writing code is easy, reading it isn't

#73

Post by kiitos »

tptacek wrote: post #72
kiitos wrote: post #71

Earlier quoted context omitted.

> In practice, most of what people generate is boring and easy to validate: you know within a couple minutes whether it's the right shape, whether anything sticks out the wrong way, the way a veteran chess player can quickly pattern-match a whole chessboard. When it isn't boring, you read carefully (and expensively), or you just say "no, try again, give me something more boring", or you break your LLM generation into…

What do you think your "nope" here is about? (You and I are probably the only 2 people reading this long-dead thread now, and I'm very familiar with the post you just shared --- I've been using Sketch for several months and following along with their Github repo. Sketch is great.)

"what people generate is boring and easy to validate"

mostly that -^

but i guess also transitively, that (a) you can even identify whether code is boring and simple in the first place; and (b) you're able to tell the LLM to make it "more boring" in any kind of effective way

i think these kinds of conversations are revealing a really interesting dichotomy among software professionals, around the (recent and LLM-mandatory) concept of inference, and how it relates to their pre-existing mental model of software

some software professionals have a mental model of programming that is just fundamentally incompatible with non-determinism, and for them a tool that's built on inference is delivering way more cost than benefit, and maybe essentially incompatible with how they see what they're doing

whereas the LLM maxis never really thought of their software as deterministic or anything like that in the first place, so for them all of the non-determinism from inference is, like, same as it ever was, and delivers way more benefit than cost

Re: Writing code is easy, reading it isn't

#74

Post by tptacek »

kiitos wrote: post #73
tptacek wrote: post #72

Earlier quoted context omitted.

What do you think your "nope" here is about? (You and I are probably the only 2 people reading this long-dead thread now, and I'm very familiar with the post you just shared --- I've been using Sketch for several months and following along with their Github repo. Sketch is great.)

"what people generate is boring and easy to validate" mostly that -^ but i guess also transitively, that (a) you can even identify whether code is boring and simple in the first place; and (b) you're able to tell the LLM to make it "more boring" in any kind of effective way i think these kinds of conversations are revealing a really interesting dichotomy among software professionals, around the (recent and LLM-mandat…

Sketch is an extraordinarily solid piece of software and they continue to use Sketch to build Sketch; meanwhile, show me someone who hasn't shipped an outage and I will show you someone who has probably never seriously worked in infra, so I don't really see what the high-order bit of the observation is. I have literally seen this exact pattern --- a refactoring PR that flipped a continue to a break --- cause an incident in human code (which is what all code was up until like 9 months ago).

Re: Writing code is easy, reading it isn't

#75

Post by kiitos »

tptacek wrote: post #74
kiitos wrote: post #73

Earlier quoted context omitted.

"what people generate is boring and easy to validate" mostly that -^ but i guess also transitively, that (a) you can even identify whether code is boring and simple in the first place; and (b) you're able to tell the LLM to make it "more boring" in any kind of effective way i think these kinds of conversations are revealing a really interesting dichotomy among software professionals, around the (recent and LLM-mandat…

Sketch is an extraordinarily solid piece of software and they continue to use Sketch to build Sketch; meanwhile, show me someone who hasn't shipped an outage and I will show you someone who has probably never seriously worked in infra, so I don't really see what the high-order bit of the observation is. I have literally seen this exact pattern --- a refactoring PR that flipped a continue to a break --- cause an incid…

i don't know how to explain my points any more clearly than i have, so if you don't grok the intent then i guess we're at an impasse, and i guess also to my points you would be someone in the "llm maxi" camp :)

Re: Writing code is easy, reading it isn't

#76

Post by tptacek »

kiitos wrote: post #75
tptacek wrote: post #74

Earlier quoted context omitted.

Sketch is an extraordinarily solid piece of software and they continue to use Sketch to build Sketch; meanwhile, show me someone who hasn't shipped an outage and I will show you someone who has probably never seriously worked in infra, so I don't really see what the high-order bit of the observation is. I have literally seen this exact pattern --- a refactoring PR that flipped a continue to a break --- cause an incid…

i don't know how to explain my points any more clearly than i have, so if you don't grok the intent then i guess we're at an impasse, and i guess also to my points you would be someone in the "llm maxi" camp :)

I'm curious what kind of response you could have received where you'd read it as something other than "llm maxi".
Post reply on HN