Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If you thought code writing speed was your problem you have bigger problems

andrewmurphy.io

Post reply on HN
Print view View original

221–230 of 231 posts

Re: If you thought code writing speed was your problem you have bigger problems

#221

Post by nunez »

petcat wrote: post #94
tcmart14 wrote: post #80

Earlier quoted context omitted.

I commented elsewhere, but that doesn't mean it's not deterministic. Deterministic means given the same input it gives the same output. Compilers can still have bugs and generate the wrong code. But so long as given the same input it generates the same wrong output, it is still deterministic.

Compilers can generate wrong output in many different ways. And they're all analogous to the same ways that a sophisticated LLM can generate wrong outputs. The compiler relies on: * Careful use of the ENV vars and CLI options * The host system, or the compilation of the target executable (for cross-compiling) * It relies on the source code specifically How is this really different from careful prompt engineering, and…

You can't guarantee that an LLM will yield the same answer twice, given the same input, by design. You can with compilers. That's the point being made here.

Re: If you thought code writing speed was your problem you have bigger problems

#222

Post by dd8601fn »

Sammi wrote: post #211
vidarh wrote: post #113

Earlier quoted context omitted.

The customer doesn't need to be shown every "wrong thing".

Then how will you know if it's the wrong thing? If you're not user testing then you're just guessing.

I build things iteratively all the time, without anyone seeing most of it. I move things up for review and QA after I'm satisfied that I've done the things that need doing, and done them well enough for purpose. Customers aren't the only opinion on what's good or bad. They're more like the final and most important.

Now that process is much, much faster.

Re: If you thought code writing speed was your problem you have bigger problems

#224

Post by brailsafe »

lanstin wrote: post #209
brailsafe wrote: post #168

Earlier quoted context omitted.

> The single biggest potential productivity gain though I think is being able to do something else while the agent is coding, like you can go review a PR and then when you come back check out what the agent produced. I've already passed through this phase and have given up on it. I'm sure everyone's experience will vary, but I just find it introduces either sufficiently more context switching or detracts sufficiently…

For years I worked at a large company with so many blockers for everything that I always worked like this all the time - have 5 projects so when one becomes blocked for external deps, you have one to pull out and work on. There is a context switch (which lead me to context preservation habits like checking everything I write into git; using tmux so that the context is sitting around in the bash history of the shell w…

I still do this, and don't really think it's avoidable, but when the expectation around compressed timelines because of the imaginary ability to rapidly do synchronous non-trivial tasks in parallel, that's both things get sone poorly.

Feels akin to something like driving in stop and go traffic while playing chess with a passenger who's shit talking me.

Re: If you thought code writing speed was your problem you have bigger problems

#225

Post by vidarh »

Sammi wrote: post #211
vidarh wrote: post #113

Earlier quoted context omitted.

The customer doesn't need to be shown every "wrong thing".

Then how will you know if it's the wrong thing? If you're not user testing then you're just guessing.

Presumably you have at least a partial spec, and some domain knowledge. If your invoices doesn't show amounts or a recipient address, for example, you don't need a user to tell you that is an error.

Re: If you thought code writing speed was your problem you have bigger problems

#226

Post by vidarh »

array_key_first wrote: post #200
vidarh wrote: post #113

Earlier quoted context omitted.

The customer doesn't need to be shown every "wrong thing".

In my experience this just makes them lose confidence in you and the company. So when it eventually is right, they're resistant. Worst case you lose the contract.

The opposite is true. If you spoonfeed them every error or gap during development, they'll quickly ditch you for someone who will not bother them with all the things they expect you to know how to fix without their involvement.

Re: If you thought code writing speed was your problem you have bigger problems

#227

Post by array_key_first »

vidarh wrote: post #226
array_key_first wrote: post #200

Earlier quoted context omitted.

In my experience this just makes them lose confidence in you and the company. So when it eventually is right, they're resistant. Worst case you lose the contract.

The opposite is true. If you spoonfeed them every error or gap during development, they'll quickly ditch you for someone who will not bother them with all the things they expect you to know how to fix without their involvement.

You have to do it in the right way. You want to catch as many gaps as possible before you build the thing and just as you start building it.

If you just fill in the gaps yourself, your software doesn't work. Software that doesn't meet the requirements is legitimately useless. The problem is, most of the requirements are in the customer's head, and you have to squeeze them out. Most of them are unconscious, they might not even know them.

Re: If you thought code writing speed was your problem you have bigger problems

#228

Post by Fire-Dragon-DoL »

brailsafe wrote: post #168
bvirb wrote: post #127

When we (the engineering team I work on) started using agents more seriously we were worried about this: that we'd speed up coding time but slow down review time and just end up increasing cycle time. So far there's no obvious change one way or the other, but it hasn't been very long and everyone is in various states of figuring out their new workflows, so I don't think we have enough data for things to average out y…

> The single biggest potential productivity gain though I think is being able to do something else while the agent is coding, like you can go review a PR and then when you come back check out what the agent produced. I've already passed through this phase and have given up on it. I'm sure everyone's experience will vary, but I just find it introduces either sufficiently more context switching or detracts sufficiently…

I find I context switch the same amount, but the cost of context switching was way higher before

Re: If you thought code writing speed was your problem you have bigger problems

#229

Post by vidarh »

array_key_first wrote: post #227
vidarh wrote: post #226

Earlier quoted context omitted.

The opposite is true. If you spoonfeed them every error or gap during development, they'll quickly ditch you for someone who will not bother them with all the things they expect you to know how to fix without their involvement.

You have to do it in the right way. You want to catch as many gaps as possible before you build the thing and just as you start building it. If you just fill in the gaps yourself, your software doesn't work. Software that doesn't meet the requirements is legitimately useless. The problem is, most of the requirements are in the customer's head, and you have to squeeze them out. Most of them are unconscious, they might…

Nobody has suggested anything else, but that still does not mean you need - or should - burden your customers/users with every problem you run into. Nobody does that because it's make your users run screaming to your competitors. It's why we hire experienced people who can make educated choices and package up changes and raise targeted sets of questions to stakeholders.

Re: If you thought code writing speed was your problem you have bigger problems

#230

Post by xigoi »

tcmart14 wrote: post #145
petcat wrote: post #94

Earlier quoted context omitted.

Compilers can generate wrong output in many different ways. And they're all analogous to the same ways that a sophisticated LLM can generate wrong outputs. The compiler relies on: * Careful use of the ENV vars and CLI options * The host system, or the compilation of the target executable (for cross-compiling) * It relies on the source code specifically How is this really different from careful prompt engineering, and…

And none of that means it isn't non-deterministic. Compilers still satisfy the, given the exact same environment and input, you get the same output. It doesn't matter the number of inputs. So long as f(3, 2) always gives 5, it's deterministic. Doesn't matter what f(x,y) does so long as it always gives the same output per input. LLM generation does not do this. If given f(3,2), sometimes it says 5, sometimes 6, someti…

It seems that the person you’re replying to is also deterministic. No matter how many times you explain what “deterministic” means, they fail to understand it.
Post reply on HN