Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Codegen is not productivity

antifound.com

Post reply on HN
Print view View original

81–90 of 103 posts

Re: Codegen is not productivity

#81

Post by sarchertech »

nubg wrote: post #27

For me it's simple: 1. Assume you're to work on product/feature X. 2. If God were to descend and give you a very good, reality-tested spec: 3. Would you be done faster? Of course, because as every AI doomer says, writing code was never the bottleneck!!1! 4. So the only bottleneck is getting to the spec. 5. Guess what AI can help you with as well, because you can iterate out multiple versions with little mental effort…

And then it turns out God wrote the spec in code because that’s what any spec sufficient to produce the same program from 2 different teams/LLMs would be.

Re: Codegen is not productivity

#82

Post by ChicagoDave »

slopinthebag wrote: post #78
ChicagoDave wrote: post #61

I continue to jump into these discussions because I feel like these upvoted posts completely miss what’s happening… - guardrails are required to generate useful results from GenAI. This should include clear instructions on design patterns, testing depth, and iterative assessments. - architecture decision records are one useful way to prevent GenAI from being overly positive. - very large portions of code can be compl…

> - No one should care about actual code ever again. It’s ephemeral. The role of software engineering is now molding features and requirements into functional results. Choosing Rust, C#, Java, or Typescript might matter depending on the domain, but then you stop caring and focus on measuring success. I think this has always been the case. "Bad programmers worry about the code. Good programmers worry about data struct…

Data structures are still conversational items. I come from the DDD community and adamantly push back on data first architectures. Modules or Bounded Contexts reveal their relationships and data over time.

Re: Codegen is not productivity

#83

Post by sarchertech »

nubg wrote: post #38

rules of thumb for when to take blog posts about AI coding seriously: - must be using the latest state of the art model from the big US labs - must be on a three digit USD per month plan - must be using the latest version of a full major harness like codex, opencode, pi - agent must have access to linting, compilation tools and IDE feedback - user must instruct agent to use test driven development and write tests for…

> All my comments are written by AI. Quite meta, isn't it, knowing you came here after I triggered you with my "guys, this is AI generated slop" comment?

Maybe knock it off since the rules changed to not allow AI comments.

Re: Codegen is not productivity

#84

Post by hypeatei »

greggyb wrote: post #48
zer00eyz wrote: post #36

I went to look at some of the authors other posts and found this: https://www.antifound.com/posts/advent-of-code-2022/ So much of our industry has spent the last two decades honing itself into a temple built around the idea of "leet code". From the interview to things like advent of code. Solving brain teasers, knowing your algorithms cold in an interview was always a terrible idea. And the sort of engineers it invit…

> The complaints about LLM's that lack any information about the domains being worked in, the means of integration (deep in your IDE vs cut and paste into vim) and what your asking it to do (in a very literal sense) are the critical factors that remain "un aired" in these sorts of laments. I'm not sure if this is a direct response to the article or a general point. The article includes an appendix about my use of LLM…

Not GP, but your appendix about LLM usage matches exactly how I use it too: mainly for rubber ducking and research. The codegen it's useful for (that I've found) is generating unit tests. Code coverage tools and a quick skim are more than sufficient for quality checks since unit tests are mostly boilerplate and you want to make sure that different branches are being covered.

Re: Codegen is not productivity

#85

Post by sarchertech »

christophilus wrote: post #34
Frieren wrote: post #22

Earlier quoted context omitted.

> I can You can because, I guess, your project may have a small scope, few people working on it, no dependencies etc. I cannot, because each line that I change has an effect in millions of other lines and hundreds of other people, and millions of users. Different situation, different needs.

> each line that I change has an effect in millions of other lines That sounds like an architectural problem.

It probably is, but show me a large AI assisted or vibe coded code base where this won’t be the case.

Re: Codegen is not productivity

#86

Post by greggyb »

hypeatei wrote: post #84
greggyb wrote: post #48

Earlier quoted context omitted.

> The complaints about LLM's that lack any information about the domains being worked in, the means of integration (deep in your IDE vs cut and paste into vim) and what your asking it to do (in a very literal sense) are the critical factors that remain "un aired" in these sorts of laments. I'm not sure if this is a direct response to the article or a general point. The article includes an appendix about my use of LLM…

Not GP, but your appendix about LLM usage matches exactly how I use it too: mainly for rubber ducking and research. The codegen it's useful for (that I've found) is generating unit tests. Code coverage tools and a quick skim are more than sufficient for quality checks since unit tests are mostly boilerplate and you want to make sure that different branches are being covered.

I've had a large project recently which has biased my view on unit testing from LLMs. It includes a lot of parsing and other workflows that require character-specific placement in strings for a lot of tests. Due to how tokenization works, that is a gnarly use case for LLMs. I am trying not to form too many strong opinions about LLM-driven-TDD based on it. My forays into other domains show better results for unit tests, but the weight of my experience is in this particular low point lately.

Re: Codegen is not productivity

#87

Post by slopinthebag »

ChicagoDave wrote: post #82
slopinthebag wrote: post #78

Earlier quoted context omitted.

> - No one should care about actual code ever again. It’s ephemeral. The role of software engineering is now molding features and requirements into functional results. Choosing Rust, C#, Java, or Typescript might matter depending on the domain, but then you stop caring and focus on measuring success. I think this has always been the case. "Bad programmers worry about the code. Good programmers worry about data struct…

Data structures are still conversational items. I come from the DDD community and adamantly push back on data first architectures. Modules or Bounded Contexts reveal their relationships and data over time.

Perhaps you can think of the modules or "Bounded Contexts" as a type of data structure and the relationships between them. Idk. I don't have a particularly great view of DDD fwiw.

Re: Codegen is not productivity

#88

Post by sarchertech »

ChicagoDave wrote: post #61

I continue to jump into these discussions because I feel like these upvoted posts completely miss what’s happening… - guardrails are required to generate useful results from GenAI. This should include clear instructions on design patterns, testing depth, and iterative assessments. - architecture decision records are one useful way to prevent GenAI from being overly positive. - very large portions of code can be compl…

> No one should care about actual code ever again. It’s ephemeral.

> very large portions of code can be completely regenerated quickly when scope and requirements change.

This is complete and utter nonsense coming from someone who isn't actually sticking around maintaining a product long enough in this manner to see the end result of this.

All of this advice sounds like it comes from experience instead of theoretical underpinning or reasoning from first principles. But this type of coding is barely a year old, so there's no way you could have enough experience to make these proclamations.

Based on what I can talk about from decades of experience and study:

No natural language specification or test suite is complete enough to allow you to regenerate very large swaths of code without changing thousands of observable behaviors that will be surfaced to users as churn, jank, and broken workflows. The code is the spec. Any spec detailed enough to allow 2 different teams (or 2 different models or prompts) to produce semantically equivalent output is going to be functionally equivalent to code. We as an industry have learned this lesson multiple times.

I'd bet $1,000 that there is no non-trivial commercial software in existence where you could randomly change 5% of the implementation while still keeping to the spec and it wouldn't result in a flood of bug reports.

The advantage of prompting in a natural language is that the AI fills in the gaps for you. It does this by making thousands of small decisions when implementing your prompt. That's fine for one offs, and it's fine if you take the time to understand what those decisions are. You can't just let the LLM change all of those decision on a whim, which is the natural result of generating large swaths of code, ignoring it, and pretending it's ephemeral.

Re: Codegen is not productivity

#89

Post by allajfjwbwkwja »

simianwords wrote: post #6
emp17344 wrote: post #4

The collaboration aspect is what many AI enthusiasts miss. As humans, our success is dependent on our ability to collaborate with others. You may believe that AI could replace many individual software engineers, but if it does so at the expense of harming collaboration, it’s a massive loss. AI tools are simply not good at collaborating. When you add many humans to a project, the result becomes greater than the sum of…

AI will allow us to collaborate on higher level decisions and not on whether we should use for loops or functional interfaces.

This attitude is acceptable for principal engineers, but anyone lower in the chain with this perspective typically presides over a product that is constantly breaking due to hundreds of unhandled edge cases, AI or not.

Re: Codegen is not productivity

#90

Post by JanneVee »

simianwords wrote: post #17
JanneVee wrote: post #11

Earlier quoted context omitted.

I read that as you have never been debugging a production issue at 3am while losing data and/or revenue.

AI helps in both, so not sure what your point is? its like saying "don't write code because we will have to debug it later".

Well you aren't writing the code, the AI is and you are letting the AI debug that created it in the first place and it doesn't learn from the experience in the same way. Hopefully you understand the problem in such a degree that you can spec away the problem in the next iteration. I'm seeing that issue now, people just forget to learn what the issue is and keep repeating mistakes that are regurgitated from the training material.
Post reply on HN