Live data from Hacker News

AI doesn't generate working products, that's still your job

weeraman.com

171–180 of 317 posts

Re: AI doesn't generate working products, that's still your job

#172

Earlier quoted context omitted.

Popped over to HackerNews, read two comment sections and the top comments in both articles were users saying the same thing: "AI can't write code! The whole thing will come crumbling down any minute! Just you wait!" I've never seen this community like this. Are these people cooked? We are years into this and they haven't been able to figure it out? They are going to continue to tell people using these tools successfu…

> Are these people cooked? Yes. This is a weird kind of denial about the LLMs that cost nation-state levels of energy doing human nerd work. They aren't perfect at all, but there's very little reason to be anything but a prompt engineer and proof reader anymore.

This is an absurdly overgeneralized take for a wide and diverse field, and it's also false. Of course for some people it's true, but hardly for everyone. I doubt it's even the majority of all programmers.

Re: AI doesn't generate working products, that's still your job

#173
post #4

The fear in most people is not of losing the job, but of losing their value in the market as ai raises the floor of capabilities of other people competing for the same job.

I think you've misidentified the fear.

People worry about losing their jobs in a market that hires fewer people (because AI makes doing more work easier) but not about losing their jobs because they can't keep up.

The whole point of AI is that you can't meaningfully fall behind. If you ever did, you could ask AI to get you up to speed again. The tech that automates work and effort is not something you need work and effort to use.

Similarly, I don't currently have a bread machine. I am not falling behind in baking skills to people who do, because I could always, at literally any point, get a bread machine and hit the exact same productivity level.

Re: AI doesn't generate working products, that's still your job

#174

I'm about to throw away multiple months of LLM generated code for one of my side projects. I was really careful writing design specs and it wasn't even a new code base the LLM worked on, but still after several months of AI changes I feel my code degraded more and more into a subtle mess. Hard to explain, each individual change looked good and logical and on the surface the codebase looks fine, but looking at the who…

This has happened to my side project too. There's so much going on in the code now that I can't possibly grok it to make changes safely by myself any more. But I think that's OK. Because I don't have to grok it anymore. I've been crafting beautiful code for almost 20 years - that itch is scratched. Now I just want to be productive and build cool stuff. AI is helping me make a better end-product in a fraction of the t…

Interesting point about "letting go". I hear that quite a lot, which I find surprising. In IT, when new technologies like cloud computing emerge people always need to adapt their workflows, and for some that is challenging e.g. going from manually maintained servers to virtual machines or containers that are just spun up and down on demand. Maybe it is similar with AI, we gain new capabilities along one dimension like speed of development and we lose some capabilities along the way e.g. manual control of quality. And of course there are large financial incentives here as well, maybe they are larger than we have ever seen before and the change also happens faster. Cloud computing took probably multiple decades to be fully adopted (I think AWS became available in 2006 and we still see large corporations migrating to the cloud from their on premise setups today, though the adoption curve is flattening), LLMs have significantly higher adoption after only around 2-3 years of them becoming "production-ready". So naturally people struggle with how to adopt them and we need to figure out where they make sense and where not. That said it's precisely an engineers' job to figure that out, people that just see the upsides of this technology seem quite naive to me.

I can see this struggle it in my organization as well, in the last year there was a big push to adopt AI everywhere and tons of initiatives to automate processes and produce code and text and other artefacts with LLMs. Now it seems the pendulum is swinging back a little as people see that all of the LLM generated stuff shows all of these subtle quality degradations, and people get tired of managing it as they are suddenly confronted with tons of additional information they need to manage.

Given that models are still evolving and becoming better at a rapid pace I think that we will solve most of these issues in the near future, but for now I don't think the age of hand crafted code is over yet.

I have been thinking about that machine code analogy before as well, I don't think it really holds. Machine code is written in an automated way but following mostly deterministic rules that have been crafted through decades of manual optimizations and testing. AI generated code has nowhere near this level of scrutiny, testing and optimization behind itself. The fault rate of compilers and optimizers is incredibly small (I can't find any numbers but it must be on the order of ppm or ppb), AI generated code has fault rates that are even in the best case on the order of 99-99.99 % maybe (i.e. between one error per hundred lines and one error per ten thousand lines in the best case), try building anything complex using such a fault rate without manual correction and review. It's impossible. I have done it, I dabbled with writing a compiler, a database and even a simple web framework from first principles. I didn't get far, even though I had good mental models of these things and I carefully wrote RFCs and documents for the LLM, specified test cases etc... If you're lucky it will regurgitate some existing code or follow documented guidelines, but when you're on new territory these models won't be able to produce anything good. I would really like to see a single example of someone vibe coding a high quality library or tool with LLMs, I haven't found anything and no one can point me to a complex codebase (say 10,000 lines or more) that was generated using high level prompts that looks decent and doesn't have multiple glaring issues that appear when looking at it in detail.

Re: AI doesn't generate working products, that's still your job

#175

Earlier quoted context omitted.

Why is the answer of LLM-maximalists always you are holding it wrong ? I think a lot of experienced programmers, who have adopted LLMs early on, have the same finding: yes, LLMs give a great productivity boost. No, you cannot let agents completely wild without plenty of human supervision, because it will lead to a big ball of mud and atrophy knowledge of humans. I think these are relevant data points and they should…

look at the sibling reply. these back-handed concessions are always either “AI is a great productivity boost if you’re just slamming out a CRUD web app” or the special snowflake “my big beautiful codebase is too complex.” maybe they are holding it, wrong with attitudes like that it’s hard to take them seriously. the phd holders at my office are some of the most prodigious vibe coders we have ps: since when was comple…

It's not even complexity; raw size suffices to make a codebase hard for the LLM to manage, which is especially bad given the high ratio of LLM lines to ideal lines for any given change. E.g., if a concurrent hashmap is a reasonable primitive, it won't check if any of the existing ones are good fits. It'll instead prefer to implement something simpler but totally broken, or else it'll try and fail to build yet another concurrent hashmap. If you fix that by giving it explicit instructions to use a particular implementation, it'll manufacture extra (incorrect or low-performance) methods out of nowhere rather than solving within the constraints of the existing API -- think things like adding a global lock around the whole thing to make "concurrent" iteration possible when iteration isn't even a primitive the thing you asked the LLM to do requires.

A colleague of mine had the observation that the problem we're seeing is a strong tendency to produce solutions which add extra, undesirable connections in the code, resulting in inordinately complex API boundaries which are hard to properly disentangle.

That's all fixable with enough effort, but it's hard to ignore that experience when somebody comes along and tells you that vibe coding really does suffice by itself, and you can just point a good model and harness at a ticket and a codebase and let it rip.

Re: AI doesn't generate working products, that's still your job

#176

I'm about to throw away multiple months of LLM generated code for one of my side projects. I was really careful writing design specs and it wasn't even a new code base the LLM worked on, but still after several months of AI changes I feel my code degraded more and more into a subtle mess. Hard to explain, each individual change looked good and logical and on the surface the codebase looks fine, but looking at the who…

LLMs can write excellent code a few thousand lines at a time but they absolutely can't create a cohesive million line codebase.

Re: AI doesn't generate working products, that's still your job

#177
post #24

The test is simple: have we seen great new products or improvements in the products we use over the past 12,24,36 months? The only great new product I’ve used is my LLM of choice, and those labs seem to be hiring more humans than ever. Maybe it’s true that Claude only just got good enough and that 12 months from now our day to day lives will be way better thanks to LLM-driven product improvements/breakthroughs. My be…

I can think few great tools like t3code (a software that does a lot) that have been mostly coded by LLMs and yet have a solid architectural foundation and implementation. The software is genuinely gargantuan in its features and capability, you really cannot appreciate it from quickly skimming the readme.

At work, LLMs helped a lot in doing the kind of work you were always tired or had no budget to do, and the codebases have never been healthier, more tested and better documented.

In any case, it's always about the effort and capabilities of the people using it. If you want to vibecode, then you're still stuck in prototype land and I think that even better models won't help, in fact they may have negative value.

The naivety of your post is in any case appalling, you seem to be missing the forest for the tree.

Re: AI doesn't generate working products, that's still your job

#179

Earlier quoted context omitted.

Popped over to HackerNews, read two comment sections and the top comments in both articles were users saying the same thing: "AI can't write code! The whole thing will come crumbling down any minute! Just you wait!" I've never seen this community like this. Are these people cooked? We are years into this and they haven't been able to figure it out? They are going to continue to tell people using these tools successfu…

you have to remember, 70 million people voted for the current US regime and many of them billionaires. The ability to delude ones self and not pay attention to the "facts on the code" is moumentally built into the human condition. Everyones experience is some unique snowflake of just how easy their mental gymnastics are.

> you have to remember, 70 million people voted for the current US regime and many of them billionaires.

Many of the 70 million Republican voters were billionaires? What are you talking about?

Post reply on HN