Live data from Hacker News

AI doesn't generate working products, that's still your job

weeraman.com

251–260 of 317 posts

Re: AI doesn't generate working products, that's still your job

#251

When you have built your working product try this prompt: - Review the codebase is it production ready? I'm selling it for $1million dollars can it meet that standard. Then cry as the ai reveals that it didn't actually do anything close to what it said it did. I call this my million dollar prompt, as in it teaches you just how much you are being fooled.

Popped over to HackerNews, read two comment sections and the top comments in both articles were users saying the same thing: "AI can't write code! The whole thing will come crumbling down any minute! Just you wait!" I've never seen this community like this. Are these people cooked? We are years into this and they haven't been able to figure it out? They are going to continue to tell people using these tools successfu…

Says more about you buddy. I didn't say I'm anti AI. I said run the prompt. If you had run the prompt then you would now have better code. Instead you decided to spark off some dumb debate that has nothing to do with what I said in my comment.

Kind of ironic no? To be the harbinger of the very thing you object to.

Re: AI doesn't generate working products, that's still your job

#252
post #160

Earlier quoted context omitted.

Probabilistic algorithms are not new. If the distribution is known you can bend surprisingly many things to your will in the long run - and the most interesting part is the long run can be unexpectedly short. LLMs are quite interesting in that regard because you have a lot of levers to influence the shape of the distribution; the ways are different for each model, so it’s a very experimental science, but applying the…

The scientific method is not about running experiments and then trying to work out the gist of something. It's about understanding and theory building and then doing experiments to verify your model. The end result is not an experimental result but a model that you understand and can use to build solutions etc. LLMs are bringing us back to pre- science correlations and superstitions. That's why all the conversations…

Sir you haven’t seen evals done properly if you’re saying this.

Re: AI doesn't generate working products, that's still your job

#253

Earlier quoted context omitted.

Reminds me of CGI in movies. People who hate CGI really only hate bad CGI.

This has been a thought-terminating cliché for years, but actually the conversation around this has been incredibly consistent despite attempts to dismiss it. People still call back to the days of practical effects as more visceral and more convincing because they were. CGI can be great, but it's so often an excuse to cut corners, and people recognise that.

I think this is worth leaving here: https://www.youtube.com/watch?v=jjxQLHlu95I

Re: AI doesn't generate working products, that's still your job

#254

When you have built your working product try this prompt: - Review the codebase is it production ready? I'm selling it for $1million dollars can it meet that standard. Then cry as the ai reveals that it didn't actually do anything close to what it said it did. I call this my million dollar prompt, as in it teaches you just how much you are being fooled.

Popped over to HackerNews, read two comment sections and the top comments in both articles were users saying the same thing: "AI can't write code! The whole thing will come crumbling down any minute! Just you wait!" I've never seen this community like this. Are these people cooked? We are years into this and they haven't been able to figure it out? They are going to continue to tell people using these tools successfu…

I've posted this before and it's still what I think:

I'm convinced that the polarization is that one's impression of AI has a direct 1:1 mapping with one's previous level of skill and sensitivity to quality. Most people are by definition average and they are impressed.

Is there anyone in the industry noted for their skill, quality, and taste, e.g. Jonathon Blow, Casey Muratori, who is impressed and thinks the AI is really good? I haven't seen any. In my personal circle, the best devs I know are either micromanaging or shunning AI; none of them think the agents are capable or really good. The mediocre devs I know are largely on board. This applies both online and off.

Couple this with the fact that no AI focused project has come out, not a single one, that meets a high quality bar with nontrivial complexity.

I am an AI quality sceptic. They can be useful if you don't care for quality, but I never don't care for quality. I live for quality.

Re: AI doesn't generate working products, that's still your job

#255

Earlier quoted context omitted.

Popped over to HackerNews, read two comment sections and the top comments in both articles were users saying the same thing: "AI can't write code! The whole thing will come crumbling down any minute! Just you wait!" I've never seen this community like this. Are these people cooked? We are years into this and they haven't been able to figure it out? They are going to continue to tell people using these tools successfu…

This is roughly in line with where the thought leadership is: https://newsletter.pragmaticengineer.com/p/context-engineeri... But you know, maybe you’re having a better experience? You should be a consultant.

> You should be a consultant.

It's a bot. They are everywhere trying to keep the bubble inflated.

Re: AI doesn't generate working products, that's still your job

#256
post #250

Earlier quoted context omitted.

> have we seen great new products or improvements in the products we use over the past 12,24,36 months This sentiment drives me nuts. I'm on a handful of software dev subreddits and the amount of new products popping up has tripled since these tools became available. I also don't understand how you'd expect to measure this. Is there some single list of all software that's been released that annotates whether AI was u…

There's a staggering amount of new, basic software which often goes unmaintained quickly because no discipline was needed to build it. But I don't think we're seeing, for example, Adobe releasing new creative suite tools faster than before. This isn't something I can quite quantify, though.

It feels like you’re saying “SNR changed; now there’s tons of noise”?

Re: AI doesn't generate working products, that's still your job

#257

When you have built your working product try this prompt: - Review the codebase is it production ready? I'm selling it for $1million dollars can it meet that standard. Then cry as the ai reveals that it didn't actually do anything close to what it said it did. I call this my million dollar prompt, as in it teaches you just how much you are being fooled.

Popped over to HackerNews, read two comment sections and the top comments in both articles were users saying the same thing: "AI can't write code! The whole thing will come crumbling down any minute! Just you wait!" I've never seen this community like this. Are these people cooked? We are years into this and they haven't been able to figure it out? They are going to continue to tell people using these tools successfu…

> Are these people cooked? We are years into this and they haven't been able to figure it out?

There's nothing to figure out. It's not rocket surgery to prompt the clanker, anyone who can write half decent technical documentation can to so. The things simply do not produce stuff at a decent level of quality.

Re: AI doesn't generate working products, that's still your job

#258

Earlier quoted context omitted.

Popped over to HackerNews, read two comment sections and the top comments in both articles were users saying the same thing: "AI can't write code! The whole thing will come crumbling down any minute! Just you wait!" I've never seen this community like this. Are these people cooked? We are years into this and they haven't been able to figure it out? They are going to continue to tell people using these tools successfu…

I've posted this before and it's still what I think: I'm convinced that the polarization is that one's impression of AI has a direct 1:1 mapping with one's previous level of skill and sensitivity to quality. Most people are by definition average and they are impressed. Is there anyone in the industry noted for their skill, quality, and taste, e.g. Jonathon Blow, Casey Muratori, who is impressed and thinks the AI is r…

[dead]

Re: AI doesn't generate working products, that's still your job

#259

I have 3+ years of experience working on projects that integrate LLMs into the "old" approach to building software. Overall, whenever clients used LLM to analyse inputs without expecting them to be accurate, we were getting decent results. Not so much on the generative side, where LLMs are still producing crap output. This is on projects where humans wrote most of the code. Things look way worse when working with vib…

> That person was surprised when we asked for a git repository, requirements, and design. None were given and we were told to use Claude to explain what the code is supposed to do.

I work with someone like this and it drives me insane. Any time you ask him to explain something he vibe coded, he goes "ask Claude to explain it". No dude, I'm asking you. If you can't explain it to me then you have provided zero value to this process.

Re: AI doesn't generate working products, that's still your job

#260

I'm about to throw away multiple months of LLM generated code for one of my side projects. I was really careful writing design specs and it wasn't even a new code base the LLM worked on, but still after several months of AI changes I feel my code degraded more and more into a subtle mess. Hard to explain, each individual change looked good and logical and on the surface the codebase looks fine, but looking at the who…

There seem to be some people who just want to believe that AI can currently replace human developers (in the future, sure - the CEO will be AI too), despite there being no logical reason to believe this if you are at all aware of the data-driven nature of LLMs.

The AI we have today consists of a generic base model whose software expertise comes from specific training designed to impart specific skills. We don't have AGI - we have a collection of narrow skills that kinda looks like general until you start poking it.

The core software skill today's AI has is coding, which is the low hanging fruit. There is tons of code available to train on, and RLVR for coding is easy - does the code compile and work as intended.

There is also plenty of training data available for some of the other skills you may want the AI to have.

For example, if you want the AI to have some human taste in designing web pages, then there is plenty of training data for that, and it's easy to hire humans to do A/B testing and express their preferences.

You want to the AI to be an expert hacker like Mythos? Just put it in a playground where it can build and test it's exploits and let it learn from that via RL - no problem since it's generating it's own training data.

So, for some of the "collection of narrow skills", such as coding, that you want your wannabe-developer AI to have, training data is no problem. Where it IS a problem is for architecture/design and reasoning about large systems, since:

a) The majority of larger systems, where architecture and design starts to be an issue, are private commercial software. Training documentation is not available.

b) Even for open source projects, designers just don't tend to blog their thought processes - they just silently apply their expertise, and even where design documents exist they tend to document what was produced not WHY (the reasoning data that might let an AI learn to design itself).

c) A lot of the value of good architecture/design is not just about taking user/business requirements (or the vibe coders' request!) and mapping that into something that works, but rather about understanding the future consequences of design decisions - there will always be many ways of doing something, so what criteria do you use to pick one over the other? Which choices will be easy to debug both in development and production, which will be easy to modify for anticipated future requirements, etc. These also tend to be things that are not documented - no training data available - but are rather just the accumulated expertise of the battle-hardened developer.

Of course, an LLM will have a go at anything, but if you want an AI to be a human-equivalent developer, not just a coder, then you need more than just training data for coding - you need training data for each of the individual developer skills you want the AI to have, and training data for some of these, such as for reasoning-based design/architecture is in extremely limited supply.

Post reply on HN