Live data from Hacker News

Eight years of wanting, three months of building with AI

lalitm.com

321–330 of 349 posts

Re: Eight years of wanting, three months of building with AI

#321

> Tests created a similar false comfort. Having 500+ tests felt reassuring, and AI made it easy to generate more. But neither humans nor AI are creative enough to foresee every edge case you’ll hit in the future; there are several times in the vibe-coding phase where I’d come up with a test case and realise the design of some component was completely wrong and needed to be totally reworked. This was a significant con…

Personally i think the challenge of testing never really changed with AI. You need to know what you want to specifically test before writing/vibe coidng with it. Otherwise it'll just manufacture tests that always passes and are of 0 value.

If some component doest benefit from being extensively tested, then it's still the same today. The difference is now it's so easy to generate something, no matter how useless it is. Worse part is, no one cares. Test passes, it doesn't affect production, line coverage increases, managers think the software is more tested, developers just let a prompt do everything. It's all just testing theatre.

I think E2E is the more important than ever. AI is pretty good at getting the local behaviour correct. So unit tests are of less value. Same can't be said for the system as a whole. The best part is, AI is actually pretty good at writing E2E tests. Ofc, given that you already know what you want to test

Re: Eight years of wanting, three months of building with AI

#322

Earlier quoted context omitted.

> It is quite literally making this technology, information, and power available to only the elite. I don't know what to say to you. More people are coding now with AI than ever coded before. If your argument was true, then that would just mean that there are more elites than ever. Obviously that's not what's happening. > What payout? Zero AI companies are profitable. Because they're reinvesting profits into continue…

> I don't know what to say to you. More people are coding now with AI than ever coded before. If your argument was true, then that would just mean that there are more elites than ever. Obviously that's not what's happening. I don't know how I can explain this any more clearly. If you need AI to create software, and the cost of AI is $200/month, then only people who can afford $200/month can create software. Costs wil…

> I don't know how I can explain this any more clearly. If you need AI to create software, and the cost of AI is $200/month, then only people who can afford $200/month can create software.

Your entire hypothetical is based on "ifs" that aren't true. Nothing in this sentence is true. You don't need AI to create software, the cost of AI development is much less than $200/month on average, and many more people can afford AI dev than programming bootcamps or classes or degrees.

> Costs will increase. The current cost is substituted by investor funding. Sell at a loss to get people hooked on the product and then raise the price to make money, a "high-growth business model" as you say.

Inference is already profitable at current pricing. Most funding goes toward R&D for new model training, not inference.

Also, inference costs dropped over 280x between Nov 2022 and Oct 2024. Inference will continue to get cheaper as we develop more specialized hardware and efficient models.

This is not Uber, subsidizing the cost of human drivers. This is real tech, chips and servers and software. Costs fall over time, not rise. Innovation does not go backwards.

Re: Eight years of wanting, three months of building with AI

#323
post #55

Refreshing to see an honest and balanced take on AI coding. This is what real AI-assisted coding looks like once you get past the initial wow factor of having the AI write code that executes and does what you asked. This experience is familiar to every serious software engineer who has used AI code gen and then reviewed the output: > But when I reviewed the codebase in detail in late January, the downside was obvious…

I'll take the other side of this. Professional software engineers like many of us have a big blind spot when it comes to AI coding, and that's a fixation on code quality. It makes sense to focus on code quality. We're not wrong. After all, we've spent our entire careers in the code. Bad code quality slows us down and makes things slow/insecure/unreliable/etc for end users. However, code quality is becoming less and l…

Controversy much :-)

I completely agree. Just going through the beginner & hobbyist forums, the change from "can you help me with code to do X" to "I used ChatGPT/Claude/Copilot to write code to do X" happened with absolutely startling speed, and it's not slowing down. There was clearly a pent-up demand here that wasn't being met otherwise.

People are using AI to get code written. They have no idea what code quality is and only care that what they built works.

AFAICT, every time technology has allowed non-technical people to do more, it's opened up new opportunities for programmers. I don't expect this to be any different, I just want to know where the opportunities are.

Re: Eight years of wanting, three months of building with AI

#324

Earlier quoted context omitted.

> LLM-assisted coding is most successful in codebases with attributes strongly associated with high code quality: predictable patterns, well-named variables, use of a type system, no global mutable state, very low mutability in general, etc. That's all very true, but what you're missing is that the proportion of codebases that need this is shrinking relative to the total number of codebases. There's an incredible pro…

Just like everyone has a 3D printer at home? People want convenience, not a way to generate an application that creates convenience.

And perhaps they'll get that convenience from an application that they don't even know came into existence because they asked their agent to do something.

Re: Eight years of wanting, three months of building with AI

#325
post #222

Earlier quoted context omitted.

Your position makes an immense amount of sense for your described situation. As I said, I'm retired, and so I've never had to clean up AI slop at $DAYJOB. Since the whole AI thing would be a learning experience for me, it would include trying to toilet train the AI itself, as others have intimated can be done in some cases, rather than dealing with a bunch of already-checked-into-the-repo-slop. And that may be a losi…

The biggest thing that has changed in my experience (at least in a professions setting) is now that people have AI agents they don’t really have any motivation to improve. If you tell them something that needs to be changed they just reprompt the agent until it’s good enough - but the most sinister thing is they keep making the same mistakes over and over again. There is no growth, no shared understanding that dissem…

That does sound like an intractable problem.

My problem, if and when I get started, would be tangential to this. It is clear that communication with LLMs is changing so rapidly that there may not be any universal long-lived lessons to be learned from optimizing your interactions with a particular model.

I know that one-shotting things is probably not best, but determining how far to take it and when to cut over and finish it myself is something that I want to learn, but perhaps not too well.

My skills are an eclectic mix of high- and low- level. I know exactly what, for example, a frequency analyzer can do for me, but controlling the $400K frequency analyzer is often best left to the guy who lives and breathes it.

Likewise, my debugging skills are exceptional, but I am not as proficient with any particular debugger as are people who live in the debugger daily because they write terrible code. My debugging skills are mostly predicated on a big part of your daily life -- reading code.

(To be fair, I have known a very few people who live in the debugger because they are dealing with intractable problems caused by other people, but those are the rarities. I, myself, used to live in the debugger a lot when I was writing graphics drivers for the mostly undocumented Windows 3.1.)

Which brings us to your reports and/or co-workers. These people have always existed. They pride themselves on and partly base their value on and derive their value from the tools they think they know inside-out.

In truth, they don't know the tools, but they are intimately familiar with the controls of the tool, like a child who knows how to make a smartphone do exactly what their parent needs it to do.

So, as long as it's a tool you need, but it's too painful for you to control directly, these people are useful. In your case, you already have cause to use the LLM directly on a regular basis, so, as you point out, the value of these people is diminishing and maybe already negative.

> why shouldn’t I replace them with AI?

You probably should. Or, at a minimum, if possible, you should restructure things so that the people who are doing things that you are already proficient at are doing them for someone else who isn't as proficient at the tools, and you can get out of that loop.

One reason I am not yet completely insane is that I realized about 40 years ago that the place I hated most being was inside someone else's debug loop. Because most people are objectively stupid, and this goes double for people who need you in that loop. So I always work to structure my responsibilities and work setup to avoid this. If I find a bug in an internal supplier's code, I create an MVCE and hand it over to them. If an internal customer claims to find a bug in my code and doesn't provide an MVCE, I figure out what they are attempting to do, create my own MVCE for their function, and either fix it if it was really my problem, or hand it back to them, and ask them to expand on it until it breaks and get back to me.

Reflecting on this, I realize that I am probably not too likely to succumb to interminable prompting loops, because that wouldn't feel much different to what I nave avoided most of my life. On the few occasions over the last four decades where being involved in someone else's debug loop was completely unavoidable, the most useful thing I brought to the table when they were out of ideas and ready to throw a lot of effort at trying random things was a series of questions like "What are you going to learn from that? What will your decision points be?"

And I'm not much of a gambler, so I won't be spending too many tokens hoping "the next time, for sure!"

Re: Eight years of wanting, three months of building with AI

#326

Refreshing to see an honest and balanced take on AI coding. This is what real AI-assisted coding looks like once you get past the initial wow factor of having the AI write code that executes and does what you asked. This experience is familiar to every serious software engineer who has used AI code gen and then reviewed the output: > But when I reviewed the codebase in detail in late January, the downside was obvious…

> This blog post shows the journey that anyone not in one of those two vocal minorities is going through right now: A realization that AI coding tools can be a large accelerator but you need to learn how to use them correctly in your workflow and you need to remain involved in the code. It’s not as clickbaity as the extreme takes that get posted all the time. It’s a little disappointing to read the part where they said hard work was still required. It is a realistic and balanced take on the state of AI coding, though.

I appreciate the balanced takes and also the notion that one can use these AI tools to build software with principled use.

However, what I am still failing to see is concrete evidence that this is all faster and cheaper than just a human learning and doing everything themself or with a small team. The cat is out of the bag, so to speak, but I think it's still correct to question these things. I am putting in a _lot_ of work to reach a principled status quo with these tools, and it is still quite unclear whether it's actually improvement versus just a side quest to wrangle tools that everyone else is abusing.

Re: Eight years of wanting, three months of building with AI

#327
post #99

Earlier quoted context omitted.

You can get better at something without understanding why, but you should be able to think about it and determine why fairly easily. This is something everyone who cares about improving in a skill does regularly - examine their improvement, the reasons behind it, and how to add to them. That’s the basis of self-driven learning.

This is an absurd statement. There are many complex undertakings in sport where even the very best get better with practice and can't tell you why. In fact, the ones who think they can tell you why are the one's to be most skeptical of. You are just making stuff up or regurgitating material from a pop science book.

[flagged]

Re: Eight years of wanting, three months of building with AI

#328

Earlier quoted context omitted.

Just like everyone has a 3D printer at home? People want convenience, not a way to generate an application that creates convenience.

And perhaps they'll get that convenience from an application that they don't even know came into existence because they asked their agent to do something.

What, in practice, is the difference between AGI and what you’re suggesting will exist in terms of agent automation?

Re: Eight years of wanting, three months of building with AI

#329

Earlier quoted context omitted.

This is where I get into much more speculative land, but I think people are underestimating the degree to which AI assistant apps are going to eat much of the traditional software industry. The same way smart phones ate so many individual tools, calculators, stop watches, iPods, etc. It takes a long time for humanity to adjust to a new technology. First, the technology needs to improve for years. Then it needs to be…

You are vastly overstating the capabilities of LLMs and the capacity and desire of non-technical individuals to use them to create applications.

What's even the point of vague replies like this that disagree with no real evidence, arguments, or examples?

Re: Eight years of wanting, three months of building with AI

#330
post #36

Earlier quoted context omitted.

+1 I’ve been driving Claude as my primary coding interface the last three months at my job. Other than a different domain, I feel like I could have written this exact article. The project I’m on started as a vibe-coded prototype that quickly got promoted to a production service we sell. I’ve had to build the mental model after the fact, while refactoring and ripping out large chunks of nonsense or dead code. But the…

I’ve found that LLMs will frequently do extremely silly things that no person would do to make typescript code pass the typechecker.

I've noticed this too, but not necessarily type checkers, but more with linters. And can't really figure out if there's even a way to solve it.

If you set up restrictive linters and don't explicitly prohibit agents from adding inline allows, most LOC will be allow comments.

Based on this learning, I've decided to prohibit any inline allows. And then agents started doing very questionable things to satisfy clippy.

Recent example:

- Claude set up a test support module so that it could reuse things. Since this was not used in all tests, rust complained about dead_code. Instead of making it work, claude decided to remove test support module and just... blow up each test.

If you enable thinking summaries, you'll always see agent saying something like: "I need to be pragmatic", which is the right choice 50% of the time.

Post reply on HN