Live data from Hacker News

My AI Adoption Journey

mitchellh.com

301–310 of 420 posts

Re: My AI Adoption Journey

#301
post #221

Earlier quoted context omitted.

These are still deterministic bugs, which is the point the OP was making. They can be found and solved once. Most of those bugs are simply not that important, so they never get attention. LLMS on the other hand are non-deterministic and unpredictable and fuzzy by design . That makes them not ideal when trying to produce output which is provably correct - sure you can output and then laboriously check the output - som…

> It's a little like using Bitcoin to replace currencies [...] At least, Bitcoin transactions are deterministic. Not many would want to use a AI currency (mostly works; always shows "Oh, you are 100% right" after losing one's money).

Sure bitcoin is at least deterministic, but IMO (an that of many in the finance industry) it's solving entirely the wrong problem - in practice people want trust and identity in transactions much more than they want distributed and trustless.

In a similar way LLMs seem to me to be solving the wrong problem - an elegant and interesting solution, but a solution to the wrong problem (how can I fool humans into thinking the bot is generally intelligent), rather than the right problem (how can I create a general intelligence with knowledge of the world). It's not clear to me we can jump from the first to the second.

Re: My AI Adoption Journey

#302

Earlier quoted context omitted.

If you've got a plan for the plan, what else could you possibly need!

You joke, but the more I iterate on a plan before any code, the more successful the first pass is. 1) Tell claude my idea with as much as I know, ask it to ask me questions. This could go on for a few rounds. (Opus) 2) Run a validate skill on the plan, reviewer with a different prompt (Opus) 3) codex reviews the plan, always finds a few small items after the above 2. 4) claude opus implements in 1 shot, usually 99% a…

I do something very similar. I have an "outside expert" script I tell my agent to use as the reviewer. It only bothers me when neither it OR the expert can figure out what the heck it is I actually wanted.

In my case I have Gemini CLI, so I tell Gemini to use the little python script called gatekeeper.py to validate it's plan before each phase with Qwen, Kimi, or (if nothing else is getting good results) ChatGPT 5.2 Thinking. Qwen & Kimi are via fireworks.ai so it's much cheaper than ChatGPT. The agent is not allowed to start work until one of the "experts" approves it via gatekeeper. Similarly it can't mark a phase as complete until the gatekeeper approves the code as bug free and up to standards and passes all unit tests & linting.

Lately Kimi is good enough, but when it's really stuck it will sometimes bother ChatGPT. Seldom does it get all the way to the bottom of the pile and need my input. Usually it's when my instructions turned out to be vague.

I also have it use those larger thinking models for "expert consultation" when it's spent more than 100 turns on any problem and hasn't made progress by it's own estimation.

Re: My AI Adoption Journey

#303

This is such a lovely balanced thoughtful refreshingly hype-free post to read. 2025 really was the year when things shifted and many first-rate developers (often previously AI skeptics, as Mitchell was) found the tools had actually got good enough that they could incorporate AI agents into their workflows. It's a shame that AI coding tools have become such a polarizing issue among developers. I understand the reasons…

> It's a shame that AI coding tools have become such a polarizing issue among developers. Frankly I'm so tired of the usual "I don't find myself more productive", "It writes soup". Especially when some of the best software developers (and engineers) find many utility in those tools, there should be some doubt growing in that crowd. I have come to the conclusion that software developers , those only focusing on the cr…

I've spent 2+ decades producing software across a number of domains and orgs and can fully agree that _disciplined use_ of LLM systems can significantly boost productivity, but the rules and guidance around their use within our industry writ large are still in flux and causing as many problems as they're solving today.

As the most senior IC within my org, since the advent of (enforced) LLM adoption my code contribution/output has stalled as my focus has shifted to the reactionary work of sifting through the AI generated chaff following post mortems of projects that should have never have shipped in the first place. On a good day I end up rejecting several PRs that most certainly would have taken down our critical systems in production due to poor vetting and architectural flaws, and on the worst I'm in full on fire fighting mode to "fix" the same issues already taking down production (already too late.)

These are not inherent technical problems in LLMs, these are organizational/processes problems induced by AI pushers promising 10x output without the necessary 10x requirements gathering and validation efforts that come with that. "Everyone with GenAI access is now a 10x SDE" is the expectation, when the reality is much more nuanced.

The result I see today is massive incoming changesets that no one can properly vet given the new shortened delivery timelines and reduced human resourcing given to projects. We get test suite coverage inflation where "all tests pass" but undermine core businesses requirements and no one is being given the time or resources to properly confirm the business requirements are actually being met. Shit hits the fan, repeat ad nauseum. The focus within our industry needs to shift to education on the proper application and use of these tools, or we'll inevitably crash into the next AI winter; an increasingly likely future that would have been totally avoidable if everyone drinking the Koolaid stopped to observe what is actually happening.

As you implied, code is cheap and most code is "throwaway" given even modest time horizons, but all new code comes with hidden costs not readily apparent to all the stakeholders attempting to create a new normal with GenAI. As you correctly point out, the biggest problems within our industry aren't strictly technical ones, they're interpersonal, communication and domain expertise problems, and AI use is simply exacerbating those issues. Maybe all the orgs "doing it wrong" (of which there are MANY) simply fail and the ones with actual engineering discipline "make it," but it'll be a reckoning we should not wish for.

I have heard from a number of different industry players and they see the same patterns. Just look at the average linked in post about AI adoption to confirm. Maybe you observe different patterns and the issues aren't as systemic as I fear. I honestly hope so.

Your implication that seniors like myself are "insecure about our jobs" is somewhat ironically correct, but not for the reasons you think.

Re: My AI Adoption Journey

#304

Earlier quoted context omitted.

The comment by user senko [1] links to a post from this same author with an example for a specific coding session that costs $15.98 for 8 hours of work. The example in this post talks about leaving agents running overnight, in which case I'd guess "twice that amount" would be a reasonable approximation. Or if we assume that the OP can only do 4 hours per sitting (mentioned in the other post) and 8 hours of overnight…

>$15.98 * 1.5 * 20 = $497,40 a month Are people seriously dropping hundreds of dollars a month on these products to get their work done?

If you make 10k/mo -- which is not that much!, $500 is 5% of revenue. All else held equal, if that helps you go 20% faster, it's an absolute no brainer.

The question is.. does it actually help you do that, or do you go 0% faster? Or 5% slower?

Inquiring minds want to know.

Re: My AI Adoption Journey

#305
How much electricity (and associated materials like water) must this use?

It makes me profoundly sad to think of the huge number of AI agents running endlessly to produce vibe-coded slop. The environmental impact must be massive.

Re: My AI Adoption Journey

#306

Earlier quoted context omitted.

If natural language is used to specify work to the LLM, how can the output ever be trusted? You'll always need to make sure the program does what you want, rather than what you said.

Just create a very specific and very detailed prompt that is so specific that it starts including instructions and you came up with the most expensive programming language.

It's not great that it's the most expensive (by far), but it's also by far the most expressive programming language.

Re: My AI Adoption Journey

#307

Earlier quoted context omitted.

> Consequences for me messing up are far less severe than the consequences for a PE messing up. if I approve a bad release that leads to an egregious failure, for me it's a prison sentence and unlimited fines in addition to being struck off and banned from the industry > That's a whole different ball game. if you say so

> if I approve a bad release that leads to an egregious failure, for me it's a prison sentence and unlimited fines Again, I'm in 510k land. The same applies to myself. No one's gonna allow me to irradiate a patient with a 10x dose because my bass ackwards software messed up scientific notation. To remove the wrong kidney because I can't convert orthonormal basis vectors correctly. But the fact remains that no one wou…

pray tell, how I can practice my craft from prison

Re: My AI Adoption Journey

#308

Earlier quoted context omitted.

That is not the issue, any potato soup recipe would be fine, the issue is that it might fetch values from different recipes and give you an abomination.

This exactly, I cook as passion, and LLMs just routinely very clearly (weighted) "average" together different recipes to produce, in the worst case, disgusting monstrosities, or, in the best case, just a near-replica of some established site's recipe.

> ... some established site's recipe.

At least with the LLM, you don't have to wade through paragraph after paragraph of "I remember playing in the back yard as a child, I would get hungry..."

In fact LLMs write better and more interesting prose than the average recipe site.

Re: My AI Adoption Journey

#309

Earlier quoted context omitted.

>$15.98 * 1.5 * 20 = $497,40 a month Are people seriously dropping hundreds of dollars a month on these products to get their work done?

If you make 10k/mo -- which is not that much!, $500 is 5% of revenue. All else held equal, if that helps you go 20% faster, it's an absolute no brainer. The question is.. does it actually help you do that, or do you go 0% faster? Or 5% slower? Inquiring minds want to know.

>If you make 10k/mo -- which is not that much!,

This is the sort of statement that immediately tells me this forum is disconnected from the real world. ~80% of full time workers in the US make less than $10k a month before tax.

Source: https://dqydj.com/income-percentile-calculator/

Re: My AI Adoption Journey

#310
post #160

Earlier quoted context omitted.

Maybe. But it's been 3 years and it still isn't good enough to actually trust. That doesn't raise confidence that it will ever get there.

You need to put this revolution in scale with other revolutions. How long did it take for horses to be super-seeded by cars? How long did powertool take to become the norm for tradesmen? This has gone unbelievably fast.

*superseded

It comes from the Latin "supersedēre", which taken literally, means "sit on top of". "Super" = above, on top of. "Sedēre" = to sit.

"Super" is already familiar to English speakers. "Sedēre" is the root of words like sedentary, sedan, sedate, reside, and preside.

The more metaphorical meaning of "supersede" as "replace" developed over time and across languages, but the literal meaning is already fairly close.

Post reply on HN