Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My AI Adoption Journey

mitchellh.com

Post reply on HN
Print view View original

221–230 of 420 posts

Re: My AI Adoption Journey

#221

Post by zx8080 »

grey-area wrote: post #172
LiamPowell wrote: post #126

Earlier quoted context omitted.

> Compilers will produce working output given working input literally 100% of my time in my career. In my experience this isn't true. People just assume their code is wrong and mess with it until they inadvertently do something that works around the bug. I've personally reported 17 bugs in GCC over the last 2 years and there are currently 1241 open wrong-code bugs. Here's an example of a simple to understand bug (not…

These are still deterministic bugs, which is the point the OP was making. They can be found and solved once. Most of those bugs are simply not that important, so they never get attention. LLMS on the other hand are non-deterministic and unpredictable and fuzzy by design . That makes them not ideal when trying to produce output which is provably correct - sure you can output and then laboriously check the output - som…

> It's a little like using Bitcoin to replace currencies [...]

At least, Bitcoin transactions are deterministic.

Not many would want to use a AI currency (mostly works; always shows "Oh, you are 100% right" after losing one's money).

Re: My AI Adoption Journey

#222

Post by LiamPowell »

rhubarbtree wrote: post #178
LiamPowell wrote: post #126

Earlier quoted context omitted.

> Compilers will produce working output given working input literally 100% of my time in my career. In my experience this isn't true. People just assume their code is wrong and mess with it until they inadvertently do something that works around the bug. I've personally reported 17 bugs in GCC over the last 2 years and there are currently 1241 open wrong-code bugs. Here's an example of a simple to understand bug (not…

This argument is disingenuous and distracts rather than addresses the point. Yes, it is possible for a compiler to have a bug. No, that is I’m mo way analogous to AI producing buggy code. I’ve experienced maybe two compiler bugs in my twenty year career. I have experienced countless AI mistakes - hundreds? Thousands? Already. These are not the same and it has the whiff of sales patter trying to address objections. Pl…

I'm not arguing that LLMs are at a point today where we can blindly trust their outputs in most applications, I just don't think that 100% correct output is necessarily a requirement for that. What it needs to be is correct often enough that the cost of reviewing the output far outweighs the average cost of any errors in the output, just like with a compiler.

This even applies to human written code and human mistakes, as the expected cost of errors goes up we spend more time on having multiple people review the code and we worry more about carefully designing tests.

Re: My AI Adoption Journey

#223

Post by rldjbpin »

not quite as technically rich as i came to expect from previous posts from op, but very insightful regardless.

not ashamed to say that i am between steps 2 and 3 in my personal workflow.

>Adopting a tool feels like work, and I do not want to put in the effort

all the different approaches floating online feel ephemeral to me. this, just like for different tools for the op, seem like a chore to adopt. i like the fomo mongering from the community does not help here, but in the end it is a matter of personal discovery to stick with what works for you.

Re: My AI Adoption Journey

#224

Post by miyuru »

EastLondonCoder wrote: post #23

This matches my experience, especially "don’t draw the owl" and the harness-engineering idea. The failure mode I kept hitting wasn’t just "it makes mistakes", it was drift: it can stay locally plausible while slowly walking away from the real constraints of the repo. The output still sounds confident, so you don’t notice until you run into reality (tests, runtime behaviour, perf, ops, UX). What ended up working for m…

This is what I experienced as well.

these are some ticks I use now.

1. Write a generic prompts about the project and software versions and keep it in the folder. (I think this getting pushed as SKIILS.md now)

2. In the prompt add instructions to add comments on changes, since our main job is to validate and fix any issues, it makes it easier.

3. Find the best model for the specific workflow. For example, these days I find that Gemini Pro is good for HTML UI stuff, while Claude Sonnet is good for python code. (This is why subagents are getting popluar)

Re: My AI Adoption Journey

#225

Post by lins1909 »

Robin_Message wrote: post #209
svilen_dobrev wrote: post #208

Earlier quoted context omitted.

let me ask a stupid/still-ignorant question - about repeatability. If one asks this generator/assistant same request/thing, within same initial contexts, 10 times, would it generate same result ? in different sessions and all that. because.. if not, then it's for once-off things only..

If I asked you for the same thing 10 times, wiping your memory each time, would you generate the same result? And why does it matter anyway? I'd the code passes the tests and you like the look of it, it's good. It doesn't need to be existentially complicated.

A pretty bad comparison. If I gave you the correct answer once, it's unlikely that I'll give you a wrong answer the next time. Also, aren't computers supposed to be more reliable than us? If I'm going to use a tool that behaves just like humans, why not just use my brain instead?

Re: My AI Adoption Journey

#226

Post by hollowturtle »

I don't understand how Agents make you feel productive. Single/Multiple agents reading specs, specs often produced with agents itself and iterated over time with human in the loop, a lot of reviewing of giant gibberish specs. Never had a clear spec in my life. Then all the dancing for this apperantly new paradigm, of not reviewing code but verifying behaviour, and so many other things. All of this to me is a total UNproductive mess. I use Cursor autocomplete from day one till to this day, I was super productive before LLMs, I'm more productive now, I'm capable, I have experience, product is hard to maintain but customers are happy, management is happy. So I can't really relate anymore to many of the programmers out there, that's sad, I can count on my hands devs that I can talk to that have hard skills and know-how to share instead of astroturfing about AI Agents

Re: My AI Adoption Journey

#227

Post by bonesss »

andai wrote: post #154
atomicnumber3 wrote: post #116

Earlier quoted context omitted.

"When was the last time you reviewed the machine code produced by a compiler?" Compilers will produce working output given working input literally 100% of my time in my career. I've never personally found a compiler bug. Meanwhile AI can't be trusted to give me a recipe for potato soup. That is to say, I would under no circumstances blindly follow the output of an LLM I asked to make soup. While I have, every day of…

This is obviously besides the point but I did blindly follow a wiener schnitzel recipe ChatGPT made me and cooked for a whole crew. It turned out great. I think I got lucky though, the next day I absolutely massacred the pancakes.

Recent experiments with LLM recipes (ChatGPT): missed salt in a recipe to make rice, then flubbed whether that type of rice was recommended to be washed in the recipe it was supposedly summarizing (and lied about it, too)…

Probabilistic generation will be weighted towards the means in the training data. Do I want my code looking like most code most of the time in a world full of Node.js and PHP? Am I better served by rapid delivery from a non-learning algorithm that requires eternal vigilance and critical re-evaluation or with slower delivery with a single review filtered through an meatspace actor who will build out trustable modules in a linear fashion with known failure modes already addressed by process (ie TDD, specs, integration & acceptance tests)?

I’m using LLMs a lot, but can’t shake the feeling that the TCO and total time shakes out worse than it feels as you go.

Re: My AI Adoption Journey

#228

Post by glitchcrab »

underdeserver wrote: post #33

> At a bare minimum, the agent must have the ability to: read files, execute programs, and make HTTP requests. That's one very short step removed from Simon Willison's lethal trifecta.

This is why I won't run Claude without additional sandboxing. I'm currently using (and quite pleased with) https://github.com/strongdm/leash

Re: My AI Adoption Journey

#229

Post by glitchcrab »

tppts wrote: post #133

So does everyone just run with giving full permissions on Claude code these days? It seems like I’m constantly coming back to CC to validate that it’s not running some bash that’s going to nuke my system. I would love to be able to fully step away but it feels like I can’t.

I sandbox everything inside https://github.com/strongdm/leash

That way the blast radius is vastly reduced.

Re: My AI Adoption Journey

#230

Post by lawgimenez »

datsci_est_2015 wrote: post #129
AloysB wrote: post #112

Earlier quoted context omitted.

Give it a read, he mentions briefly how he uses for PR triages and resolving GH issues. He doesn't go in details, but there is a bit: > Issue and PR triage/review. Agents are good at using gh (GitHub CLI), so I manually scripted a quick way to spin up a bunch in parallel to triage issues. I would NOT allow agents to respond, I just wanted reports the next day to try to guide me towards high value or low effort tasks.…

Okay I think this somewhat answers my question. Is this individual a solo developer? “Triaging GitHub issues” sounds a bit like open source solo developer. Guess I’m just desperate for an article about how organizations are actually speeding up development using agentic AI. Like very practical articles about how existing development processes have been adjusted to facilitate agentic AI. I remain unconvinced that agen…

Can't believe you don't know who the author is my man.
Post reply on HN