Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building better AI tools

hazelweakly.me

Post reply on HN
Print view View original

111–120 of 189 posts

Re: Building better AI tools

#111

Post by phillipcarter »

tptacek wrote: post #54

This is a confusing piece. A lot of it would make sense if Weakly was talking about a coding agent (a particular flavor of agent that worked more like how antirez just said he prefers coding with AI in 2025 --- more manual, more advisory, less do-ing). But she's not: she's talking about agents that assist in investigating and resolving operations incidents. The fulcrum of Weakly's argument is that agents should stay…

Lost in a bit of the discourse around anomaly detection and incident management is that not all problems are equal. Many of them actually are automatable to some extent. I think the issue is understanding when something is sufficiently offloadable to some cognitive processor vs. when you really do need a human engineer involved. To your point, yes, they are better at detecting patterns at scale … until they’re not. Or knowing if a pattern is meaningful. Of course not all humans can fill these gaps either.

Re: Building better AI tools

#112

Post by cess11 »

biophysboy wrote: post #48
taylorallred wrote: post #14

One thing that has always worried me about AI coding is the loss of practice. To me, writing the code by hand (including the boilerplate and things I've done hundreds of times) is the equivalent of Mr. Miyagi's paint-the-fence. Each iteration gets it deeper into your brain and having these patterns as a part of you makes you much more effective at making higher-level design decisions.

A retort you often hear is that prior technologies, like writing or the printing press, may have stunted our calligraphy or rhetorical skills, but they did not stunt our capacity to think. If anything, they magnified it! Basically, the whole Steve Jobs' bicycle-for-the-mind idea. My issue with applying this reasoning to AI is that prior technologies addressed bottlenecks in distribution, whereas this more directly at…

Bad examples. Computer keyboards killed handwriting, the Internet killed rhetoric.

Re: Building better AI tools

#113

Post by emehex »

biophysboy wrote: post #48
taylorallred wrote: post #14

One thing that has always worried me about AI coding is the loss of practice. To me, writing the code by hand (including the boilerplate and things I've done hundreds of times) is the equivalent of Mr. Miyagi's paint-the-fence. Each iteration gets it deeper into your brain and having these patterns as a part of you makes you much more effective at making higher-level design decisions.

A retort you often hear is that prior technologies, like writing or the printing press, may have stunted our calligraphy or rhetorical skills, but they did not stunt our capacity to think. If anything, they magnified it! Basically, the whole Steve Jobs' bicycle-for-the-mind idea. My issue with applying this reasoning to AI is that prior technologies addressed bottlenecks in distribution, whereas this more directly at…

Counter-counter-point: handwriting > typing for remembering things (https://www.glamour.com/story/typing-memory)

Re: Building better AI tools

#114

Post by mikaylamaki »

hazelweakly wrote: post #108
mikaylamaki wrote: post #99

The code gen example given later sounds an awful lot like what AWS built with Kiro[1] and it's spec feature. This article is kinda like the theory behind the practice in that IDE. I wish the tools described existed instead of all these magic wands [1] https://kiro.dev/blog/introducing-kiro/

Indeed it is! In fact I actually had something like Kiro almost perfectly in mind when I wrote the article (but didn’t know AWS was working on it at the time). I was very happy to see AWS release Kiro. It was quite validating to me seeing them release it and follow up with discussions on how this methodology of integrating AI with software development was effective for them

Yeah! It's a good sign that this idea is in the air and the industry is moving towards it. Excited to get rid of all the magic wands everywhere :D

Re: Building better AI tools

#115

Post by staticshock »

The spirit of this post is great. There are real lessons here that the industry will struggle to absorb until we reach the next stage of the AI hype cycle (the trough of disillusionment.)

However, I could not help but get caught up on this totally bonkers statement, which detracted from the point of the article:

> Also, innovation and problem solving? Basically the same thing. If you get good at problem solving, propagating learning, and integrating that learning into the collective knowledge of the group, then the infamous Innovator’s Dilemma disappears.

This is a fundamental misunderstanding of what the innovator's dilemma is about. It's not about the ability to be creative and solve problems, it is about organizational incentives. Over time, an incumbent player can become increasingly disincentivized from undercutting mature revenue streams. They struggle to diversify away from large, established, possibly dying markets in favor of smaller, unproven ones. This happens due to a defensive posture.

To quote Upton Sinclair, "it is difficult to get a man to understand something when his salary depends upon his not understanding it." There are lots of examples of this in the wild. One famous one that comes to mind is AT&T Bell Labs' invention of magnetic recording & answering machines that AT&T shelved for decades because they worried that if people had answering machines, they wouldn't need to call each other quite so often. That is, they successfully invented lots of things, but the parent organization sat on those inventions as long as humanly possible.

Re: Building better AI tools

#116

Post by tptacek »

cmiles74 wrote: post #110
tptacek wrote: post #74

Earlier quoted context omitted.

Nobody disputes this. Weakly posits a bright line between agents suggesting active steps and agents actually performing active steps. The problem is that during incident investigations, some active steps make a lot of sense for agents to perform, and others don't; the line isn't where she seems to claim it is.

Understood. To your example about the logs, my concern would be be that the AI chooses the wrong thing to focus on and people decide there’s nothing of interest in the logs, thus overlooking a vital clue.

You wouldn't anticipate using AI tools to one-shot complex incidents, just to rapidly surface competing hypotheses.

Re: Building better AI tools

#117

Post by tptacek »

JoshTriplett wrote: post #100
tptacek wrote: post #93

Earlier quoted context omitted.

You can simply disagree with me and we can hash it out. The "early career" thing is something Weakly herself has called out. I disagree with you that incident responders learn best by e.g. groveling through OpenSearch clusters themselves. In fact, I think the opposite thing is true: LLM agents do interesting things that humans don't think to do, and also can put more hypotheses on the table for incident responders to…

The only mention I see of early-career coming up in the article is "matches how I would teach an early career engineer the process of managing an incident". That isn't a claim that only early career engineers learn this way or benefit from working in this style. Your comment implied that the primary people who might want to work in the way proposed in this article are those early in their career. I would, indeed, dis…

Previous writing, Josh, and I'm done now litigating whether I wrote the "early career" thing in bad faith and expect you to be too.

I don't see you materially disagreeing with me about anything. I read Weakly to be saying that AI incident response tools --- the main focus of her piece --- should operate with hands tied behind their back, delegating nondestructive active investigation steps back to human hands in order to create opportunities for learning. I think that's a bad line to draw. In fact, I think it's unlikely to help people learn --- seeing the results of investigative steps all lined up next to each other and synthesized is a powerful way to learn those techniques for yourself.

Re: Building better AI tools

#118

Post by saltcured »

arscan wrote: post #79
biophysboy wrote: post #48

Earlier quoted context omitted.

A retort you often hear is that prior technologies, like writing or the printing press, may have stunted our calligraphy or rhetorical skills, but they did not stunt our capacity to think. If anything, they magnified it! Basically, the whole Steve Jobs' bicycle-for-the-mind idea. My issue with applying this reasoning to AI is that prior technologies addressed bottlenecks in distribution, whereas this more directly at…

I’ve been thinking of LLMs a bit like a credit-card-for-the-mind, it reduces friction to accessing and enabling your own expertise. But if you don’t have that expertise already, be careful, eventually it’ll catch up to you and a big bill will be due.

I think this phrase is beautiful

assuming you were referencing "bicycle for the mind"

Re: Building better AI tools

#119

Post by vlod »

cwnyth wrote: post #76
taylorallred wrote: post #14

One thing that has always worried me about AI coding is the loss of practice. To me, writing the code by hand (including the boilerplate and things I've done hundreds of times) is the equivalent of Mr. Miyagi's paint-the-fence. Each iteration gets it deeper into your brain and having these patterns as a part of you makes you much more effective at making higher-level design decisions.

My LLM-generated code has so many bugs in it, that I end up knowing it better since I have to spend more time debugging/figuring out small errors. This might even be better: you learn something more thoroughly when you not only practice the right answers, but know how to fix the wrong answers.

For me the process of figuring out wtf I need to do and how I'm going to do it is my learning process.

For beginners my I think this is a very important step in learning how to break down problems (into smaller components) and iterating.

Re: Building better AI tools

#120

Post by danphilibin »

donsupreme wrote: post #17
taylorallred wrote: post #14

One thing that has always worried me about AI coding is the loss of practice. To me, writing the code by hand (including the boilerplate and things I've done hundreds of times) is the equivalent of Mr. Miyagi's paint-the-fence. Each iteration gets it deeper into your brain and having these patterns as a part of you makes you much more effective at making higher-level design decisions.

Many analog to this IRL: 1) I can't remember the last time I write something meaningfully long with an actual pen/pencil. My handwriting is beyond horrible. 2) I can't no longer find my way driving without a GPS. Reading a map? lol

On 2) I've combatted this since long before AI by playing a game of "get home without using GPS" whenever I drive somewhere. I've definitely maintained a very good directional sense by doing this - it forces you to think about main roads, landmarks, and cardinal directions.
Post reply on HN