Live data from Hacker News

Developers are attached to tools because tools encode trust

stackoverflow.blog

21–30 of 148 posts

Re: Developers are attached to tools because tools encode trust

#21

[flagged]

Yes that is what happened to SO. They now get practically zero questions, zero answers, zero page views, and zero ad revenue. It is their own fault because they froze everyone out of the site and then once LLMs became an alternative, everyone started asking their questions to LLMs. They are now trying to somehow pivot to AI to make revenue again, starting with Stack Overflow for Agents, and now with wordy vacuous blog posts to show off how AI they are.

Re: Developers are attached to tools because tools encode trust

#22
post #15
post #12

I feel like these abstractions like "CI might not work well in the era of agentic tooling" are fine for thought-leadership posts but there's so much hands-on work to be done. The last word on AI computer use shouldn't be bash utils that were already feature-complete before MJ recorded Thriller. There is some movement in this direction--there is a new 'gh' subcommand called repo read-file for example, that lets agents…

MCP is the answer to not using bash, right? Bash is a great control surface anyway for LLMs as it is wordy and powerful.

Problem is that bash is too sharp to handle to smart and gullible clankers without a sandbox - That I think everyone should be using anyways, for everything, even things not related to clankers - Android and Qubes are right. The app/vm, and whatever it tries, should not be considered trusted by default.

Re: Developers are attached to tools because tools encode trust

#23
post #18

[flagged]

> Apparently you just deserve a life of pain. Don't we all? (I can see that you are starting to get downvoted as well. Spread the love.) Stack overflow was interesting. Its design was the only thing like it at the time, and made it a Schelling point for programming knowledge distribution, but also a welcoming environment for the programming equivalent of grammar nazis. Some of those programming nazis, of course, had…

You can't just call everyone you don't like a Nazi.

Re: Developers are attached to tools because tools encode trust

#24
post #15
post #12

I feel like these abstractions like "CI might not work well in the era of agentic tooling" are fine for thought-leadership posts but there's so much hands-on work to be done. The last word on AI computer use shouldn't be bash utils that were already feature-complete before MJ recorded Thriller. There is some movement in this direction--there is a new 'gh' subcommand called repo read-file for example, that lets agents…

MCP is the answer to not using bash, right? Bash is a great control surface anyway for LLMs as it is wordy and powerful.

Yeah as far as what I'm talking about bash CLI vs MCP doesn't matter (there could be a file_sample MCP tool)---I'm saying that by default Windows, Linux etc don't have this venetianblinds affordance of seeing equally spaced samples. It's a trivial algo but it's very handy these days cause LLMs can't really just be like 'okay I'm gonna open this file at random and scroll around'--the file itself is an unknown blob (JSON data, Python, Typescript, a log file etc) without coordinates. So they fall back to thinking they've gotten a good sense of the file from head/grep or they write ad-hoc Python to manipulate the file.

Another venetianblinds survey, of the Paul Graham 'What I Worked On' article that's used in many LlamaIndex examples:

npx github:firasd/venetianblinds pgworkedon.txt

  --- sample 1/20 char 0 line 1 col 1 range 0:60
  Before college the two main things I worked on, outside of s

  --- sample 2/20 char 3946 line 19 col 237 range 3886:4006
  d an intelligent computer called Mike, and a PBS documentary that showed Terry Winograd using SHRDLU. I haven't tried re

Re: Developers are attached to tools because tools encode trust

#25
post #2

I have to admit, I asked ChatGPT to do a TLDR summary because I found the writing meandered quite a bit. I think the overall point is sound: > "Developers become attached to tools like Vim, Emacs, or an IDE because years of experience make those tools predictable extensions of their thinking. The attachment is less about features and more about accumulated trust, muscle memory, and a workflow built around known bound…

If you are spending 40 hours fixing everything that was broken, the question I have is does your AI tools have the necessary context to be successful and not result in a lot of broken items? Also, is there ways for AI to help prevent the loss of deep understanding of your code base without you having to know every line of code deeply?

I was a bit hazy on numbers because I don't scientifically record them, but I guess what I'll say is the fixing and verifying takes a lot longer than writing the initial code. This was true before LLMs, but when writing code by hand I had the context of the code in my head, so potential issues, blast radius, etc. was a lot more obvious. By definition most of the things that break are things that are not trivially testable. Unfortunately, it's not as easy as saying "Claude, the thing broke, plz fix"; I've had to spend a lot of time recently helping it with context from debuggers, or just debugging myself manually, adding log statements, etc.

Am I giving the agent enough context? Well, I'm giving it as much as I can. Each submodule has an AGENTS.md, I have the agents add gotchas and instructions for some feature work when I discover where an agent went wrong, the codebase has a lot of comments along the lines of "if you edit this section, you need to also edit XYZ", and I lean on the type system as much as I can to make wrong-code not compile. It has access to playwright for driving the UI if it wants to. (Weirdly, I've found that Claude is really inconsistent about using these tools -- even though the instructions make it clear that it's allowed and encouraged. I think if your workflow differs from the models training and thusly you have to tell it so in AGENTS.md/CLAUDE.md, then it's very inconsistent about following those instructions. For instance, I don't want Claude to commit and I don't want it to sign commit messages, and it still does that all the time even though it's my like #1 directive of "don't mess with my git history")

There are some things though that are very hard for it to test. I'm exporting essentially a programming language to three game engine runtimes. They all have automated tests, but, I think people that have worked in video games know that games are very hard to automate testing on. This isn't really the fault of the agent I would say, just the nature of the problem, but it is worth noting.

I guess this is a long winded way of saying, even with LLMs tech debt is a thing you have to manage, and I think managing tech debt becomes even more important when you're dealing with LLMs, not less important.

Re: Developers are attached to tools because tools encode trust

#26
I'm not sure the difference changes the conclusion but I think projects demanded certain workflows and resultant processes, not the other way around. That's why Jetbrains has so many workstream specific IDEs that sold very well. The processes didn't go away but a lot of us changed our IDE surface. Those processes still need to exist, largely, but the way in which we invoke them is moving and changing. To some degree, the processes are also changing because other factors are changing outside of the tooling.

For example, I use Codex and Claude Code by default, but when I need to look at the API surface, read tests, etc I have those tools setup to open Zed. Zed is also rapidly evolving in the other direction, where it's closer to the tools that are opening it. It won't be long, I think, until I can continue my prompt from inside Zed.

Re: Developers are attached to tools because tools encode trust

#27
post #8

Earlier quoted context omitted.

If you are spending 40 hours fixing everything that was broken, the question I have is does your AI tools have the necessary context to be successful and not result in a lot of broken items? Also, is there ways for AI to help prevent the loss of deep understanding of your code base without you having to know every line of code deeply?

"40 hours" in his context here is actually a work day, so 7-8hrs. He says "40 hours" because he feels like he's managed to do 40 hours worth of work in this time, but then has to spend another "40 hours" (actually: 1 day) just going around kicking tyres. Obviously the implication is that it's a net gain of some kind, but he's unsure if he caught everything. (sorry to reiterate the GP, but I feel like you missed the i…

Sorry, my original phrasing was confusing which you should not be downvoted for. I've edited my original comment to clarify what I meant (hopefully).

Re: Developers are attached to tools because tools encode trust

#28
post #9
post #2

I have to admit, I asked ChatGPT to do a TLDR summary because I found the writing meandered quite a bit. I think the overall point is sound: > "Developers become attached to tools like Vim, Emacs, or an IDE because years of experience make those tools predictable extensions of their thinking. The attachment is less about features and more about accumulated trust, muscle memory, and a workflow built around known bound…

I was surprised to see you say you have automated tests. To me this makes most of the difference, but you have you actually have a good test suite, like one that actually proves the code does what you want it to do. Unit tests, property tests, e2e tests. The other part that makes all difference, is you have to be all up in the model's business about architecture. Pick something that wants to testable and isolation fr…

I think testing is really important (even without LLMs). Currently I have 2247 unit tests across 132 files in a ~150K LOC codebase (test code included in that count). It takes about 50s to run. There could be more, but it's not nothing. There is playwright for it to test drive the UI, although if I'm being perfectly honest even before LLMs I thought that kind of test tends to be brittle and annoying to write (I guess I don't have to write them anymore, but they are still brittle). I'm honestly trying to give it as much structure as I possibly can -- I'm not trying to setup the agent to fail so I can be like "gotcha!"

I'll also just point out my philosophy for using LLMs for this project, which is that I'm not trying to go as fast as I can. (I want to go at a good pace, but this isn't an experiment to just finish something over a weekend). The 150k LOC have come about since February, with some mix of me writing code and LLMs, so on average I'm probably bringing in about 800 LOC per day, which I imagine a lot of vibers would find to be glacial. To me that's the sustainable rate of what I can do when you factor in that I need to test drive every feature, make sure it doesn't conflict with another feature, check for bugs, check that the code looks reasonable, and debugging. (I also think that rate limit is specific to this project: I could see easier to test things going much faster, and harder to test things going slower)

Re: Developers are attached to tools because tools encode trust

#29
post #2

I have to admit, I asked ChatGPT to do a TLDR summary because I found the writing meandered quite a bit. I think the overall point is sound: > "Developers become attached to tools like Vim, Emacs, or an IDE because years of experience make those tools predictable extensions of their thinking. The attachment is less about features and more about accumulated trust, muscle memory, and a workflow built around known bound…

> Anyway I wouldn't say these tools aren't useful, but, I'm deeply skeptical of all the productivity claims because I think people just look at one dimension of it while ignoring all the other important dimensions. Yeah you can generate a lot of crap fast, but most of it is not shippable and making it shippable does take time.

My own stance is that there's never any reason to go fast on anything. Communication has always been the bottleneck. Whether it's about gathering requirements or understanding the purpose of a badly written code, any speed improvements I get has always been a small percentage of the overall progress.

What has helped more is my understanding of the platform and some theoretical knowledge. Because one I get the information, I can quickly derive a solution in my mind. And that solution has always been easy and fast to implement, at least the happy path. 90% of the time taken in coding is always about handling all the edge cases, aka fixing bugs. And writing tests so that you're not easily introducing more bugs.

Re: Developers are attached to tools because tools encode trust

#30
post #15
post #12

I feel like these abstractions like "CI might not work well in the era of agentic tooling" are fine for thought-leadership posts but there's so much hands-on work to be done. The last word on AI computer use shouldn't be bash utils that were already feature-complete before MJ recorded Thriller. There is some movement in this direction--there is a new 'gh' subcommand called repo read-file for example, that lets agents…

MCP is the answer to not using bash, right? Bash is a great control surface anyway for LLMs as it is wordy and powerful.

Also, this is a trivial example but I would rather an LLm call a recursive "grep" than using 1000 read_file MCP calls.

Having a scripting language as a tool is powerfull, and can help remove unnecessary stuff from the LLM's context.

Post reply on HN