191–200 of 217 posts
I'm spending an inordinate amount of time turning that video into an essay, but I feel like I'm being scooped already, so here's my current draft in case anyone wants to get a sneak preview: https://valdottown--89ed76076a6544019f981f7d4397d736.web.val...
Feedback appreciated :)
What I noticed: AI development constantly breaks my flow. It makes me more tired, and I work for shorter time periods on coding. It's a myth that you can code a whole day long. I usually do intervals of 1-3 hours for coding, with some breaks in between. Procrastination can even happen on work related things, like reading other project members code/changes for an hour. It has a benefit to some extent, but during this…
It would be interesting to record what one do in a day at the desk. Probably quite depressing to watch. Like, I think 1h would be streaching it for mature codebases.
Like doom scrolling on social media: Let's see what the fancy new guy got done this week. I need to feel better, I'm just going to look at the commits of the guy in the other team that always breaks production. Let's see how close he got to that recently, ...
Earlier quoted context omitted.
I have written every line of code in the code base I mostly work in and I still find it incredibly valuable. Millions use these tools and a large percentage of them find them useful in their familiar code base. Again, overwhelming anecdote and millions of users > "study"
> Interestingly the developers predict that AI will make them faster, and continue to believe that it did make them faster, even after completing the task slower than they otherwise would! In this case clearly anecdotes are not enough. If that quote from the article is accurate, it shows that you cannot trust the developers time perception. I agree, its only one study and we should not take it as the final answer. It…
The scientific method goes right out the window when it comes to true believers. It reminds me of weed-smokers who insist getting high makes them deep-thinkers: it feels that way in the moment, but if you've ever been a sober person caught up in a "deep" discussion among people high on THC, oh boy...
Earlier quoted context omitted.
If this makes sense, how is the study able to give a reasonable measure of how long an issue/task should have taken, vs how long it took with AI to determine that using AI was slower? Or it's comparing how long the dev thought it should take with AI vs how long it actually took, which now includes the dev's guess of how AI impacts their productivity? When it's hard to estimate how difficult an issue should be to comp…
An easier way to think about it might be if you timed how long it took each ticket in your backlog. You also recorded whether you were drunk or not when you worked on it, and the ticket was selected at random from your backlog. The assumption (null-hypothesis) is that being drunk has no effect on ticket completion time. Using the magic of statistics, if you have completed enough tickets, we can determine whether the…
> The inability of developers to tell if a tool sped them up or slowed them down is fascinating in itself, probably applies to many other forms of human endeavour, and explains things as varied as why so many people think that AI has made them 10 times more productive, why I continue to use Vim, why people drive in London etc. In boating, there's a notion of a "set and drift" which describes how wind and current push…
Linux/UNIX users are convinced of the superiority of keyboard control and CLI tools, but studies have shown that the mouse is faster for almost all common tasks.
Keyboard input feels faster because there are more actions per second.
Earlier quoted context omitted.
An easier way to think about it might be if you timed how long it took each ticket in your backlog. You also recorded whether you were drunk or not when you worked on it, and the ticket was selected at random from your backlog. The assumption (null-hypothesis) is that being drunk has no effect on ticket completion time. Using the magic of statistics, if you have completed enough tickets, we can determine whether the…
The challenge with “controlled experiments” is that saying to developers to “use AI for all of your tickets for a month” forces a specific tool onto problems that may not benefit from that tool.
Only one developer in this study had more than 50h of Cursor experience, including time spent using Cursor during the study. That one developer saw a 25% speed improvement. Everyone else was an absolute Cursor beginner with barely any Cursor experience. I don't find it surprising that using tools they're unfamiliar with slows software engineers down. I don't think this study can be used to reach any sort of conclusio…
Hey, thanks for digging into the details here! Copying a relevant comment ( https://news.ycombinator.com/item?id=44523638 ) from the other thread on the paper, in case it's help on this point. 1. Some prior studies that find speedup do so with developers that have similar (or less!) experience with the tools they use. In other words, the "steep learning curve" theory doesn't differentially explain our results vs. oth…
I think we should all focus on how the effectivity is going to change in the long-term. We all know AI tooling is not going to disappear but to become better and better. I wouldn't be afraid to lose some productivity for months if I would acquire new skills for the future.
Hey HN -- study author here! (See previous thread on the paper here [1].) I think this blog post is an interesting take on one specific factor that is likely contributing to slowdown. We discuss this in the paper [2] in the section "Implicit repository context (C.1.5)" -- check it out if you want to see some developer quotes about this factor. > This is why AI coding tools, as they exist today, will generally slow so…
Earlier quoted context omitted.
There was a recent study that found that LLM users in general tend to feel like they were more productive with AI while actually being less productive.
presumably the study this very HN discussion responds to.
> The inability of developers to tell if a tool sped them up or slowed them down is fascinating in itself, probably applies to many other forms of human endeavour, and explains things as varied as why so many people think that AI has made them 10 times more productive, why I continue to use Vim, why people drive in London etc. In boating, there's a notion of a "set and drift" which describes how wind and current push…
> The inability of developers to tell if a tool sped them up or slowed them down is fascinating in itself Linux/UNIX users are convinced of the superiority of keyboard control and CLI tools, but studies have shown that the mouse is faster for almost all common tasks. Keyboard input feels faster because there are more actions per second.
Do you think that daily CLI Linux/UNIX users might have a different list of what they consider "common tasks"?