I obviously don't mean that people literally write "write me some typescript", because nobody wants code that does something arbitrary. I'm also not saying that every reaction to ai falls between love and skeptical: I wrote a 3 sentence comment on a complex topic to sketch out an idea. The tone of your comment suggests that my comment upset you, which wasn't my intent. But you have to try to be a little generous when…
Ironically, his comment does not come off at all as offended, but yours does.
It came off a little offensive ("...incredible reductive scale...") in my opinion.
What bothers me more than any of this particular discussion is that we seem to be incapable of determining programmer productivity in a meaningful way since my debut as a programmer 40 years ago.
But nevertheless, productivity objectively exists. Some people/teams are more productive as others. I suppose it would be simpler to compare productivity for people working on standard, "normalized" tasks, but often every other task a programmer is assigned is something different to the previous one, and different developers get different tasks. It's difficult to measure productivity based on real-world work, but we…
The outcome of complex systems cannot be trivially mapped to arbitrary components.
Focusing on the speed of outputting code or closing tickets is shortsighted from a software engineering standpoint.
The thing about tools is that they need to be predictable. I can't remember the source, but it's a concept I read that really stuck with me. A predictable tool can be used skillfully and accurately because the user can anticipate how it works and deploy it effectively. It will always be aligned with the user intent because the user decides how and when it is used. A tool that constantly adapts to how it is used will…
Sounds to me like you're not describing its limited usefulness in general, you're describing its limited usefulness when as considered a tool. That might seem like a distinction without a point but people are not tools and people are useful. If someone argues people are tools then they don't fit your definition any more than an LLM does. LLMs are as useful as people I would say, and as unpredictable. I run a small co…
It's difficult for me to judge their usefulness as a tool. I've used them in lieu of a google search and gotten good information, but i've also got bad information from them. The lack of reliability means I can't just ask an LLM and keep rolling, but the high probability that it will save me some time scouring the internet means there's some value there. I'm not sure I can provide a definitive judgement on them.
I think the dichotomy you see with how positive people are about ai has almost entirely to do with the kind of questions they ask. That seems obvious, but a consequence of that is that people who are sceptical of ai (like me) only use it when they've exhausted other resources (like google). You ask very specific questions where not a lot of documentation is available and inevetably even o3 ends up being pretty useles…
> You ask very specific questions where not a lot of documentation is available and inevetably even o3 ends up being pretty useless. You have any example questions where o3 failed to be helpful? I use it pretty similarly to you, only resorting to it to unblock myself basically, otherwise I'm mostly the one doing the actual work, LLMs help with specific functions or specific blockers, or exploring new "spaces". But al…
I had to create a Cython module wrapping some C, used Claude 4 and GPT 4.1, they were worse than useless. One can imagine why I needed help with that project.
Copilot is going to feel "amazing" at helping you quickly work within just about any subject that you're not already an expert in. Whether or not a general purpose foundation model for coding is trained on more backend or frontend code is largely irrelevant in this specific context.
Really a great piece of work. At the opposite of the usual studies posted here.
With the headline we can easily guess that the study should be flawed, with the sample not representative, or developers not expert enough with the AI or so, and then, they give a very well done list of all valable objections with arguments about why they don't think that contradict the study.
That replied to all the questions I could have had in the end.
It's astonishing. A bit scary actually. Can easily see the role of front-end slowly morphing into a single person team managing a set of AI tools. More of an architecture role.
A yes I’m more of a backend guy that loves tailwind.
Can you say more than literally "you're using it wrong"? Otherwise this is a no true scotsman (super common when LLM advocates are touting their newfound productivity). Here are my prompts, lightly redacted: First prompt: ``` Build a new package at . Use the package at as an example. The new package should work like the package, but instead of receiving events over HTTP, it should receive events as JSON over a Google…
Yeah, that was a pretty lazy response on my part. Let me try again. In my opinion, it takes several weeks of active use to nail down your preferred workflow with these tools and to get a meaningful understanding of their abilities and limitations. I.e., yes they hallucinate and don't have great understanding of truth/fact (however you choose to define those terms), but you need to develop an intuition for how to work…
Thanks! Uh, it helps a little. I'll see about making an AGENTS.md. I guess I can agree that there's something about like, maximizing the amount of good output from an agent, and to do that you need to give it a lot of access and info. I thought I did that though by giving it the whole codebase and linking pubsub docs, maybe the job I gave it was too small, but that's hard for me to accept: I wanted an entirely new module, is the next step like, ask it to make all the modules? What if I only need one?
I am waiting to see your 4 year human-equivalent project in a couple of months. Words without actions are junk. You are asserting something you have no proof for. Proove it then. Amaze us all with your productivity, out in the open. Shred those pilled up open issues on open source projects and then give us a report of how fast-easy it. If it is "easily true" you'll be done by next month
I don’t need to prove to you something I experience daily, both professionally and in hobby side projects. You can take my anecdotes or you can keep believing me and people like me are wrong, your choice.
If you don't need to prove it to me then you you wouldn't have replied to my initial comment to begin with.
But I agree, you don't need to prove anything to me. You need to proove it to yourself first.
Now do a study that specifically gauges how useful an LLM (including smart tab completion) is for a frontend dev working in react/next/tailwind on everyday Jira tickets. These were maintainers of large open source projects. It's all relative. It's clearly providing massive gains for some and not as much for others. It should follow that it's benefit to you depends on who you are and what you are working on. It isn't…
It's a very well controlled study about... what the study claims to do. Yes, they didn't study a different thing, for _many_ reasons. Yes, we shouldn't haphazardly extrapolate to other parts of Engineering. But it looks like it's a good study nonetheless. There are some very good findings though, like how the devs thought they were sped up but they were actually slowed down.
Agreed. My comment was anticipating the extrapolation part :)