The AI Productivity Gap

That’s a wildly uncharitable read.

Empirical evidence through observation or self-reporting, sampling in some meaningful way, would obviously be preferable but is also often just not practical.

Guessing at numbers to check whether your thesis even works with some plausible assumptions is a meaningful first step and to my mind a good way to reason through something like this and make it discussable.

A possible outcome of such an exercise is also that for your thesis to work out you need to make wildly implausible assumptions, so that helps you to discard that thesis.

From my perspective this is a very useful way to approach a hypothesis where empirical evidence is scarce or at least hard to get. No reason to dismiss it immediately – especially since the fact that those are guesses was never hidden.

Yup. I can expect an llm to write proper code to update an hashmap or update a db, but ffs the amount of race conditions, use after free and general concurrency issues I found on colleagues PRs developed fully with these things is disgusting.

No. There is this persistent belief in the industry that programmers should be good at everything, not just programming itself: communication, product management, design, sysops, UX, coaching, management, testing and so on. The most visible product of this belief was the once hyper hyped role of “full stack” developer.

The really is that you could have experts in each area doing what they’re good, which means letting programmers actually program most of the time, and let business analysts figure out requirements, product owners decide features, designers decide UX and design, QA perform in-depth testing… sure , every programmer will have to manage some of this themselves to not get blocked the whole time waiting for someone to decide something, but that is NOT the same as just having programmers handle everything!

I think that if you only ever hire programmers who are also kind of people person, you definitely have to accept missing out on the antisocial but genius ones who are very likely the only ones capable of tackling the really hard problems! Unpopular view, I know, but it takes a certain type of person to achieve excellence in some areas. Just look at the most successful artists, writers, actors and especially CEOs. Programmers are clearly in that category. I’ve seen “normies” trying to write a little code. They don’t last an hour before they decide it’s bullshit that you need a semicolon precisely placed for the code to not explode, or that they can’t compile on this system until you’ve installed some tool chain that requires a bunch of commands no one knows by heart but you just need to make sure to follow exactly, otherwise hell may break lose.

Compare the performance of a 980 and a 5050 and I am sure that will answer your question.

Also models baked into the silicon are able to achieve efficiency that is simply impossible to achieve with programmable circuits, there is a general slowdown in the raw capabilities that transformers can achieve and agentic tool use is simply an amplifier that will reach a wall eventually. It wouldn't surprise me if we saw within 5 to 10 years accelerator cards that you're able to purchase and plug into via usb-c that are able to achieve thousands of tok/s as well as api costs going down to what we already see with subscriptions today.

There has been quite a lot of off-ramping going on where people feel satisfied with the performance they're getting out of the models and simply staying there instead of using SOTA.