Viewing profile — Ianjit

I guess it all comes down to what a meaningful gain is. I agree that 10-30% is meaningful and if “software is a gas” this will lead to more software. But my expectations had become…

I don’t work in SWE so I am just reacting to the claims that LLMs 10x productivity and are leading to mass layoff in the industry. In that context the 6-12% productivity gain at a …

A few studies over different time frames: [1] https://www.youtube.com/watch?v=1OzxYK2-qsI [2] https://www.coderabbit.ai/blog/state-of-ai-vs-human-code-gen... [3] https://www.youtub…

Sorry for the late response! My guess is they didn't publish it because they only measured it at one company, if they had the data across the cohort they would have published. The …

Studies have shown that software engineers are very bad at judging their own productivity. When a software engineer feels more productive the inverse is just as likely to be true. …

Please provide links to the studies, I am genuinely curious. I have been looking for data but most studies I find showing an uplift are just looking at LOC or PRs, which of course …

I think an OpenAI paper showed 25% of GPT usage is “seeking information”. In that case Google also has a an advantage from being the default search provider on iOS and Android. I d…

The productivity studies on software engineers directly don't show much of a productivity gain certainly nowhere near the 10x the frontier labs would like to claim. When including …

"History doesn't repeat itself, but it often rhymes", except in the world of computer science where history does repeat.

Radiology has proven to be one of the most defensive jobs in medicine, radiologists beat AI once already! https://www.worksinprogress.news/p/why-ai-isnt-replacing-rad...

My bad. What was the result when they measured productivity after rework across the entire co hort?

I think you are quoting productivity measured before checking the code actually works and correcting it. After re-work productivity drops to 1%. Tinestamp 14:04.

This study does a good job of measuring the productivity impact. It found 1% uplift in dev productivity from using AI. https://youtu.be/JvosMkuNxF8?si=J9qCjE-RvfU6qoU0

This Stanford study on developer productivity found 0 correlation between developers assessment of their own productivity and independent measures of their productivity. Any anecdo…

Isn’t this what Tao is addressing in the link, that LLMs haven’t encoded reasoning? Success in IMO is misleading because they are synthetic problems with known solutions that are s…

I think people make comments on LLMs not being smart in reaction to the comments from the leaders of AI labs that LLMs are so smart they could/will lead to mass unemployment.

Are SWE’s really experiencing a productivity uplift? When studies attempt to measure the productivity impact of AI in software the results I have seen are underwhelming compared to…

My impression from Tao’s answers is that there is a gulf between the reality of what mathematicians actually do and the benchmarks that the frontier labs tout. It doesn’t seem triv…

Meta internal study showed a 6-12% productivity uplift. https://youtu.be/1OzxYK2-qsI?si=8Tew5BPhV2LhtOg0

The productivity uplift is massive, Meta got a 6-12% productivity uplift from AI coding! https://youtu.be/1OzxYK2-qsI?si=8Tew5BPhV2LhtOg0

The studies I have seen show that AI written software is 70% more buggy than human written code [1]. I am curious where you get your data on AI code having “a similar reliability t…

Maybe you are right, but maybe it’s radiology all over again.