Live data from Hacker News

Survey: a third of senior developers say over half their code is AI-generated

fastly.com

71–80 of 388 posts

Re: Survey: a third of senior developers say over half their code is AI-generated

#71

Earlier quoted context omitted.

> If I … generate fairly exhaustive unit tests of a trivial function … then you are not a senior software engineer

Neither are you if that's your understanding of a senior engineer

We're not a licensed profession with universally defined roles. It's whatever the speaker wants it to be given how wildly it varies.

Re: Survey: a third of senior developers say over half their code is AI-generated

#72
post #56

> survey of 791 developers We have got to stop. In a universe of well over 25 million programmers a sample of 791 is not significant enough to justify such headlines. We’ve got to do better than this, whatever this is.

Validity of the sample size is not determined by its fraction of the whole population. I don't know the formulas and I'm not a statistician. Maybe someone can drop some citations.

Re: Survey: a third of senior developers say over half their code is AI-generated

#73
post #56

> survey of 791 developers We have got to stop. In a universe of well over 25 million programmers a sample of 791 is not significant enough to justify such headlines. We’ve got to do better than this, whatever this is.

You should read more about statistical significance. Under some reasonable assumptions, you can confidently certain deduce things with small sample sizes.

From another perspective: we've deduced a lot of things about how atoms work without any given experiment inspecting more than an insignificant fraction of all atoms.

TL;DR: The population size (25e6 total devs, 1e80 atoms in observable universe) is almost entirely irrelevant to hypothesis testing.

Re: Survey: a third of senior developers say over half their code is AI-generated

#74
This is self reported unless I missed something. I bet that skews these results quite a bit. Many are very hesitant to say they use AI, and I suspect that's much more likely to be the case when you are new to the field.

Also, green coding? That's new to me. I guess we'll see optional carbon offset purchasing in our subs soon.

Re: Survey: a third of senior developers say over half their code is AI-generated

#75
post #68

Earlier quoted context omitted.

Neither are you if that's your understanding of a senior engineer

I think the parent commentors point was that it is nearly trivial to generate variations on unit tests in most (if not all) unit test frameworks. For example: Java: https://docs.parasoft.com/display/JTEST20232/Creating+a+Para... C# (nunit, but xunit has this too): https://docs.nunit.org/articles/nunit/technical-notes/usage/... Python: https://docs.pytest.org/en/stable/example/parametrize.html cpp: https://google.gith…

The useful part is generating the mocks. The various auto mocking frameworks are so hit or miss I end up having to manually make mocks which is time consuming and boring. LLMs help out dramatically and save literally hours of boring error prone work.

Re: Survey: a third of senior developers say over half their code is AI-generated

#76
post #70

So how many developers in that survey are those? They surveyed 791 developers (:D) and "a third of senior developers" do that. That's... generiously, what... 20 people? It's amazing how everyone can massage numbers when they're trying to sell something.

The other thing they do is conveniently not mention all the negative stuff about AI that the source article mentions, they only report on the portion of content from the source that's in any way positive of AI.

And of course, its an article based on a source article based on a survey (of a single company), with the source article written by a "content marketing manager", and the raw data of the survey isn't released/published, only some marketing summary of what the results (supposedly) were. Very trustworthy.

Re: Survey: a third of senior developers say over half their code is AI-generated

#77
post #63

This article goes completely against my experience so far. I teach at an internship program and the main problem with interns since 2023 has been their over reliance on AI tools. I feel like I have to teach them to stop using AI for everything and think through the problem so that they don't get stuck. Meanwhile many of the seniors around me are stuck in their ways, refusing to adopt interactive debuggers to replace…

> Meanwhile many of the seniors around me are stuck in their ways, refusing to adopt interactive debuggers to replace their printf() debug habits, let alone AI tooling... When I was new to the business, I used interactive debugging a lot. The more experienced I got, the less I used it. printf() is surprisingly useful, especially if you upgrade it a little bit to a log-level aware framework. Then you can leave your de…

This is absolutely true. If anything, interactive debuggers are a crutch and actual logging is the real way of debugging. You really can't debug all sorts of things in an interactive debugger, things like timing issues, thread problems, and you certainly can't find the actual hard bugs that are in running services in production, you know, where the bugs actually happen and are found. Or on other people's machines that you can't just attach a debugger. You need good logging with a good logging library that doesn't affect performance too much when it's turned off, and those messages can also provide very useful context to what things are going on, many times as good if not better than a comment, because at least the log messages are compiled in and type checked, as opposed to comments, which can easily go stale.

Re: Survey: a third of senior developers say over half their code is AI-generated

#78

Seems about right for me (older developer at a big tech company). But we need to define what it means that the code is AI-generated. In my case, I typically know how I want the code to look like, and I'm writing a prompt to tell the agent to do it. The AI doesn't solve any problem, it just does the typing and helps with syntax. I'm not even sure I'm ultimately more productive.

I’m not yet up to half (because my corporate code base is a mess that doesn’t lend itself well to AI)

But your approach sounds familiar to me. I find sometimes it may be slower and lower quality to use AI, but it requires less mental bandwidth from me, which is sometimes a worthwhile trade off.

Re: Survey: a third of senior developers say over half their code is AI-generated

#79

Seems about right for me (older developer at a big tech company). But we need to define what it means that the code is AI-generated. In my case, I typically know how I want the code to look like, and I'm writing a prompt to tell the agent to do it. The AI doesn't solve any problem, it just does the typing and helps with syntax. I'm not even sure I'm ultimately more productive.

Yeah I’m still not more productive. Maybe 10% more. But it alleviates a lot of mental energy, which is very nice at the age of 40.

Re: Survey: a third of senior developers say over half their code is AI-generated

#80

Earlier quoted context omitted.

Claude is making $72k a year for a consistent $300/day spend.

Bear in mind those are revenue figures, they're costing claude hundreds a day. One imagines Leadership won't be so pleased after the inevitably price hike (which, given the margins software uses, is going to be in the 1-3 thousands a day) and the hype wears off enough for them to realize they're spending a full salary automating a partial FTE.

But, by the looks of things, models will be more efficient by then and a cheaper-to-run model will produce comparable output. At least that's how it's been with OSS models, or with the Openai api model. So maybe the inevitable price hike (or rate limiting) may lead to switching models / providers and the results being just as good.
Post reply on HN