Where is all this code? If the tools are so useful it should show up in widely used open source projects, but I'm not seeing anything.
Survey: a third of senior developers say over half their code is AI-generated
201–210 of 388 posts
Re: Survey: a third of senior developers say over half their code is AI-generated
#202Seems about right for me (older developer at a big tech company). But we need to define what it means that the code is AI-generated. In my case, I typically know how I want the code to look like, and I'm writing a prompt to tell the agent to do it. The AI doesn't solve any problem, it just does the typing and helps with syntax. I'm not even sure I'm ultimately more productive.
Can you be specific about what you're asking it to write?
I'm pretty confidient that I couldn't get it to implement a element in a web browser. I'm talking about C++ in WebKit or Chromium, not a custom element in HTML/JS. Let's say browsers wanted a new data-table element that natively implemented a scrolling window such that you registered events to supply elements and it asked for only the portion that were visible. I feel like those code bases are too complex for LLMs to add this (could certainly be wrong).
In any case, more concrete examples would help these discussions.
I can give a concrete example: I just asked ChatGPT (should have asked something more integrated into my editor), to give me JavaScript to directly parse meta data out of an MP4 file in JS in the browser. It gave my typescript but told me I should look at mp4box.
I spent 15-20 minutes getting a mp4box example setup (probably should have asked for that too). Only to find that I think mp4box does not get me the data I wanted.
So I went back to ChatGPT and asked for JavaScript because I was hacking in something like jsfiddle. It gave me that and it worked!
I then said I wanted the title and comments meta data as that was missing from what it gave me. That worked too, first try.
Re: Survey: a third of senior developers say over half their code is AI-generated
#203Survey: a third of surveys say that over half of them are bullshit
Re: Survey: a third of senior developers say over half their code is AI-generated
#204Where is all this code? If the tools are so useful it should show up in widely used open source projects, but I'm not seeing anything.
How would you know?
Re: Survey: a third of senior developers say over half their code is AI-generated
#205Earlier quoted context omitted.
Why mock at all? Spend the time making integration tests fast. There is little reason a database, queue, etc. can't be set up in a per-test group basis and be made fast. Reliable software is built upon (mostly) reliable foundations.
hmmmm. I do like integration tests, but I often tell people the art of modern software is to make reliable systems on top of unreliable components. And the integration tests should 100% include times when the network flakes out and drops 1/2 of replies and corrupts msgs and the like.
Minor nit. I wouldn't call those failing systems tests integration tests.
Unit tests are for validation of error paths. Unit tests can leverage mocks or fakes. Need 3 retires with exponential back off, use unit tests and fakes. Integration tests should use real components. Typically, integration tests are happy path and unit are error paths.
Making real components fail and having tests validate failure handling in a more complete environment jumps from integration testing to resilience or chaos testing. Being able to accurately validate backoffs and retries may diminish, but validating intermediate or ending state can be done with artifact monitoring via sinks.
There is unit-integration testing which fakes out as little as possible but still fakes out some edges. The difference being that the failures are introduced via fake vs managing actual system components. If you connect to a real db on unit-integration tests, you typically wouldn't kill the db or use Comcast to slow the network artificially. That would be reserved for the next layer in the test pyramid.
Re: Survey: a third of senior developers say over half their code is AI-generated
#206LLMs are tools. If you master your tools, you become more productive. I've had the same mixed experiences with LLMs that others have. Some good, some not so great. But I've been merging quite a bit of codex created PRs in the last weeks. Some needed manual intervention. Some were tedious to create. And some clearly saved me a lot of work.
I always have more work than I can handle. Part of my job is deciding what not to do and what to drop because it hasn't got the right priority. Me spending an afternoon on a thing that is fun but not valuable is usually a bad use of my time. With LLMs, I'm taking on a few more of the things that I previously wouldn't have. That started with a few hobby projects that I'm now doing that I previously wasn't. And it's creeping into work as well. LLMs struggle on larger code bases. But less so with recent model releases.
Re: Survey: a third of senior developers say over half their code is AI-generated
#207Where is all this code? If the tools are so useful it should show up in widely used open source projects, but I'm not seeing anything.
How would you know?
Various projects have proposed a "Co-Developed-By" tag. Even if that were not used I'd expect some open source developers to have become much more productive, and written about it. We'd also see widespread adoption of ancillary tools like AI code reviewers (which have had some adoption but in my experience make suggestions that are worse than useless).
Re: Survey: a third of senior developers say over half their code is AI-generated
#208In terms of LOCs maybe, in terms of importance I think is much less. At least that's how I use LLMs. While I understand that might produce the meaty bits as well, I believe that having a truck factor of basically 0 (since no-one REALLY understands the code) is a recipe for a disaster and I dare say long term maintainability of a code base. I feel that you need to have someone in any team that needs to have that level…
> test fixtures
I'm curious- how does the AI know what you want?
Re: Survey: a third of senior developers say over half their code is AI-generated
#209Earlier quoted context omitted.
Bear in mind those are revenue figures, they're costing claude hundreds a day. One imagines Leadership won't be so pleased after the inevitably price hike (which, given the margins software uses, is going to be in the 1-3 thousands a day) and the hype wears off enough for them to realize they're spending a full salary automating a partial FTE.
But, by the looks of things, models will be more efficient by then and a cheaper-to-run model will produce comparable output. At least that's how it's been with OSS models, or with the Openai api model. So maybe the inevitable price hike (or rate limiting) may lead to switching models / providers and the results being just as good.
There is an interesting substack post about this. LLM costs are dropping 10x/year but the amount of tokens used have gone up like crazy https://open.substack.com/pub/ethanding/p/ai-subscriptions-g...
Re: Survey: a third of senior developers say over half their code is AI-generated
#210Earlier quoted context omitted.
The old fogeys don't rely on printf because they can't use a debugger, but because a debugger stops the entire program and requires you to go step by step. Printf gives you an entire trace or log you can glance at, giving you a bird's eye view of entire processes.
Most decent debuggers have condițional breakpoints.
Not to mention tracepoints (logging breakpoints), which are functionally the same as printf but don't require compiling/restarting.