I wonder how much of that code is boilerplate vs. actual functionality.
Google CEO says more than a quarter of the company's new code is created by AI
651–660 of 1001 posts
Re: Google CEO says more than a quarter of the company's new code is created by AI
#652I work for Google, and I just got done with my work day. I was just writing I guess what you'd call "AI generated code." But the code completion engine is basically just good at finishing the lines I'm writing. If I'm writing "function getAc..." it's smart enough to complete to "function getActionHandler()", and maybe suggest the correct arguments and a decent jsdoc comment. So basically, it's a helpful productivity…
I'm working on a CRM with a flexible data model, and ChatGPT has written most of the code. I don't use the IDE integrations because I find them too "low level" - I work with GPT more in a sort of "pair programming" session: I give it high level, focused tasks with bits of low level detail if necessary; I paste code back and forth; and I let it develop new features or do refactorings. This workflow is not perfect but…
I share your sentiment, I've written three apps where I've used language models extensively (a different one for each: ChatGPT, Mixtral and Llama-70B) and while I agree that they where immensely helpful in terms of velocity, there are a bunch of caveats:
- it only works well when you write code from scratch, context length is too short to be really helpful for working on existing codebase.
- the output code is pretty much always broken in some way, and you need to be accustomed to doing code reviews to use them effectively. If you trust the output and had to debug it later it would be a painfully slow process.
Also, I didn't really noticed a significant difference in code quality, even the best model (GPT-4) write code that doesn't work, and I find it much more efficient to use open models on Groq due to the really fast inference. Looking at ChatGPT slowly typing is really annoying (I didn't test o1 and I have no interest in doing so because of its very low throughput).
Re: Google CEO says more than a quarter of the company's new code is created by AI
#653I work for Google, and I just got done with my work day. I was just writing I guess what you'd call "AI generated code." But the code completion engine is basically just good at finishing the lines I'm writing. If I'm writing "function getAc..." it's smart enough to complete to "function getActionHandler()", and maybe suggest the correct arguments and a decent jsdoc comment. So basically, it's a helpful productivity…
I also work at google (until last Friday). Agree with what you said. My thoughts are
1. This quote is clearly meant to exaggerate reality, and they are likely including things like fully automated CL/PR's which have been around for a decade as "AI generated".
2. I stated before that if a team of 10 is equally as productive as a team of 8 utilizing things like copilot, it's fair to say "AI replaced 2 engineers", in my opinion. More importantly, Tech leaders would be making this claim if it were true. Copilot and it's clones have been around long enough know for the evidence to be in, and no one is stating "we've replaced X% of our workforce with AI" - therefor my claim is (by 'denying the consequent'), using copilot does not materially accelerate development.
Re: Google CEO says more than a quarter of the company's new code is created by AI
#654I'm waiting for some Google developer to say "More than a quarter of the CEOs statements are now created by AI"... ;-)
Re: Google CEO says more than a quarter of the company's new code is created by AI
#655I saw code on master which was parsing HTML with regex. The author was proud that this code was mostly generated by AI.
:)
Re: Google CEO says more than a quarter of the company's new code is created by AI
#656Earlier quoted context omitted.
These tools also require the developer class to that they are intended to replace to continue to do what they currently do (create the knowledge source to train the AI on). It's not like the AIs are going to be creating the accessible knowledge bases to train AIs on, especially for new language extensions/libraries/etc. This is a one and f'd development. It will give a one time gain and then companies will be shocked…
One of my concerns is that AI may actually slow innovation in software development (tooling, languages, protocols, frameworks and libraries), because the opportunity cost of adopting them will increase, if AI remains unable to be taught new knowledge quickly.
It also bugs me that these tools will reduce the incentive to write better frameworks and language features if all the horrible boilerplate is just written by an LLM for us rather than finding ways to design systems which don't need it.
The idea that our current languages might be as far as we get is absolutely demoralising. I don't want a tool to help me write pointless boilerplate in a bad language, I want a better language.
Re: Google CEO says more than a quarter of the company's new code is created by AI
#657I wanted a new feature in our customer support console and the dev lead suggested I write a JIRA. I’m the CEO, so this is not my usual thing (and probably should not be). I told Claude what I wanted and pasted in a .js file from the existing project so that it would get a sense of the context. It cranked out a fully functional React component that actually looks quite nice too. Two new API calls were needed, but Clau…
> Everyone knows this was “made by AI” because there’s no way in hell I would ever have the time.
Doubtful. A decent fraction of the people reading it will guess that you've wasted your time writing incoherent nonsense in the jira. Engineers don't usually have much insight into what the C suite are doing. It would be a prudent move to spend the couple of seconds to write "something like this AI sketch:" before the copy&paste.
Re: Google CEO says more than a quarter of the company's new code is created by AI
#658Everyone here is arguing about the average AI code quality and I'm here just not believing the claim. Is Google out there monitoring the IDE activity of every engineer, logging the amount of code created, by what, lines, characters, and how it was generated? Dubious.
Very good point. How was the 25% measured?
Re: Google CEO says more than a quarter of the company's new code is created by AI
#659So we're using CoL as a metric now?
Re: Google CEO says more than a quarter of the company's new code is created by AI
#660I work for Google, and I just got done with my work day. I was just writing I guess what you'd call "AI generated code." But the code completion engine is basically just good at finishing the lines I'm writing. If I'm writing "function getAc..." it's smart enough to complete to "function getActionHandler()", and maybe suggest the correct arguments and a decent jsdoc comment. So basically, it's a helpful productivity…
I also work at Google and I agree with the general sentiment that AI completion is not doing engineering per se, simply because writing code is just a small part of engineering.
However in my experience the system is much more powerful than you described. Maybe this is because I'm mostly writing C++ for which there is a much bigger training corpus than JavaScript.
One thing the system is already pretty good at is writing entire short functions from a comment. The trick is not to write:
function getAc...
// This function smargls the bleurgh
// by flooming the trux.
function getAc...