Google CEO says more than a quarter of the company's new code is created by AI • Hacker News Board

Re: Google CEO says more than a quarter of the company's new code is created by AI

#201

Post by wslh » Wed, Oct 30, 2024, 9:24 PM UTC

ntulpule wrote: post #88

Hi, I lead the teams responsible for our internal developer tools, including AI features. We work very closely with Google DeepMind to adapt Gemini models for Google-scale coding and other Software Engineering usecases. Google has a unique, massive monorepo which poses a lot of fun challenges when it comes to deploying AI capabilities at scale. 1. We take a lot of care to make sure the AI recommendations are safe and…

As someone working in cybersecurity and actively researching vulnerability scanning in codebases (including with LLMs), I’m struggling to understand what you mean by “safe.” If you’re referring to detecting security vulnerabilities, then you’re either working on a confidential project with unpublished methods, or your approach is likely on par with the current state of the art, which primarily addresses basic vulnerabilities.

Re: Google CEO says more than a quarter of the company's new code is created by AI

#202

Post by macintux » Wed, Oct 30, 2024, 9:24 PM UTC

taeric wrote: post #191

gerash wrote: post #177

Earlier quoted context omitted.

I believe most people use AI to help them quickly figure out how to use a library or an API without having to read all their (often out dated) documentation instead of helping them solve some mathematical challenge

If the documentation is out of date, such that it doesn't help, this doesn't bode well for the training data of the AI helping it get it right, either?

AI can presumably integrate all of the forum discussions talking about how people really use the code.

Assuming discussions don't happen in Slack, or Discord, or...

Re: Google CEO says more than a quarter of the company's new code is created by AI

#203

Post by AnimalMuppet » Wed, Oct 30, 2024, 9:26 PM UTC

paradox242 wrote: post #142

LinuxBender wrote: post #110

Earlier quoted context omitted.

Is AI ready to crawl through all open source and find / fix all the potential security bugs or all bugs for that matter? If so will that become a commercial service or a free service? Will AI be able to detect bugs and back doors that require multiple pieces of code working together rather than being in a single piece of code? Humans have a hard time with this. - Hypothetical Example: Authentication bugs in sshd that…

Seems like there is more gain on the adversary side of this equation. Think nation-states like North Korea or China, and commercial entities like Pegasus Group.

Google's AI would have the advantage of the source code. The adversaries would not. (At least, not without hacking Google's code repository, which isn't impossible...)

Re: Google CEO says more than a quarter of the company's new code is created by AI

#204

Post by QuercusMax » Wed, Oct 30, 2024, 9:26 PM UTC

devonbleak wrote: post #197

It's Go. 25% of the code is just basic error checking and returning nil.

In Java, 25% of the code is import statements and curly braces

Re: Google CEO says more than a quarter of the company's new code is created by AI

#205

Post by soperj » Wed, Oct 30, 2024, 9:26 PM UTC

The real question is how many lines of code was it responsible for removing.

Re: Google CEO says more than a quarter of the company's new code is created by AI

#206

Post by randomNumber7 » Wed, Oct 30, 2024, 9:27 PM UTC

I cannot imagine this to be true, cause imo current LLM's coding abilities are very limited. It definitely makes me more productive to use it as a tool, but I use it mainly for boilerplate and short examples (where I had to read some library documentation before).

Whenever the problem requires thinking, it horribly fails because it cannot reason (yet). So unless this is also true for google devs, I cannot see that 25% number.

Re: Google CEO says more than a quarter of the company's new code is created by AI

#207

Post by eco » Wed, Oct 30, 2024, 9:27 PM UTC

skrebbel wrote: post #127

To my experience, AIs can generate perfectly good code relatively easy things, the kind you might as well copy&paste from stackoverflow, and they'll very confidently generate subtly wrong code for anything that's non-trivial for an experienced programmer to write. How do people deal with this? I simply don't understand the value proposition. Does Google now have 25% subtly wrong code? Or do they have 25% trivial code…

I'd be terribly scared to use it in a language that isn't statically typed with many, many compile time error checks.

Unless you're the type of programmer that is writing sabots all day (connecting round pegs into square holes between two data sources) you've got to be very critical of what these things are spitting out.

Re: Google CEO says more than a quarter of the company's new code is created by AI

#208

Post by ThinkBeat » Wed, Oct 30, 2024, 9:27 PM UTC

This is quite interesting to know.

I will be curious to see if it has any impact positive or negative over a couple of years.

Will the code be more secure since the AI does not make the mistakes humans do?

Or will the code, not well enough understood by the employees, exposes exploits that would not be there?

Will it change average up time?

Re: Google CEO says more than a quarter of the company's new code is created by AI

#209

Post by akoboldfrying » Wed, Oct 30, 2024, 9:29 PM UTC

chongli wrote: post #169

toasteros wrote: post #144

Earlier quoted context omitted.

> the kind you might as well copy&paste from stackoverflow This bothers me. I completely understand the conversational aspect - "what approach might work for this?", "how could we reduce the crud in this function?" - it worked a lot for me last year when I tried learning C. But the vast majority of AI use that I see is...not that. It's just glorified, very expensive search. We are willing to burn far, far more fuel t…

we've decided we can't be bothered with traditional search Traditional search (at least on the web) is dying. The entire edifice is drowning under a rapidly rising tide of spam and scam sites. No one, including Google, knows what to do about it so we're punting on the whole project and hoping AI will swoop in like deus ex machina and save the day.

>The entire edifice is drowning under a rapidly rising tide of spam and scam sites.

You make this claim with such confidence, but what is it based on?

There have always been hordes of spam and scam websites. Can you point to anything that actually indicates that the ratio is now getting worse?

Re: Google CEO says more than a quarter of the company's new code is created by AI

#210

Post by masfuerte » Wed, Oct 30, 2024, 9:30 PM UTC

chongli wrote: post #169

toasteros wrote: post #144

Earlier quoted context omitted.

> the kind you might as well copy&paste from stackoverflow This bothers me. I completely understand the conversational aspect - "what approach might work for this?", "how could we reduce the crud in this function?" - it worked a lot for me last year when I tried learning C. But the vast majority of AI use that I see is...not that. It's just glorified, very expensive search. We are willing to burn far, far more fuel t…

we've decided we can't be bothered with traditional search Traditional search (at least on the web) is dying. The entire edifice is drowning under a rapidly rising tide of spam and scam sites. No one, including Google, knows what to do about it so we're punting on the whole project and hoping AI will swoop in like deus ex machina and save the day.

Google results are not polluted with spam because Google doesn't know how to deal with it.

Google results are polluted with spam because it is more profitable for Google. This is a conscious decision they made five years ago.