Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI in software engineering at Google: Progress and the path ahead

research.google

Post reply on HN
Print view View original

11–20 of 245 posts

Re: AI in software engineering at Google: Progress and the path ahead

#11

Post by bdjsiqoocwk »

jprete wrote: post #6
trybackprop wrote: post #2

From the blog post: > We observe that with AI-based suggestions, the code author increasingly becomes a reviewer, and it is important to find a balance between the cost of review and added value. We typically address the tradeoff with acceptance rate targets. In the past year since GPT-4 came out, I've also found this to be the case. I'm an ML/backend engineer with little experience in frontend development. Yet, I've…

This isn't a good trend. Reviewers never have the depth of knowledge or understanding of authors.

Also, people shouldn't be allowed to use computers unless they understand how transistors work. If you don't have the depth of knowledge you get nothing.

Re: AI in software engineering at Google: Progress and the path ahead

#12

Post by runlevel1 »

Humans have limited RAM, so we have to put our ideas into an external medium that can then be refined.

I've been finding AI's suggestions -- even when rather wrong -- help me do that initial step faster. Which, I think, jives with their findings here.

Re: AI in software engineering at Google: Progress and the path ahead

#13

Post by desolved »

bdjsiqoocwk wrote: post #11
jprete wrote: post #6

Earlier quoted context omitted.

This isn't a good trend. Reviewers never have the depth of knowledge or understanding of authors.

Also, people shouldn't be allowed to use computers unless they understand how transistors work. If you don't have the depth of knowledge you get nothing.

It took me way too long to realize this was a joke.

Re: AI in software engineering at Google: Progress and the path ahead

#14

Post by bigiain »

I'm looking forward to the day that some spicy autocomplete regurgitates an obvious chunk of AGPL code that it's stolen without permission or attribution - and it ends up in some critical part of Googles money printing machine, and the outside world finds out about it.

I'm gonna need a _lot_ of popcorn.

Re: AI in software engineering at Google: Progress and the path ahead

#15

Post by ks2048 »

cloudking wrote: post #3

The most interesting chart is the "fraction of code created with AI assistance via code completion" trending up to 50%

Seems they need to compare against "dumb" code completion. It seems that even when they are error-free, "large" AI-code-completions are just boilerplate that should be abstracted away in some functions rather than inserted into your code base.

On a related note, maybe they should measure number of code characters that can be REMOVED by AI rather than inserted!

Re: AI in software engineering at Google: Progress and the path ahead

#16

Post by jjallen »

bigiain wrote: post #14

I'm looking forward to the day that some spicy autocomplete regurgitates an obvious chunk of AGPL code that it's stolen without permission or attribution - and it ends up in some critical part of Googles money printing machine, and the outside world finds out about it. I'm gonna need a _lot_ of popcorn.

How would the outside world find out about private Google code?

Maybe the fact could be leaked? Unlikely though.

Re: AI in software engineering at Google: Progress and the path ahead

#17

Post by refulgentis »

jprete wrote: post #6
trybackprop wrote: post #2

From the blog post: > We observe that with AI-based suggestions, the code author increasingly becomes a reviewer, and it is important to find a balance between the cost of review and added value. We typically address the tradeoff with acceptance rate targets. In the past year since GPT-4 came out, I've also found this to be the case. I'm an ML/backend engineer with little experience in frontend development. Yet, I've…

This isn't a good trend. Reviewers never have the depth of knowledge or understanding of authors.

Source? This is unintuitive to me, I can't come up with a rationale.

Re: AI in software engineering at Google: Progress and the path ahead

#18

Post by refulgentis »

endisneigh wrote: post #7

Anyone who works at Google knows they are force feeding the employees with this stuff. Same thing they’re doing with the general public and search. Anyone who worked at google during + can see this is the same half hearted type of force feeding that will fail, with the rationalizations to boot. Disclaimer: I hold google stock but don’t work at google

That's not true, I'm a xoogler as of October, and at least 2 of my ex-colleagues continue to generally wonder if AI can write code or not, and if it can, they haven't tried it. Last update 60 days ago.

It does look like there's an auto-installed cider extension, which is fine, the worst case for this stuff is "it's in my autocomplete list" -- that's fine!

Re: AI in software engineering at Google: Progress and the path ahead

#19

Post by runlevel1 »

cplat wrote: post #4
cloudking wrote: post #3

The most interesting chart is the "fraction of code created with AI assistance via code completion" trending up to 50%

Yes. Although I’d like to see a deeper investigation. Of course, quality of completions have improved. But there could be a confounding phenomenon where newer folks might just be accepting a lot of suggestions without scrutiny.

I accept a lot of wrong suggestions because they look close enough that it's quicker for me to fix than it is to write the whole thing out from scratch. Which, IIUC, their metric captures:

> Continued increase of the fraction of code created with AI assistance via code completion, defined as the number of accepted characters from AI-based suggestions divided by the sum of manually typed characters and accepted characters from AI-based suggestions. Notably, characters from copy-pastes are not included in the denominator.

Re: AI in software engineering at Google: Progress and the path ahead

#20

Post by HeatrayEnjoyer »

jjallen wrote: post #16
bigiain wrote: post #14

I'm looking forward to the day that some spicy autocomplete regurgitates an obvious chunk of AGPL code that it's stolen without permission or attribution - and it ends up in some critical part of Googles money printing machine, and the outside world finds out about it. I'm gonna need a _lot_ of popcorn.

How would the outside world find out about private Google code? Maybe the fact could be leaked? Unlikely though.

An insider could tip them off, then IP owner sues and Google has to produce it during discovery.
Post reply on HN