Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI in software engineering at Google: Progress and the path ahead

research.google

Post reply on HN
Print view View original

151–160 of 245 posts

Re: AI in software engineering at Google: Progress and the path ahead

#151

Post by DebtDeflation »

bamboozled wrote: post #112
knallfrosch wrote: post #105

Earlier quoted context omitted.

Assuming Google has tens of thousands of software engineers (for a lower bound of 10,000) and artificial intelligence increases productivity by at least 0.01%, the first engineer has already been replaced.

So how long till AI will be fully replacing a SWE at Google? Notice parent said "fully"

It's the wrong question.

It's like asking "How long until a bulldozer [1] fully replaces a human construction worker?" A bulldozer is not a full replacement for a human construction worker. However, 1 worker with a bulldozer can do the work of 10 workers with shovels. And 10 workers with bulldozers can build things no amount of workers with shovels can.

[1] I'm using bulldozer as shorthand for all automated construction equipment - front end loaders, backhoes, cranes, etc.

Re: AI in software engineering at Google: Progress and the path ahead

#152

Post by DannyBee »

tmpz22 wrote: post #25
cloudking wrote: post #3

The most interesting chart is the "fraction of code created with AI assistance via code completion" trending up to 50%

Question for any Googlers in the thread - do folks speak up if they see flaws in the methodology or approach of this research or is the pressure from the top so strong on this initiative that people hush up?

Fwiw, ai code completion initiatives at Google well predate current hype (ie overall research path, etc, was started a long time ago, though obviously modified as capability has changed).

So this particular thing was a very well established program, path, and methodology by the time AI hype came.

Whether that is good or bad I won't express an opinion, but it might mean you get a different answer to your question for this particular thing.

Re: AI in software engineering at Google: Progress and the path ahead

#153

Post by danpalmer »

endisneigh wrote: post #135
danpalmer wrote: post #101

Earlier quoted context omitted.

I have AI suggested edits turned off. It's the 4th setting down in the settings menu. AI suggestions in Cider can be disabled. And all of these things are web apps for which googlers have a rich history of creating Chrome extensions to change things they don't like or disagree with. I don't feel it's obvious at all that this will become forced. What would that even mean – you don't get to edit code anymore but can on…

You can turn off giving ai suggestions, not receiving them. Not all AI affordances in cider can be disabled. There are plenty of people complaining about this on yaqs and buganizer. > no one is forced to use them so at worst they're a minor UI annoyance. This attitude is exactly the problem with Google and is why Google’s AI rollout has been terrible thus far. Luckily the stock is still going up, so whatever I guess.

You don't receive AI suggestions in code review, you receive suggestions from your reviewers. If they are providing bad recommendations, regardless of the source, I suggest providing them feedback.

I agree this may not be how they are used, but honestly, code review is a skill and many people are bad at it. Blindly suggesting things because an AI (or a presubmit) suggests it is bad form regardless of the source. People slam that "please fix" button without looking at what it's asking for, or without reading my detailed explanations of why I'm ignoring a recommendation, etc. This was already a problem and AI isn't changing that.

We can make code review better, but that comes through training reviewers better, not by ruling out AI tools just because they're AI tools.

Re: AI in software engineering at Google: Progress and the path ahead

#154

Post by endisneigh »

danpalmer wrote: post #153
endisneigh wrote: post #135

Earlier quoted context omitted.

You can turn off giving ai suggestions, not receiving them. Not all AI affordances in cider can be disabled. There are plenty of people complaining about this on yaqs and buganizer. > no one is forced to use them so at worst they're a minor UI annoyance. This attitude is exactly the problem with Google and is why Google’s AI rollout has been terrible thus far. Luckily the stock is still going up, so whatever I guess.

You don't receive AI suggestions in code review, you receive suggestions from your reviewers. If they are providing bad recommendations, regardless of the source, I suggest providing them feedback. I agree this may not be how they are used, but honestly, code review is a skill and many people are bad at it. Blindly suggesting things because an AI (or a presubmit) suggests it is bad form regardless of the source. Peop…

> You don't receive AI suggestions in code review, you receive suggestions from your reviewers

If a reviewer makes a comment, Critique will create an AI suggestion based on their comment, even if they didn’t explicitly do so (unless they turn off giving AI suggestions on their end, but there’s no way to stop from seeing it from the other).

> not by ruling out AI tools just because they're AI tools.

This was not my point at all.

The point is that Google force feeds AI tools, and makes it difficult to opt out. In many cases it is not possible as shown with GenAI search.

Re: AI in software engineering at Google: Progress and the path ahead

#155

Post by anonyfox »

jprete wrote: post #6
trybackprop wrote: post #2

From the blog post: > We observe that with AI-based suggestions, the code author increasingly becomes a reviewer, and it is important to find a balance between the cost of review and added value. We typically address the tradeoff with acceptance rate targets. In the past year since GPT-4 came out, I've also found this to be the case. I'm an ML/backend engineer with little experience in frontend development. Yet, I've…

This isn't a good trend. Reviewers never have the depth of knowledge or understanding of authors.

not every developer knows how exactly his modern CPU oder memory layers work, or how electromagnetic waves are building up a signal.

people use tools to make things. Its okay. Some "hardcore folks" advance the "lower level" tooling, other creative folks build actually useful things for daily life, and mostly these two groups have very little overlap IMO.

Re: AI in software engineering at Google: Progress and the path ahead

#156

Post by sumeno »

zackproser wrote: post #49
skywhopper wrote: post #41

Earlier quoted context omitted.

I’m not seeing any evidence that any of this is actually good for Google’s business. Their observations that half of code checked in is from suggestions by an LLM is not really surprising in a regimented dev platform with tons of boilerplate. That stat tells us nothing about actual code quality, development velocity, or skill curves over time, much less business impact. What product of Google’s has been improved by t…

What you say tracks - but I'm wondering what happens if they manage to unlock some meaningful velocity increase to the point where they can begin tackling other domains and shuttling out products at a higher rate...Agree with your thoughts on search - it's f'ing unusable now and frustrating to look at.

Coding velocity is not a barrier to google tackling other domains.

Re: AI in software engineering at Google: Progress and the path ahead

#157

Post by [deleted] »

discreteevent wrote: post #141
reacharavindh wrote: post #115

Earlier quoted context omitted.

This is how progress (in developer productivity) has always been made. We coded in assembler, then used macros, then a language like C, Fortran, then more of Java/Go/Puthon/Rust/Ruby et al. A developer writing a for loop over a list in Python need to necessarily know about linked lists and memory patterns because Python takes care of it. This frees up that developer from abstracted details and think one level closer…

> We coded in assembler, then used macros, then a language like C, Fortran, then more of Java/Go/Puthon/Rust/Ruby et al. The difference is that: 1) All of those things are deterministic [1] 2) In all of those cases I can debug at the level of the abstraction. [1] Meaning: Do exactly what I say. Don't make it up.

[deleted]

Re: AI in software engineering at Google: Progress and the path ahead

#158

Post by mewpmewp2 »

jprete wrote: post #6
trybackprop wrote: post #2

From the blog post: > We observe that with AI-based suggestions, the code author increasingly becomes a reviewer, and it is important to find a balance between the cost of review and added value. We typically address the tradeoff with acceptance rate targets. In the past year since GPT-4 came out, I've also found this to be the case. I'm an ML/backend engineer with little experience in frontend development. Yet, I've…

This isn't a good trend. Reviewers never have the depth of knowledge or understanding of authors.

Well it is not actually review where you have a PR. It is more like you are guiding and reviewing a very fast typer in your decided order that in any simple cases handles it 99 percent of the time.

Re: AI in software engineering at Google: Progress and the path ahead

#159

Post by vitiral »

CoastalCoder wrote: post #150
discreteevent wrote: post #141

Earlier quoted context omitted.

> We coded in assembler, then used macros, then a language like C, Fortran, then more of Java/Go/Puthon/Rust/Ruby et al. The difference is that: 1) All of those things are deterministic [1] 2) In all of those cases I can debug at the level of the abstraction. [1] Meaning: Do exactly what I say. Don't make it up.

I agree with your overall point. In a certain sense I'd say optimizing compilers aren't deterministic: The same source code can produce different object code, depending on data and algorithms into which a typical programmer has little insight.

Being a bit pedantic. Globally but not locally deterministic

Re: AI in software engineering at Google: Progress and the path ahead

#160

Post by mattgreenrocks »

mycologos wrote: post #149
_the_inflator wrote: post #95

Earlier quoted context omitted.

I totally agree. Reviewing the solution is limited. What you don’t get are the myriads of other ways that didn’t work out. Elegant solutions are the result of weeding out dozens of other messy ways. So what gets perpetuated here then is the Dunnimg Kruger effect. While it might be speed things up in many normal circumstances, it devalues hard work in the long run. Not good.

The Dunning-Kruger effect, and its many misspellings, might be the most overused term on this website.

The industry is rife with it, and this is a haven for experienced professionals as well as newcomers.

One need only look at open source projects to see the massive variance in code quality that is out there.

Post reply on HN