Live data from Hacker News

AI in software engineering at Google: Progress and the path ahead

research.google

91–100 of 245 posts

Re: AI in software engineering at Google: Progress and the path ahead

#91

Earlier quoted context omitted.

I’m not seeing any evidence that any of this is actually good for Google’s business. Their observations that half of code checked in is from suggestions by an LLM is not really surprising in a regimented dev platform with tons of boilerplate. That stat tells us nothing about actual code quality, development velocity, or skill curves over time, much less business impact. What product of Google’s has been improved by t…

Xoogler here. This exactly. There is so much boilerplate involved in writing anything inside Google. AI was great to cut that down a bit. It's still nowhere near what it's like in the outside and/or non-Java world. Which isn't to say that this isn't progress - just that that stat should be taken with context.

Yeah I kind of agree that when LLMs work REALLY well for autocomplete of your codebase -- that might be an indication that the language and library abstractions you use don't fit the problem very well.

Code is read more than it's written. And it should be written to be read.

If you are barfing out a lot of auto-completed stuff, it's probably not very easy to read.

You have to read code to maintain it, modify it, analyze its performance, handle production incidents, etc.

Re: AI in software engineering at Google: Progress and the path ahead

#92

Humans have limited RAM, so we have to put our ideas into an external medium that can then be refined. I've been finding AI's suggestions -- even when rather wrong -- help me do that initial step faster. Which, I think, jives with their findings here.

> I've been finding AI's suggestions -- even when rather wrong -- help me do that initial step faster.

I have no idea how I could even integrate AI into my workflow so that it's useful. It's even less reliable than search is for basic research and can't even cite its sources....

This argument held a lot more weight when it was a search engine playing the role of our memory.

Re: AI in software engineering at Google: Progress and the path ahead

#93
post #59
post #39

Earlier quoted context omitted.

"Humans have limited RAM" I would suggest we have flexible RAM. Also, we have an awful lot of it. The analogy breaks down as soon as you look at it too seriously! In IT we largely deal with compute, persistent storage and non-persistent storage. Roughly speaking: CPU, RAM, HDD. In humans we might be considered to have similar "abilities" but unlike IT there is a mostly a single thing that performs all of those functi…

Just because the brain is a single “thing” doesn’t mean it doesn’t have distinct types of memory. Consider looking up “working memory” as it’s probably the best analogue to RAM here.

Not really, it's more like CPU registers. Very limited and stuff has to be in their to be computed on (consciously) (lots of unconscious computation as well of course; a lot of efficiency to be found in moving computation from conscious to unconscious).

Re: AI in software engineering at Google: Progress and the path ahead

#95
post #28

Earlier quoted context omitted.

Source? This is unintuitive to me, I can't come up with a rationale.

I know of no review process that produces the same level of understanding as does authorship, because the author must build the model from scratch and so must see all the details, while the reviewer is able to do less work because they're fundamentally riding on the author's understanding. In fact, in a high-trust system, e.g. a good engineering culture in a tech company, the reviewer will learn even less, because th…

I totally agree.

Reviewing the solution is limited. What you don’t get are the myriads of other ways that didn’t work out.

Elegant solutions are the result of weeding out dozens of other messy ways.

So what gets perpetuated here then is the Dunnimg Kruger effect.

While it might be speed things up in many normal circumstances, it devalues hard work in the long run. Not good.

Re: AI in software engineering at Google: Progress and the path ahead

#96
post #6

From the blog post: > We observe that with AI-based suggestions, the code author increasingly becomes a reviewer, and it is important to find a balance between the cost of review and added value. We typically address the tradeoff with acceptance rate targets. In the past year since GPT-4 came out, I've also found this to be the case. I'm an ML/backend engineer with little experience in frontend development. Yet, I've…

This isn't a good trend. Reviewers never have the depth of knowledge or understanding of authors.

Depends on what's being authored. Many times more experienced engineers review PRs from junior engineers and do have a higher level of knowledge.

Re: AI in software engineering at Google: Progress and the path ahead

#99

As others have mentioned, unless you have a strong conscience and really know what you're doing, it's far too tempting to just accept AI-generated suggestions without really thinking, and IMHO losing that understanding is a dangerous path to go down. AI can only increase quantity, not quality. The industry desperately needs far more of the latter. Related: https://navendu.me/posts/ai-generated-spam-prs/

> As others have mentioned, unless you have a strong conscience and really know what you're doing

I don't have a strong conscience - I just don't trust it to do anything right.

Re: AI in software engineering at Google: Progress and the path ahead

#100
post #6

Earlier quoted context omitted.

This isn't a good trend. Reviewers never have the depth of knowledge or understanding of authors.

Source? This is unintuitive to me, I can't come up with a rationale.

It is similar to how much a student learns from working hard to solve a problem versus from being given the final solution. The effort to solve it yourself tends to give a deeper understanding and make it easier to remember.

Maybe that's not the case in all fields, but it is in my experience at least, including in software. Code I've written I know on a personal level, while I can much more easily forget code I've only reviewed.

Post reply on HN