Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI in software engineering at Google: Progress and the path ahead

research.google

Post reply on HN
Print view View original

141–150 of 245 posts

Re: AI in software engineering at Google: Progress and the path ahead

#141

Post by discreteevent »

reacharavindh wrote: post #115
WalterSear wrote: post #109

Earlier quoted context omitted.

IMHO, review is a misnomer for where software engineering is going. I'm not sure where we are going, but review implies less responsibility for the outcome. But I do think that we will have less depth of knowledge of the underlying processes. That's the point of having a machine do it. I expect this, however, to be a good trend: the systems will need to be up to a task before it makes sense to rely on them.

This is how progress (in developer productivity) has always been made. We coded in assembler, then used macros, then a language like C, Fortran, then more of Java/Go/Puthon/Rust/Ruby et al. A developer writing a for loop over a list in Python need to necessarily know about linked lists and memory patterns because Python takes care of it. This frees up that developer from abstracted details and think one level closer…

> We coded in assembler, then used macros, then a language like C, Fortran, then more of Java/Go/Puthon/Rust/Ruby et al.

The difference is that:

1) All of those things are deterministic [1]

2) In all of those cases I can debug at the level of the abstraction.

[1] Meaning: Do exactly what I say. Don't make it up.

Re: AI in software engineering at Google: Progress and the path ahead

#142

Post by jillesvangurp »

LLMs are undeniably useful for programming. The core challenge in making them more useful is the right UX for making this more seamless. I use intellij and things like codegpt.

A few weeks ago they enabled auto complete. I disabled it after a day. Reason: most of the suggestions weren't great and it drowned out the traditional auto complete, which I depend on. I just found the whole thing too distracting.

I also have the chat gpt desktop app installed since a few weeks. This adds a key binding (option + space) to be able to ask it questions. I've found myself using it more because of that. Copy some code, option+space, ask it a question and there it goes.

The main issue is that it is a bit ground hog day in the sense that I have to explain it in detail what I want every time. I can't just ask it to generate a unit test (which it does very well). Instead I have to specify that I want it to generate a unit test, use kotlin-test and kotest-assertions, and not use backticks for the function names (doesn't work with kotlin-js). Every time. If I don't it will just default to the wrong things because it doesn't remember your preferences for frameworks, style, habits, etc. and it doesn't look at the whole code base to infer any of that.

Mostly, progress here is going to come in the form of better IDE support and UX. The right key bindings would help. A bigger context so it can just grab your whole git project and be able to suggest appropriate code using all the right idioms, frameworks, etc. would then be possible. Additionally, it would be able to generate files and directories as needed and fill them with all the right stuff. I don't think that's going to take very long. Gpt-4o already has a quite large context and with the progress in OSS models, we might be doing some of this stuff locally pretty soon.

Re: AI in software engineering at Google: Progress and the path ahead

#143

Post by longerd2 »

lifty wrote: post #129
nomel wrote: post #38

Earlier quoted context omitted.

First we extended the reach of our perception with language. Then we extended the energy in our calories reserves with crops/livestock. Then we extended the length of our memories with writing. Then we extended the breadth of our thinking with AI?

Then we extend the vastness of our consciousness with the Borg.

please let me leave my flesh prison

Re: AI in software engineering at Google: Progress and the path ahead

#144

Post by williamcotton »

jprete wrote: post #6
trybackprop wrote: post #2

From the blog post: > We observe that with AI-based suggestions, the code author increasingly becomes a reviewer, and it is important to find a balance between the cost of review and added value. We typically address the tradeoff with acceptance rate targets. In the past year since GPT-4 came out, I've also found this to be the case. I'm an ML/backend engineer with little experience in frontend development. Yet, I've…

This isn't a good trend. Reviewers never have the depth of knowledge or understanding of authors.

This is a false equivalence.

When someone is using an LLM they are still the author.

Think about it like someone who is searching through record crates for a good sample. They're going to "review" certain drum breaks and then decide if it should be included in an artwork.

The reviewing that you're alluding to is like a book reviewer who has nothing to do with the finished product.

Re: AI in software engineering at Google: Progress and the path ahead

#145

Post by williamcotton »

_the_inflator wrote: post #95
jprete wrote: post #28

Earlier quoted context omitted.

I know of no review process that produces the same level of understanding as does authorship, because the author must build the model from scratch and so must see all the details, while the reviewer is able to do less work because they're fundamentally riding on the author's understanding. In fact, in a high-trust system, e.g. a good engineering culture in a tech company, the reviewer will learn even less, because th…

I totally agree. Reviewing the solution is limited. What you don’t get are the myriads of other ways that didn’t work out. Elegant solutions are the result of weeding out dozens of other messy ways. So what gets perpetuated here then is the Dunnimg Kruger effect. While it might be speed things up in many normal circumstances, it devalues hard work in the long run. Not good.

I've found that it is a force multiplier for my hard work.

Re: AI in software engineering at Google: Progress and the path ahead

#146

Post by belter »

suyash wrote: post #94

So how long till AI will be fully replacing a SWE at Google? That is where the drive for productivity at organizations like Google are leading towards.

Not if they use Gemini Advanced...It's so far behind Claude or ChatpGPT4 as to be laughable.

Re: AI in software engineering at Google: Progress and the path ahead

#147

Post by williamcotton »

ks2048 wrote: post #15
cloudking wrote: post #3

The most interesting chart is the "fraction of code created with AI assistance via code completion" trending up to 50%

Seems they need to compare against "dumb" code completion. It seems that even when they are error-free, "large" AI-code-completions are just boilerplate that should be abstracted away in some functions rather than inserted into your code base. On a related note, maybe they should measure number of code characters that can be REMOVED by AI rather than inserted!

boilerplate that should be abstracted away in some functions rather than inserted into your code base

Boilerplate is often tedious to write and just as often easy to read. Abstraction puts more cognitive load on the developer and sometimes this is not worth the impact on legibility.

Re: AI in software engineering at Google: Progress and the path ahead

#148

Post by ffsm8 »

gerdesj wrote: post #39
runlevel1 wrote: post #12

Humans have limited RAM, so we have to put our ideas into an external medium that can then be refined. I've been finding AI's suggestions -- even when rather wrong -- help me do that initial step faster. Which, I think, jives with their findings here.

"Humans have limited RAM" I would suggest we have flexible RAM. Also, we have an awful lot of it. The analogy breaks down as soon as you look at it too seriously! In IT we largely deal with compute, persistent storage and non-persistent storage. Roughly speaking: CPU, RAM, HDD. In humans we might be considered to have similar "abilities" but unlike IT there is a mostly a single thing that performs all of those functi…

These kinds of similies make less and less sense nowadays because we've got nvme storage nowadays, and that can be as fast as 7GByte/s. That's a lot faster then the RAM in most devices today. And with less latency too.

RAMs differentiating factor is increasingly just that it can handle a lot of read/write cycles, not it's speed. And that doesn't map to anything in biology

Re: AI in software engineering at Google: Progress and the path ahead

#149

Post by mycologos »

_the_inflator wrote: post #95
jprete wrote: post #28

Earlier quoted context omitted.

I know of no review process that produces the same level of understanding as does authorship, because the author must build the model from scratch and so must see all the details, while the reviewer is able to do less work because they're fundamentally riding on the author's understanding. In fact, in a high-trust system, e.g. a good engineering culture in a tech company, the reviewer will learn even less, because th…

I totally agree. Reviewing the solution is limited. What you don’t get are the myriads of other ways that didn’t work out. Elegant solutions are the result of weeding out dozens of other messy ways. So what gets perpetuated here then is the Dunnimg Kruger effect. While it might be speed things up in many normal circumstances, it devalues hard work in the long run. Not good.

The Dunning-Kruger effect, and its many misspellings, might be the most overused term on this website.

Re: AI in software engineering at Google: Progress and the path ahead

#150

Post by CoastalCoder »

discreteevent wrote: post #141
reacharavindh wrote: post #115

Earlier quoted context omitted.

This is how progress (in developer productivity) has always been made. We coded in assembler, then used macros, then a language like C, Fortran, then more of Java/Go/Puthon/Rust/Ruby et al. A developer writing a for loop over a list in Python need to necessarily know about linked lists and memory patterns because Python takes care of it. This frees up that developer from abstracted details and think one level closer…

> We coded in assembler, then used macros, then a language like C, Fortran, then more of Java/Go/Puthon/Rust/Ruby et al. The difference is that: 1) All of those things are deterministic [1] 2) In all of those cases I can debug at the level of the abstraction. [1] Meaning: Do exactly what I say. Don't make it up.

I agree with your overall point.

In a certain sense I'd say optimizing compilers aren't deterministic:

The same source code can produce different object code, depending on data and algorithms into which a typical programmer has little insight.

Post reply on HN