Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google fixed more Chrome bugs in June than over the past two years, thanks to AI

blog.google

Post reply on HN
Print view View original

31–40 of 668 posts

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#31

Post by NitpickLawyer »

truncate wrote: post #5

Not that I don't believe its possible to fix a lot of bugs, I also wonder what the actual dynamic was. Were the people in team working much more than usual as well? Given its Google, I wouldn't be surprised if there was an "internal push" to fix more bugs over next X sprints so that they can publish this blog and some manager can show impact and AI adaption to his superior.

Google's entire modus operandi has been "automate everything" for decades. They've been doing this with fuzzers, with project zero and so on. Adding LLMs on top is a very obvious next step. And LLMs improving and finding more bugs also follows. Then improve the harness and the dev tools, to better use the LLMs. And then everything together end-to-end to find-triage-fix-confirm. Your LLMs are as good as the loop they run in, and the loop is as good as the verifier. Seems a reasonable enough dynamic without (or despite, depends how cynical you wanna be) the need for managers to show number goes up on some chart.

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#32

Post by flohofwoe »

WhyIsItAlwaysHN wrote: post #29
VBprogrammer wrote: post #17

I've recently been using AI a lot for performance optimisation during a particularly busy period at work. I would say it was almost completely useless at the high-level direction - it would point out suspicious parts of SQL queries for example but on back to back testing these almost never resulted in any performance change. In fact, if it wasn't for the fact that it made making the actual changes I identified much e…

The mistake there is to point it at code to figure out performance optimizations. The place to find them would be performance profiles, query plans, telemetry. The guidance for perf still applies, measure before and after change. The issue is that the code often does not contain the information to do a perf optimization. Eg. you can't tell your cache size, the volumes of data in your DB or the latency of your network…

> The place to find them would be performance profiles, query plans, telemetry.

Tbh, once this information is available (which is the tricky part), in 99% of cases there's really no AI needed to analyze the data, since the 'low hanging fruits' will usually stand out anyway. And once you get into the area where optimization hotspots are no longer obvious, you're already deep in the diminishing returns area and optimizations for one use case or hardware configuration may degrade performance on others. That's my experience anyway.

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#34

Post by herrkanin »

VBprogrammer wrote: post #17

I've recently been using AI a lot for performance optimisation during a particularly busy period at work. I would say it was almost completely useless at the high-level direction - it would point out suspicious parts of SQL queries for example but on back to back testing these almost never resulted in any performance change. In fact, if it wasn't for the fact that it made making the actual changes I identified much e…

The thing that makes it work really well is to make sure it has all the tooling to verify its hypotheses. If you allow it to run the full lifecycle in loops you will be surprised how well it works.

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#35

Post by brainwad »

asdaqopqkq wrote: post #25

Did anyone check the code changes manually? What if it is just slop code?

I don't know about Chromium specifically, but in general Google requires 2 humans to have looked at every change (normally one is the author, but not necessarily for bot-authored changes).

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#36

Post by dewey »

flohofwoe wrote: post #32
WhyIsItAlwaysHN wrote: post #29

Earlier quoted context omitted.

The mistake there is to point it at code to figure out performance optimizations. The place to find them would be performance profiles, query plans, telemetry. The guidance for perf still applies, measure before and after change. The issue is that the code often does not contain the information to do a perf optimization. Eg. you can't tell your cache size, the volumes of data in your DB or the latency of your network…

> The place to find them would be performance profiles, query plans, telemetry. Tbh, once this information is available (which is the tricky part), in 99% of cases there's really no AI needed to analyze the data, since the 'low hanging fruits' will usually stand out anyway. And once you get into the area where optimization hotspots are no longer obvious, you're already deep in the diminishing returns area and optimiz…

> there's really no AI needed once this information is available

Most people are not arguing that problems are too tricky for a human to solve once presented with it, but AI can look at 1000 things at the same time across a whole code base and then just come back with the results a human can review.

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#37

Post by brainwad »

greenknight wrote: post #8
ed_mercer wrote: post #6

AI meaning Claude or Codex, because Gemini is a laughingstock for coding. Also Gemini was mentioned only once in the article.

The Gemini (and Claude / Codex) that we have, is much different to their internal models / harnesses.....

The latest and greatest internally at Google is more or less just Antigravity 2.0 with the latest Gemini models.

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#38

Post by iLoveOncall »

dabedee wrote: post #16

How many of those automated fixes were reverted? How many introduced a new bug? What's the false positive rate on the finding agents? The post has counts for everything that went right and nothing for what could go wrong.

> The post has counts for everything that went right and nothing for what could go wrong.

That's AI for you.

At Amazon we have many forums to share our AI wins, but none to share AI failures or disappoinments.

No wonder execs make bad decisions regarding AI, they only hear completely one-sided stories.

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#39

Post by albinahlback »

dabedee wrote: post #16

How many of those automated fixes were reverted? How many introduced a new bug? What's the false positive rate on the finding agents? The post has counts for everything that went right and nothing for what could go wrong.

And how many were introduced by AI?

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#40

Post by kypro »

VBprogrammer wrote: post #17

I've recently been using AI a lot for performance optimisation during a particularly busy period at work. I would say it was almost completely useless at the high-level direction - it would point out suspicious parts of SQL queries for example but on back to back testing these almost never resulted in any performance change. In fact, if it wasn't for the fact that it made making the actual changes I identified much e…

I've noticed the same thing.

The trend I've noticed is that AI struggles to think outside the box when making optimisations, which exactly what's needed when you've made all of the practical DB and logic optimisations to the existing code.

Often you need to take a step back and question how the system is working and if there would be better ways to design it so the bottlenecks you're hitting wouldn't exist in the first place. Caching things, adding indexes, tweaking logic – these can help, but you'll quickly hit diminishing returns once you've done all of the obvious stuff.

I've seen people here say how AI is great at optimising code though, but I'm not sure if that's because they're giving it optimisation problems with a lot of low hanging fruit or if they're successfully getting AI to rework their systems to remove bottlenecks. This one area I find AI to still be particularly bad at.

Post reply on HN