Live data from Hacker News

Google fixed more Chrome bugs in June than over the past two years, thanks to AI

blog.google

241–250 of 667 posts

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#241
The privatization and obfuscation of the security industry is worrying.

Google claims to help open source developers, but donates millions to the Alpha Omega Foundation that did not give access to Mythos to Curl and only found one issue.

The Alpha Omega people appear to be selected on physical appearance and take the money away from open source without doing much.

The entire Chrome article is obviously directed at suits and unsurprisingly hypes AI for better promotion chances.

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#242
post #5

Not that I don't believe its possible to fix a lot of bugs, I also wonder what the actual dynamic was. Were the people in team working much more than usual as well? Given its Google, I wouldn't be surprised if there was an "internal push" to fix more bugs over next X sprints so that they can publish this blog and some manager can show impact and AI adaption to his superior.

This is just my hypothesis, but I suspect that at Chrome‘s engineering team there has been a culture of inaction for the past decade or more. That nothing gets done, no bugs fixed, unless some higher ups at Google sees a business interest in resolving it.

And what has changed now is that the higher ups at Google do indeed see a business interest in fixing bugs and giving the credit to AI to sell us more AI.

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#243
post #210

> Google fixed more Chrome bugs in June than over the past two years, thanks to AI And introduced how many new bugs? thanks to AI ...

At this time, a generic AI snark is tiring. AI is not perfect, but it's an amazingly useful tool, we should accept it and move on.

AI hype is also tiring.

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#244

I've recently been using AI a lot for performance optimisation during a particularly busy period at work. I would say it was almost completely useless at the high-level direction - it would point out suspicious parts of SQL queries for example but on back to back testing these almost never resulted in any performance change. In fact, if it wasn't for the fact that it made making the actual changes I identified much e…

The thing that makes it work really well is to make sure it has all the tooling to verify its hypotheses. If you allow it to run the full lifecycle in loops you will be surprised how well it works.

Very few problems really have "all the tooling to verify its hypotheses" though. even if you want to construct such an harness.

Also let me ask you why we need better and better and models if what we have already can produce good output with 'all the tooling to verify its hypotheses'

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#245

Earlier quoted context omitted.

The thing that makes it work really well is to make sure it has all the tooling to verify its hypotheses. If you allow it to run the full lifecycle in loops you will be surprised how well it works.

Just like humans, AI needs a "reality" to check things against. If we don't provide that in an automated way, somehow, it will always make mistakes and we'd have to point it out. Since, at that point, we're just mediating with what reality gives us and what an AI should do with it. AI doesn't have enough senses yet. It's trapped in a box.

The AI box experiment has been resolved. We let the AI out of the box if it promises to summarise and desummarise emails.

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#247

Earlier quoted context omitted.

And how many were introduced by AI?

It's the opposite: humans. The most critical one (sandbox escape) has been sitting there for 13 years.

The question is how many new bugs were introduced through those AI fixes.

Is it less than a human would have? Is it more? Is it the same type of bugs?

I could be convinced either way and it's an interesting thing to ponder in my opinion.

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#250
post #21
post #5

Not that I don't believe its possible to fix a lot of bugs, I also wonder what the actual dynamic was. Were the people in team working much more than usual as well? Given its Google, I wouldn't be surprised if there was an "internal push" to fix more bugs over next X sprints so that they can publish this blog and some manager can show impact and AI adaption to his superior.

1. Our backlog of bugs gets processed quicker because instead of staring at the code for 10 minutes fiuring out what's happening, there's a tool that can reason about it quicker. 2. Code reviews and security reviews happen quicker and produce more findings. I would think that (m)any team(s) using AI might also be seeing a higher rate of finding and fixing issues. Even the Linux Kernel (I'd say Windows and Apple too)…

Linus: "it keeps finding embarrassing bugs"

Linux Kernel: https://lore.kernel.org/all/CAHk-=wi4zC+Ze8e+p3tMv8TtG_80Kzs...

The idea that software has gotten so complex that a machine can evaluate code paths better than a human, seems to bristle the fur of many. Some people didn't think we would see the day where that comparative human limitation was laid bare in simpler tasks than they expected. I believe older developers are less likely to be offended, having to deal with this as a matter of course (as the mind declines).

Post reply on HN