Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google fixed more Chrome bugs in June than over the past two years, thanks to AI

blog.google

Post reply on HN
Print view View original

61–70 of 667 posts

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#61

Post by cubefox »

ikekkdcjkfke wrote: post #15
mw888 wrote: post #3

AI critique often funnels itself into a narrow bucket: creating code blindly with AI is bad. That's easy to grant. Adversarial testing, checking developer assumptions, refactor suggestions , small dev tools and even some guided coding all sit on the other side of the spectrum of what you can do with coding and AI. For larger and larger codebases even simple things like tracing dependencies or behavior might be greatl…

Yes. AI is a tool, it’s supposed to be used a certain way, anything else is a misunderstanding og what AI is. You have to aim it in the direction you want it to go, not expect it to solve all your problems magically

LLMs are increasingly not tools anymore but agents. The difference is still one of degree, but it's clear in which direction we are currently moving. It was even clear to some people 10 years ago: https://gwern.net/tool-ai

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#62

Post by Cthulhu_ »

asdaqopqkq wrote: post #25

Did anyone check the code changes manually? What if it is just slop code?

Did you go and look at their open source repositories and open code review tools instead of fearmongering on HN? The answer can be found.

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#63

Post by cognitiveinline »

staszewski wrote: post #27
dabedee wrote: post #16

How many of those automated fixes were reverted? How many introduced a new bug? What's the false positive rate on the finding agents? The post has counts for everything that went right and nothing for what could go wrong.

[flagged]

Or people don't ask questions for which the answer is known to be "None, really."

I get that many don't like what LLMs are doing to the industry, but this is just incorrect reaction to a very specific benefit that's proven beyond doubt (Security hardening).

Accept it imo - LLMs are solving very large problems that have plagued software security.

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#64

Post by Cthulhu_ »

Oras wrote: post #23
greenknight wrote: post #8

Earlier quoted context omitted.

The Gemini (and Claude / Codex) that we have, is much different to their internal models / harnesses.....

Trust me bro?

Kind of, but you can also assert it yourself; the big tech companies have billions of lines of code not publicly available, they can train their in-house code assistants on those instead of or in addition to what's available out in the open.

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#65

Post by cognitiveinline »

albinahlback wrote: post #39
dabedee wrote: post #16

How many of those automated fixes were reverted? How many introduced a new bug? What's the false positive rate on the finding agents? The post has counts for everything that went right and nothing for what could go wrong.

And how many were introduced by AI?

It's the opposite: humans. The most critical one (sandbox escape) has been sitting there for 13 years.

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#66

Post by qsort »

sigmoid10 wrote: post #30
VBprogrammer wrote: post #17

I've recently been using AI a lot for performance optimisation during a particularly busy period at work. I would say it was almost completely useless at the high-level direction - it would point out suspicious parts of SQL queries for example but on back to back testing these almost never resulted in any performance change. In fact, if it wasn't for the fact that it made making the actual changes I identified much e…

Which model/agent/harness tool did you use? I've found what you describe was my exact experience some ~6-8 months ago, but since about a month or so the game has completely changed. Using 5.6 Sol with highest reasoning setting in Codex or Fable in Code, the models come up with a list of possible improvements from static analysis (ranked by complexity/benefit), write and run their own custom profilers and deliver sign…

Very hard to say anything definitive on this because it's a moving target, but last time I tried models still had a distinct sense of "consistently good, sometimes great at micro, bad at macro". Similar to how, even for relatively pedestrian CRUD, they'll do code that's objectively fine at the function/file/class level but can still make a mess if you don't supervise them at least at a high-level.

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#67

Post by izacus »

VBprogrammer wrote: post #17

I've recently been using AI a lot for performance optimisation during a particularly busy period at work. I would say it was almost completely useless at the high-level direction - it would point out suspicious parts of SQL queries for example but on back to back testing these almost never resulted in any performance change. In fact, if it wasn't for the fact that it made making the actual changes I identified much e…

Yep, as a performance specialist I've had the same outcomes - it'll find "smoking guns" galore which will sound plausible and be completely wrong.

But that's ok - I still get a lot of value of the tool elsewhere (e.g. writing data analysis scripts, exploring code, breaking down bug artifacts...), but it doesn't stop the tiring BS from other people going "why not just AI it?" (especially from managers).

I usually let them try to "just AI it" and most of them learn quickly that it's not that simple. (The others stay deranged and are making my job miserable.)

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#68

Post by ramanbanka »

mw888 wrote: post #3

AI critique often funnels itself into a narrow bucket: creating code blindly with AI is bad. That's easy to grant. Adversarial testing, checking developer assumptions, refactor suggestions , small dev tools and even some guided coding all sit on the other side of the spectrum of what you can do with coding and AI. For larger and larger codebases even simple things like tracing dependencies or behavior might be greatl…

AI has skills no individual can have. That part is true.

But AI is not smarter than you, AI is as smart as the user who uses it. It often makes wrong decisions unless user corrects it.

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#69

Post by goldenarm »

Cthulhu_ wrote: post #58
goldenarm wrote: post #51

Elephant in the room : how many of these bugs were written by LLMs in the first place ? Because creating 100x more bugs and fixing 100x more isn't something to be proud of.

Can't have been much, as the Chrome project is 20+ years old and LLMs have only recently become a thing; they did not slacken their code review and testing practices with the advent of LLM code generators. And I doubt they do much development on the affected areas at the moment, they mention a 13 year old issue for example. But it's an open source project, you can go and figure out whether your assertion is correct.

The single 13yo issue is anecdata.

They obviously have the full git blame statistics but chose not to include them in the blog post, which is a bit concerning.

Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI

#70

Post by izacus »

glimshe wrote: post #41

A lot of people here seem to be living in a different universe than me or simply don't know how to work with AI. I think detractors believe you should just let AI do the job blindly instead of leveraging it as a tool to accelerate you. They get mad at Excel for the poor investment returns. At this point, this is such a strawman, it isn't worth counter arguing. I think I'll abandon this discussion and keep using AI qu…

Did you think for a second that maybe other people work with other languages, domains, tools and work approaches than you?

It's so bizarre that noone realizes that there's a bit of a difference between banging together JavaScript into websites, writing firmware for widgets, herding servers or fixing medical devices?

Post reply on HN