Elephant in the room : how many of these bugs were written by LLMs in the first place ?
Because creating 100x more bugs and fixing 100x more isn't something to be proud of.
Google fixed more Chrome bugs in June than over the past two years, thanks to AI
51–60 of 668 posts
Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI
#52I am actually using AI to learn about Windows OS performance. I have a couple of prompts scheduled to research, test, and implement performance gains in the OS, then post them here [0] and here [1]. I always admired Brendan Greg and Bryan Cantrill at Sun/Illumos/ for their work on OS performance but never had the time to sit down and catch up to their level due to my time in the military.
I got Codex to find and apply a bunch of settings I didn't know about and in the future I hope to use AI to make real changes in performance that venders probably don't have time or resources for.
that being said, what I've learned so far is essentially WhyIsItAlwaysHN's entire comment "The place to find them would be performance profiles, query plans, telemetry."
I just find it so frustrating that for a user interface that was basically solved in the 90s, Windows GUI still struggles to survive on hardware exceptionally more powerful than what we had 30 years ago
0 - https://www.lacksan.com/updates/ 1 - https://github.com/Lacksan-Dev/HP-ZBook-Performance
Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI
#53I've recently been using AI a lot for performance optimisation during a particularly busy period at work. I would say it was almost completely useless at the high-level direction - it would point out suspicious parts of SQL queries for example but on back to back testing these almost never resulted in any performance change. In fact, if it wasn't for the fact that it made making the actual changes I identified much e…
As others have said, these were uninformed guesses which LLMs seem still to be willing to hand out deliberately. If you provide more information (your schemas, the distribution of values in your database tables, the EXPLAIN ANALYZE outputs, DB configuration etc.) it will do a much better job.
Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI
#54Not that I don't believe its possible to fix a lot of bugs, I also wonder what the actual dynamic was. Were the people in team working much more than usual as well? Given its Google, I wouldn't be surprised if there was an "internal push" to fix more bugs over next X sprints so that they can publish this blog and some manager can show impact and AI adaption to his superior.
I would guess they have been confronted with an initial flood of newly discovered bugs whenever they added a new analysis tool or approach (e.g. automated static analysis, fuzzing, ...). Once working through that initial flurry of newly discovered bugs one would assume that the frequency goes down again.
Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI
#55I've recently been using AI a lot for performance optimisation during a particularly busy period at work. I would say it was almost completely useless at the high-level direction - it would point out suspicious parts of SQL queries for example but on back to back testing these almost never resulted in any performance change. In fact, if it wasn't for the fact that it made making the actual changes I identified much e…
> I've recently been using AI
That's uninteresting as long as you don't specify the model you used. For example, Mythos was far better at finding security bugs than previous models.
Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI
#56I've recently been using AI a lot for performance optimisation during a particularly busy period at work. I would say it was almost completely useless at the high-level direction - it would point out suspicious parts of SQL queries for example but on back to back testing these almost never resulted in any performance change. In fact, if it wasn't for the fact that it made making the actual changes I identified much e…
The thing that makes it work really well is to make sure it has all the tooling to verify its hypotheses. If you allow it to run the full lifecycle in loops you will be surprised how well it works.
What tooling makes this go?
Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI
#57How many of those automated fixes were reverted? How many introduced a new bug? What's the false positive rate on the finding agents? The post has counts for everything that went right and nothing for what could go wrong.
> The post has counts for everything that went right and nothing for what could go wrong. That's AI for you. At Amazon we have many forums to share our AI wins, but none to share AI failures or disappoinments. No wonder execs make bad decisions regarding AI, they only hear completely one-sided stories.
At my company there's a lot of discussion about AI and complaints that people run out of tokens within a day, but zero results are shown. No measurable (or measured) gains. Or nothing that people are willing to talk about, in any case.
Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI
#58Elephant in the room : how many of these bugs were written by LLMs in the first place ? Because creating 100x more bugs and fixing 100x more isn't something to be proud of.
Can't have been much, as the Chrome project is 20+ years old and LLMs have only recently become a thing; they did not slacken their code review and testing practices with the advent of LLM code generators. And I doubt they do much development on the affected areas at the moment, they mention a 13 year old issue for example.
But it's an open source project, you can go and figure out whether your assertion is correct.
Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI
#59The biggest bug still has not been fixed here, which is: Google. We really need an alternative to this greedy and evil corporations de-facto controlling a huge portion of the modern www stack. All decision-making processes are here subjugated to what fits Google's adEmpire. This is a perpetual system of control amplifying abusive systems in place. We already see this with the recent age sniffing; Google just yesterda…
> We really need an alternative to this greedy and evil corporations de-facto controlling a huge portion of the modern www stack.
We do / have? Multiple even. Of course, the vast majority of consumers doesn't care so they don't get as much traction.
Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI
#60So will the Chromium bug 5569 finally get fixed?
I'm not sure why you're asking here instead of opening the ticket and having a look at the current status.