Convenient that the LLM vendor is the one saying their own product works, isn’t it?
Company advertises their products. In other news, water is wet
Google fixed more Chrome bugs in June than over the past two years, thanks to AI
121–130 of 667 posts
Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI
#122I assume this was thanks to Project Glasswing + Mythos Preview?
Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI
#123I’d rephrase this to something like: “People fixed more Chrome bugs (…), thanks to people who leveraged AI tools”
Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI
#124Earlier quoted context omitted.
Or people don't ask questions for which the answer is known to be "None, really." I get that many don't like what LLMs are doing to the industry, but this is just incorrect reaction to a very specific benefit that's proven beyond doubt (Security hardening). Accept it imo - LLMs are solving very large problems that have plagued software security.
Sorry it is your reaction that is really weird. These are legitimate questions, and I have definitely seen Claude finding the wrong cause and then implementing completely incorrect/irrelevant fixes, only to find that it didn't work and need to start over. Not saying humans don't do the same thing, but LLMs are far from perfect, and it would be delusional to only talk about successes.
I've done security work before, and I've done it now with frontier. I've seen the difference first hand, and as the OP and so many other articles show, so have the leading experts in the world.
So either they are lying, or you may not yet be seeing and experiencing what they are. If that's wierd, ok.
Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI
#125I can't comment on Chrome bugs, but I can comment on Safari bugs. According to a search of the WebKit Bugzilla, I'm the most prolific filer of web inspector bugs outside of the WebKit team itself.
Recently I noticed an unusually high number of web inspector bugs fixed by one longtime WebKit engineer, and I suspected AI assistance, though the commit messages include no disclosure of this. Nonetheless, the increase in volume was quite dramatic.
The other day, an update to Safari Technology Preview was released, and I attempted to verify whether a couple of my bugs were indeed fixed. However, it was impossible for me to verify, because the latest version of Safari Technology Preview introduced a new web inspector bug that totally broke the features I was testing.
Thus, I remain unimpressed.
Without a doubt, LLMs have demonstrable skills and can produce code much faster than humans. I never thought that producing code fast was wise, though, even before LLMs arrived. For many years I've criticized software development based on management-driven release schedules, where developers are forced to pump out code regardless of quality, regardless of whether it's ready. Your boss may not like it, but code is done when it's done, not when your boss says it has to be done. If software complexity were predictable and reproducible, then indeed management could replace engineers with automation, but that's not how it works in reality.
Back to Chrome, the browser is notorious for continually adding invasive new "features" to the web that nobody wants except advertisers. I wonder how many Chrome bugs were the result of Google push push pushing all of this new crap on users over the years?
Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI
#126Earlier quoted context omitted.
> I've recently been using AI That's uninteresting as long as you don't specify the model you used. For example, Mythos was far better at finding security bugs than previous models.
Do you really think they mean they’ve been using mythos
Obviously no. There is also a significant capability difference e.g. between Opus 4.8 and Fable.
Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI
#127Earlier quoted context omitted.
Tests! Unit tests, integration tests, random adhoc scripts. You know - TDD! I’ve been working on UI component improvements and it was doing a lousy job until i specifically told it to test in a headless browser to validate it works. I think somewhere in an AGENTS.md i have an instruction to “don’t state your guesses as fact - validate findings and results”.
In this particular case tests won't help you, without the actual data. But overall you're right.
"tests" here could mean have a benchmark the agent can run to determinately evaluate quality.
I don't strictly mean junit unit tests.
Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI
#128I have a software degree and some work experience but have been doing other things for a decade or so.
I'm curious to try out some "vibe engineering" but it seems a bit daunting to get started, there are so many new tools available on top of the AI itself.
Are there good resources for somebody like me to get started, that'll guide me though how to think about prompting, and using CI and how go have the agent successfully write specs and tests and what-not. I don't mind paying some for a good course, but I find it hard to figure out which one would be worthwhile, or whether there are some youtubers that would be better to follow.
Any tips?
Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI
#129Earlier quoted context omitted.
You have to learn to use your tools, not try whatever intuitively made sense to you at first (expecting the tool to do all the work) and then whining on Hacker News when it doesn’t work out for you.
AI can not fail. It can only be failed. AI can not have limitations. Only the user can be limited.
Are you still speaking about a tool or starting a religion?
Re: Google fixed more Chrome bugs in June than over the past two years, thanks to AI
#130AI critique often funnels itself into a narrow bucket: creating code blindly with AI is bad. That's easy to grant. Adversarial testing, checking developer assumptions, refactor suggestions , small dev tools and even some guided coding all sit on the other side of the spectrum of what you can do with coding and AI. For larger and larger codebases even simple things like tracing dependencies or behavior might be greatl…
Yes. AI is a tool, it’s supposed to be used a certain way, anything else is a misunderstanding og what AI is. You have to aim it in the direction you want it to go, not expect it to solve all your problems magically
AI isn’t “supposed to be” anything (other than “intelligent”). It just turned out that LLMs have to be used in particular ways to be useful. Nobody planned or designed them to be that way. We all as users are put to do the job of discovering the ways.