Google fixed more Chrome bugs in June than over the past two years, thanks to AI

Post by simonw » Fri, Jul 31, 2026, 2:41 PM UTC

dominotw wrote: post #244

herrkanin wrote: post #34

Earlier quoted context omitted.

The thing that makes it work really well is to make sure it has all the tooling to verify its hypotheses. If you allow it to run the full lifecycle in loops you will be surprised how well it works.

Very few problems really have "all the tooling to verify its hypotheses" though. even if you want to construct such an harness. Also let me ask you why we need better and better and models if what we have already can produce good output with 'all the tooling to verify its hypotheses'

Post by iamgopal » Fri, Jul 31, 2026, 2:42 PM UTC

Post by cshores » Fri, Jul 31, 2026, 2:43 PM UTC

Post by jrockway » Fri, Jul 31, 2026, 2:45 PM UTC

tzone wrote: post #251

VBprogrammer wrote: post #17

I've recently been using AI a lot for performance optimisation during a particularly busy period at work. I would say it was almost completely useless at the high-level direction - it would point out suspicious parts of SQL queries for example but on back to back testing these almost never resulted in any performance change. In fact, if it wasn't for the fact that it made making the actual changes I identified much e…

Which model are you using? There is monumental difference between models. Even between "frontier models". When people tell these stories, it would be great to also add which model you were using. As an example Opus 5.0 is in completely different class compared to Cursor Grok 4.5 even if the benchmarks don't show such massive difference. Not even talking about regular stuff like Sonnet or Composer or stuff like that.

Post by simonw » Fri, Jul 31, 2026, 2:45 PM UTC

maccard wrote: post #44

colechristensen wrote: post #28

Earlier quoted context omitted.

Then you're not grounding it to reality properly. LLMs get bad pretty quickly when you just tell them to pull the answer out of thin air. Pin them to reality with actual performance tests to run to test theories and the results will be much better. Performance optimization involves simulating real world loads and measurable results. Give them the opportunity to actually have a closed loop if you want more than trivia…

Have you any example materials that show a harness that an agent can be pointed at an app with some sort of telemetry tool, the results it gained and the cost of doing so? Because my experience is the same as the parents - the LLM goes on massive tangents, and the more tangents it goes on the worse the results get.