Can someone explain what the hell is going on here? Do websites want to prevent automated tooling, as indicated by everyone putting everything behind Cloudfare and CAPTCHAs since forever, or do websites want you to be able to automate things? Because I don't see how you can have both. If I'm using Selenium it's a problem, but if I'm using Claude it's fine??
In a nutshell: Google wants your websites to be more easily used by the agents they are putting in the browser and other products. They own the user layer and models, and get to decide if your product will be used. Think search monopoly, except your site doesn't even exist as far as users are concerned, it's only used via an agent, and only if Google allows. The work of implementing this is on you. Google is building…
Just one example: Prompting the browser to "register example.com" means that Google/Anthropic gets to hustle registrars for SEO-style priority. Using countermeasures like captcha locks you out of the LLM market.
Google's incentive to allow you to shop around via traditional web search is decreased since traditional ads won't be as lucrative (businesses will catch on that blanket targeted ads aren't as effective as a "referral" that directs an LLM to sign-up/purchase/exchange something directly)... expect web search quality to decline, perhaps intentionally.
The only way to combat this, as far as I can conceptualize, is with open models, which are not yet as good as private ones, in no small part due to the extraordinary investment subsidization. We can hope for the bubble to pop, but plan for a deader Internet.
Meanwhile, trust online, at large, begins to evaporate as nobody can tell what is an LLM vs a human-conducted browser. The Internet at large is entering some very dark waters.