Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude can now search the web

anthropic.com

Post reply on HN
Print view View original

101–110 of 758 posts

Re: Claude can now search the web

#101

Post by jfim »

losteric wrote: post #83
1shooner wrote: post #71

Earlier quoted context omitted.

>You can now use Claude to search the internet to provide more up-to-date and relevant responses. It's a search engine. You 'ask it to read the web' just like you asked Google to, except Google used to actually give the website traffic. I appreciate the concept of an AI User-agent, but without a business model that pays for the content creation, this is just going to lead to the death of anonymously accessible conten…

What was the web like before wide spread internet ads, auth, and search engines? Did all those old sites have “business models”? What did the web feel like back then? (This is rhetorical - I had niche hobby sites back then, in the same way some people put out free zines, and wouldn’t give a damn about today’s AI agents so long as they were respectful. The web was better back then, and I believe AI slop and agents bri…

It was much smaller and people wrote on Usenet to connect with one another, not to shout in the void while corporations hoover all the content.

Re: Claude can now search the web

#102

Post by tcdent »

Searching the web is a great feature in theory, but every implementation I've used so far looks at the top X hits and then interprets it to be the correct answer.

When you're talking to an LLM about popular topics or common errors, the top results are often just blogspam or unresolved forum posts, so the you never get an answer to your problem.

More of an indicator that web search is more unusable than ever, but interesting that it affects the performance of generative systems, nonetheless.

Re: Claude can now search the web

#103

Post by jrflowers »

TheDudeMan wrote: post #78
mvdtnz wrote: post #68

Earlier quoted context omitted.

No thank you, when I define a robots.txt file I expect all automated systems to respect it.

But this isn't automated. This is user-driven.

If this feature isn’t already part of the Claude API it likely will be at some point, in which case many Claude requests will be automated with no way to distinguish between user-driven or otherwise.

Re: Claude can now search the web

#104

Post by CalChris »

I find myself Googling less often these days. Frustrated with both the poor search results and impressed with the quality of AI to do the same thing and more, I think search's days are numbered. AOL lasted as an email address for quite some time after America Online ceased to be a relevant portal. Maybe Gmail will as well.

Re: Claude can now search the web

#105

Post by rustc »

geekrax wrote: post #25
micromacrofoot wrote: post #14

Earlier quoted context omitted.

almost no one does, robots.txt is practically a joke at this point — right up there with autocomplete=off

I have replaced all robots.txt rules with simple WAF rules, which are cheaper to maintain than dealing with offending bots.

The WAF rule matches based on the user agent header? Perplexity is known to use generic browser user agents to bypass that.

Re: Claude can now search the web

#106

Post by elefanten »

darepublic wrote: post #75
1shooner wrote: post #71

Earlier quoted context omitted.

>You can now use Claude to search the internet to provide more up-to-date and relevant responses. It's a search engine. You 'ask it to read the web' just like you asked Google to, except Google used to actually give the website traffic. I appreciate the concept of an AI User-agent, but without a business model that pays for the content creation, this is just going to lead to the death of anonymously accessible conten…

Well I expect eventually the agent will be able to act on your behalf with your credentials.

And as advertisers get declining human views on their ads, the value of the business model will dwindle until it needs to be replaced by other forms of revenue. Content that can't shift business models and requires revenue to continue will die off.

Edit: Maybe that's fine, maybe that's bad. Maybe new models will emerge and things will reshape. But I'm just supporting the case that AI agents will pressure the current "free" content economy.

Re: Claude can now search the web

#107

Post by Tostino »

haswell wrote: post #88
Filligree wrote: post #64

Earlier quoted context omitted.

If a human triggers a browser by pressing a button, should it ignore robots.txt?

Are you arguing that these are equivalent actions? The entire web was built on the understanding that humans generally operate browsers, and robots.txt is specifically for scenarios in which they do not. To pretend that the automated reading of websites by AI agents is not something different…is quite a stretch.

I see it as very different. I the human want the data from that request. I am using a tool to get it for me.

Should I not be able to execute curl to download a webpage because the "understanding that humans generally operate browsers"?

Re: Claude can now search the web

#110

Post by joshstrange »

Massive props to Anthropic for announcing a feature _and_ making it available for everyone right away.

OpenAI is so annoying in this aspect. They will regularly give timelines for rollout that not met or simply wrong.

Edit: "Everyone" = Everyone who pays. Sorry if this sounds mean but I don't care about what the free tier gets or when. As a paying user for both Anthropic and OpenAI I was just pointing out the rollout differences.

Edit2: My US-bias is showing, sorry I didn't even parse that in the message.

Post reply on HN