Live data from Hacker News

Claude can now search the web

anthropic.com

31–40 of 758 posts

Re: Claude can now search the web

#31
Funny thing is that I have the obsidian-mcp-tools installed and today claude-desktop just starting fetching stuff from the web through that because it exposes a fetch tool to claude.

So this limitation is a bit arbitrary anyway.

Re: Claude can now search the web

#33
post #22
post #10

I wonder if it will actually respect the robots.txt this time.

robots.txt is meant for automated crawlers, not human-driven actions.

It must form the search index somehow. That is prior the human action. Simply it would not find the page at all if it respects.

Re: Claude can now search the web

#34

There is already a 100 ways of doing it using MCP https://glama.ai/mcp/servers?searchTerm=search What's the benefit of bringing native integration?

The native app that allows for MCP is only available officially on Mac's and the web interface is generally more convenient for non-technical users. Searching and interacting with the web has become a table-stakes feature and was a glaring gap in Claude.

Re: Claude can now search the web

#35
post #28

Earlier quoted context omitted.

In what circles is it a joke? Google bots seem to respect it on my sites according to logs.

Apparently, the regular search crawler does it, but the ai thingie doesn't.

Huh? You can add Google-Extended[1] to opt out from Generative AI summaries.

[1] https://blog.google/technology/ai/an-update-on-web-publisher...

Re: Claude can now search the web

#36
post #28

Earlier quoted context omitted.

In what circles is it a joke? Google bots seem to respect it on my sites according to logs.

Apparently, the regular search crawler does it, but the ai thingie doesn't.

Can confirm. My website is flooded with AI bots despite attempts to block crawlers to certain parts of it.

Re: Claude can now search the web

#39
post #10

I wonder if it will actually respect the robots.txt this time.

I don't think it should. If a user asks the AI to read the web for them, it should read the web for them. This isn't a vacuum charged with crawling the web, it's an adhoc GET request.

Re: Claude can now search the web

#40

kagi already lets me use claude to search the web. how is this different?

kagi is searching the web for you, and then injecting the results into the context of the prompt.

Are there any downsides to that approach? It seems like we're moving towards empowering llm's to interact with stuff as if that's better than us doing it for them - is it really?

Eg say I want to build an agent to make decisions, shall I write some code to insert the data that informs the decision into the prompt, return structured data, and then write code to implement the decision?

Or should I empower the llm do those things with function calls?

Post reply on HN