Live data from Hacker News

Claude can now search the web

anthropic.com

521–530 of 758 posts

Re: Claude can now search the web

#521
post #278

Earlier quoted context omitted.

That could very well be because the jobs are effectively useless. By no means does that mean the people are, nor is what the income allows them to do. But most jobs sure do seem pointless.

Maybe we already have Universal Basic Income, you just need to have a pointless job to collect it.

USAID seems to demonstrate that to an extent, though it was far from universal

Re: Claude can now search the web

#522
post #421

Earlier quoted context omitted.

Why would there not still be gutter cleaning or ditch digging companies? Or people cleaning their own gutters? I'm not familiar with UBI proposals that do away with traditional enterprises; it's generally suggested as raising the floor. People would have more time to clean their own gutters or use the money they receive to pay someone else. In terms of charity cleaning services, there are people who clean hoarder's h…

> for free on YouTube For free on YouTube in exchange for ad revenue

Imagine not using an ad blocker in this day and age.

Re: Claude can now search the web

#523

Earlier quoted context omitted.

This is actually not true. I'm getting traffic from ChatGpt and Perplexity to my website which is fairly new, just launched a few months ago. Our pages rarely rank in the top 4, but the AI answer engines mange to find them anyways. And I'm talking about traffic with UTM params / referrals from chatgpt, not their scraper bots.

If chatgpt is scraping the web, why can they not link tokens to source of token? being able to cite where they learned something would explode the value of their chatbot. At least a couple of orders of magnitude more value. Without this chatbots are mostly a coding-autocomplete tool for me—lots of people have takes, but it's the tying into the internet that makes a take from an unknown entity really valuable. Perplex…

That's asking for the life stories and photos and pedigrees and family histories of all the chickens that went into your McNuggets. It's just not the way LLMs work. It's an enormous vat of pink slime of unknown origins, blended and stirred extremely well.

https://en.wikipedia.org/wiki/Pink_slime

Re: Claude can now search the web

#524

Earlier quoted context omitted.

We don't just have "bullshit jobs" (which is an actual term these days), we have a "bullshit economy" as well - centered around advertising because without advertising most of the bullshit just wouldn't sell. Like, if you already got a car, you can drive it for 10-20 years easily, or more if you take well care of it. But advertising makes you think you "need" a new car every few years... because that keeps the econom…

I agree on almost all of your points, but what makes you think it's only/primarily the "public sector" that is being blown up by bullshit jobs? I've worked for a fair amount of private sector companies and the amount of "bosses nephew", "copy data from one form to another twice a day" and "waste everyone's time by creating pointless meetings" jobs was already more than enough to explain the status quo.

> I agree on almost all of your points, but what makes you think it's only/primarily the "public sector" that is being blown up by bullshit jobs?

"the economy" = private sector / everything not government; "public sector" = government / fully government owned companies.

And both are horribly blown up due to all the bullshit and onerous bureaucracy that's mostly there because apparently you can't trust people that you do entrust a dozens-of-millions-of-euros worth train carriage to correctly deal with the cash register of the onboard restaurant.

Re: Claude can now search the web

#525

Which frontier model provider let's me specific what websites to search and it will only search those?

You can just prompt ChatGPT 4o saying only get information from web

Yes but you cant limit to a set of websites, I have tried sometimes it works but rarely and if it does search your set it will still go off and pick others. In the field I work in there are around a dozen specialist sites and I just want it to query those. Perhaps I need to develop around it.

Re: Claude can now search the web

#526

Earlier quoted context omitted.

The LLM shouldn’t. robots.txt is intended to control recursive fetches. It is not intended to block any and all access. You can test this out using wget. Fetch a URL with wget. You will see that it only fetches that URL. Now pass it the --recursive flag. It will now fetch that URL, parse the links, fetch robots.txt, then fetch the permitted links. And so on. wget respects robots.txt. But it doesn’t even bother lookin…

While robots.txt is not there to directly prevent automated requests, it does prevent crawling which is needed for search indices. Without recursive crawling, it will not possible for a engine to know what are valid urls[1]. They will otherwise either have to brute-force say HEAD calls for all/common string combinations and see if they return 404s or more realistically have to crawl the site to "discover" pages. The…

> it does prevent crawling

No it doesn’t. It politely requests to crawlers that they do not, and if said crawlers choose to honour it than those specific crawlers will not crawl. That’s it. It can and is ignored without penalty or enforcement.

It’s like suggesting that putting a sign in your front yard saying “please don’t rob my house” prevents burglaries.

> Robots.txt does absolutely apply to LLMs engines and search engines equally

No it doesn’t because again, it’s a request system. It applies only to whatever chooses to pay attention to it, and further, decides to abide by any request within it which there is no requirement to do.

From google themselves:

“The instructions in robots.txt files CANNOT ENFORCE crawler behavior to your site; it's up to the crawler to obey them.”

And as already pointed out, there is no requirement a crawler follow them, let alone anything else.

If you want to control access, and you’re using robots.txt, you’ve no idea what you’re doing and probably shouldn’t be in charge of doing it.

Re: Claude can now search the web

#527
post #489

Earlier quoted context omitted.

Purely on its technical merits Grok is pretty good and fills a niche in the selection of AI agents. But I can absolutely understand not wanting to use an AI owned by somebody who makes Nazi salutes and is dismantling the US government.

[flagged]

His gesture was _exactly_ like Hitler's. A gesture he repeated.

The gesture was quite different to that he'd used previously for 'giving people his heart'.

He's known to be a white supremacist. That is apparently his heritage too.

He supports far right parties in Europe.

Other 'Republican' politicians have repeated the gesture from the dais; but they seem to have made other excuses.

None of the many videos or photos that supposedly show other politicians doing similar gestures actually pass scrutiny. It's possible to inadvertently end with the same hand position. But the full fascist salute, on video, multiple times in succession. That's no accident.

Someone who hadn't meant it would have come back on stage, when it was pointed out to them, and made an apology. Or at least immediately issued a statement/press release.

I would believe he'd planned it as a joke - 'I bought this election, I'm going to throw a Nazi salute for memes'. But I'm not sure that's ultimately any better.

Perhaps you believe he's just a catastrophically idiotic person with no-one around him helping him?

Re: Claude can now search the web

#528
post #343
post #278

Earlier quoted context omitted.

Maybe we already have Universal Basic Income, you just need to have a pointless job to collect it.

One suggested weakness of UBI is a lack of purpose. I wonder if the "solution" is somewhat as you implied: jobs without a strict return on investment. You get your stipend, but you're keeping your block clean by sweeping and mulching. They're getting theirs in exchange for cranking out sourdough at cost for the neighbourhood. Someone else gardens for elderly residents.

Cleaning, gardening and baking is proper jobs though.

Re: Claude can now search the web

#529

Earlier quoted context omitted.

> I asked it "Rust crate to access Postgres with Arrow support" Is that how you actually use llms? Like a Google search box?

It's funny because many people type full sentence questions into search engines too. It's usually a sign of being older and/or not very experienced with computers. One thing about geeks like me is we will always figure out what the bare minimum is (at least for work, I hope everyone has at least a few things they enjoy and don't try to optimise).

It's not about being young or old, search engines have moved away from pure keyword searches and often typing your actual query gives better results than searching for keywords, especially with Google.
Post reply on HN