Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Updates to our web search products and Programmable Search Engine capabilities

programmablesearchengine.googleblog.com

Post reply on HN
Print view View original

61–70 of 207 posts

Re: Updates to our web search products and Programmable Search Engine capabilities

#61

Post by Antibabelic »

jamesbelchamber wrote: post #34

Are competing search indexes (Bing, Ecosia/Qwant, etc) objectively worse in significant ways, or is Google just so entrenched that people don't want to "risk it" with another provider (and/or preferences and/or inertia). I suppose I'm asking whether this is actually a _good thing_ in that it will stimulate competition in the space, or if it's just a case that Google's index is now too good for anyone to reasonably ca…

Bing's index is smaller than Google's, and anecdotally I get fewer relevant results when using it, particularly from sites like Reddit that have exclusive search deals with Google.

Re: Updates to our web search products and Programmable Search Engine capabilities

#62

Post by zelphirkalt »

baubino wrote: post #44
Gigachad wrote: post #26

Earlier quoted context omitted.

I feel like soon there won’t even be a point having a search engine since almost the entire internet will be useless AI slop.

The old internet is still there. It hasn‘t gone away; it‘s just undiscoverable with ad-based search. The more slop there is, the more necessary it is to have good search engines.

Recently, I set up a fresh system on a laptop. Ahahahaaa, how utterly crap Google search results now are! It fills me with some stress and disgust to use that. Now one of the first things I do, right after emergency using duckduckgo to search for uBlock Origin and NoScript, is to get Kagi search installed as default search. Then I can continue setting things up more calmly.

Re: Updates to our web search products and Programmable Search Engine capabilities

#63

Post by Hackbraten »

mrweasel wrote: post #16
Antibabelic wrote: post #3

Relevant: Waiting for dawn in search: Search index, Google rulings and impact on Kagi https://news.ycombinator.com/item?id=46708678

This might be me reading it wrong, but isn't shutting down the full-web search going against the ruling mentioned in the Kagi post? > Google must provide Web Search Index data (URLs, crawl metadata, spam scores) at marginal cost. Maybe they're shutting down the good integration and then Kagi, Ecosia and others can buy index data in an inconvenient way going forward?

If I understand Kagi's blog post correctly, then here's what happened, chronologically:

Kagi makes deals with many search engines so they can have raw search results in exchange for money.

Google says: no, you can't have raw search results because only whales can get those. Only thing we can offer you is search results riddled with ads and we won't allow you to reorder or filter them.

Kagi thinks Google's offer is unacceptable, so Kagi goes to a third party SERP API, which scrapes Google at scale and sells the raw search results to Kagi and others.

August 2024: Court says Google is breaking the law by selling raw search results only to whales.

December 2025: Court orders that for the next six years, 1. Google must no longer exclude non-whales from buying raw search results, 2. Google must offer the raw search results for a reasonable price, and 3. Google can no longer force partners to bundle the results with ads.

December 2025: Google sues the third-party scraping companies.

January 2026: Google says "hey, the old search offering is going to go away, there's going to be a new API by 2027, stay tuned."

Re: Updates to our web search products and Programmable Search Engine capabilities

#65

Post by blell »

johnofthesea wrote: post #33
tweetle_beetle wrote: post #18

Earlier quoted context omitted.

It's a noble effort, but they're so late to the game that it's hard to see them making a significant dent. I hope I'm wrong. They were: > aiming to serve 30% of French search queries [by end of 2025] https://blog.ecosia.org/launching-our-european-search-index/

Better late than never. > The French index is at an advanced stage of completion, we have started creating the German language index, and the English one should start shortly. All progress is quickly integrated into the Qwant STAAN API. https://noc.social/@327ppm/115934198650900394

They can build whatever they want with lots of #hashtags and public money, but that doesn't mean they'll get 30% of French people to use it.

But of course they managed to cut themselves a nice salary with EU funds, paid in part by me and you, so that's all that matters.

Re: Updates to our web search products and Programmable Search Engine capabilities

#66

Post by raincole »

01jonny01 wrote: post #2

Google quietly announced that Programmable Search (ex-Custom Search) won’t allow new engines to “search the entire web” anymore. New engines are capped at searching up to 50 domains, and existing full-web engines have until Jan 1, 2027 to transition. If you actually need whole-web search, Google now points you to an “interest form” for enterprise solutions (Vertex AI Search etc.), with no public pricing and no guaran…

> “search the entire web”

TIL they allowed that before. It sounds a bit crazy. Like Google is inviting people to repackage google search itself and sell it / serve with their own ads.

Re: Updates to our web search products and Programmable Search Engine capabilities

#67

Post by direwolf20 »

saltysalt wrote: post #54
orf wrote: post #50

Earlier quoted context omitted.

Sure, but the point is results are not relevant at all? It’s cool though, and really fast

I'll work on that adjustment, it's fair feedback thanks!

Unfortunately this is the bulk of search engine work. Recursive scraping is easy in comparison, even with CAPTCHA bypassing. You either limit the index to only highly relevant sites (as Marginalia does) or you must work very hard to separate the spam from the ham. And spam in one search may be ham in another.

Re: Updates to our web search products and Programmable Search Engine capabilities

#68

Post by direwolf20 »

Gigachad wrote: post #26
tweetle_beetle wrote: post #18

Earlier quoted context omitted.

It's a noble effort, but they're so late to the game that it's hard to see them making a significant dent. I hope I'm wrong. They were: > aiming to serve 30% of French search queries [by end of 2025] https://blog.ecosia.org/launching-our-european-search-index/

I feel like soon there won’t even be a point having a search engine since almost the entire internet will be useless AI slop.

A search engine doesn't have to search the entire internet. Most of them are extremely opinionated about what they index.

Re: Updates to our web search products and Programmable Search Engine capabilities

#69

Post by direwolf20 »

baubino wrote: post #44
Gigachad wrote: post #26

Earlier quoted context omitted.

I feel like soon there won’t even be a point having a search engine since almost the entire internet will be useless AI slop.

The old internet is still there. It hasn‘t gone away; it‘s just undiscoverable with ad-based search. The more slop there is, the more necessary it is to have good search engines.

If you haven't tried Marginalia Search yet, do so. It's a small web search.

Re: Updates to our web search products and Programmable Search Engine capabilities

#70

Post by direwolf20 »

jonplackett wrote: post #47

Are search engines like Kagi completely screwed by this or is there a way for them to keep operating?

Kagi doesn't have a partnership with Google - they work under adversarial interoperability, stealing results from Google against their will, and paying some third-party to enable this. They'd like to simply pay Google, but Google doesn't want their money.
Post reply on HN