Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Updates to our web search products and Programmable Search Engine capabilities

programmablesearchengine.googleblog.com

Post reply on HN
Print view View original

91–100 of 207 posts

Re: Updates to our web search products and Programmable Search Engine capabilities

#91

Post by jfindley »

saltysalt wrote: post #17
01jonny01 wrote: post #2

Google quietly announced that Programmable Search (ex-Custom Search) won’t allow new engines to “search the entire web” anymore. New engines are capped at searching up to 50 domains, and existing full-web engines have until Jan 1, 2027 to transition. If you actually need whole-web search, Google now points you to an “interest form” for enterprise solutions (Vertex AI Search etc.), with no public pricing and no guaran…

I built my own web search index on bare metal, index now up to 34m docs: https://greppr.org/ People rely too much on other people's infra and services, which can be decommissioned anytime. The Google Graveyard is real.

Unfortunately the index is the easy part. Transforming user input into a series of tokens which get used to rank possible matches and return the top N, based on likely relevence, is the hard part and I'm afraid this doesn't appear to do an acceptable job with any of the queries I tested.

There's a reason Google became so popular as quickly as it did. It's even harder to compete in this space nowadays, as the volume of junk and SEO spam is many orders of magnitude worse as a percentage of the corpus than it was back then.

Re: Updates to our web search products and Programmable Search Engine capabilities

#92

Post by shevy-java »

Google has consistently ruined its search engine in the last (almost) 10 years. You can find numerous articles about this, as well as videos on youtube (which is also controlled by google).

Not long ago they ruined ublock origin (for chrome; ublock origin lite is nowhere near as good and effective, from my own experience here).

Now Google is also committing towards more evil and trying to ruin things for more - people, competitors, you name it. We can not allow Google to continue on its wiched path here. It'll just further erode the quality. There is a reason why "killed by google" is more than a mere meme - a graveyard of things killed by google.

We need alternatives, viable ones, for ALL Google services. Let's all work to make this world better - a place without Google.

Re: Updates to our web search products and Programmable Search Engine capabilities

#93

Post by shevy-java »

raincole wrote: post #66
01jonny01 wrote: post #2

Google quietly announced that Programmable Search (ex-Custom Search) won’t allow new engines to “search the entire web” anymore. New engines are capped at searching up to 50 domains, and existing full-web engines have until Jan 1, 2027 to transition. If you actually need whole-web search, Google now points you to an “interest form” for enterprise solutions (Vertex AI Search etc.), with no public pricing and no guaran…

> “search the entire web” TIL they allowed that before. It sounds a bit crazy. Like Google is inviting people to repackage google search itself and sell it / serve with their own ads.

It basically means that Google is now transitioning into a private web.

Others have to replace Google. We need access to public information. States can not allow corporations to hold us here hostage.

Re: Updates to our web search products and Programmable Search Engine capabilities

#94

Post by jsnell »

monooso wrote: post #84
doublerabbit wrote: post #9

Earlier quoted context omitted.

Kagi

Kagi does not use Google's search index. From their post which made the front page of HN yesterday [1]: > Google does not offer a public search API. The only available path is an ad-syndication bundle with no changes to result presentation - the model Startpage uses. Ad syndication is a non-starter for Kagi’s ad-free subscription model. [1]: https://news.ycombinator.com/item?id=46708678

They then go on to say that they pay a 3rd party company to scrape Google results (and serve those scraped results to their users). So their search engine is indeed based on unauthorized and uncompensated use of Google's index.

But since they're not using/paying for a supported API but just taking what they want, they indeed are unlikely to be impacted by this API turndown.

Re: Updates to our web search products and Programmable Search Engine capabilities

#95

Post by whs »

raincole wrote: post #66
01jonny01 wrote: post #2

Google quietly announced that Programmable Search (ex-Custom Search) won’t allow new engines to “search the entire web” anymore. New engines are capped at searching up to 50 domains, and existing full-web engines have until Jan 1, 2027 to transition. If you actually need whole-web search, Google now points you to an “interest form” for enterprise solutions (Vertex AI Search etc.), with no public pricing and no guaran…

> “search the entire web” TIL they allowed that before. It sounds a bit crazy. Like Google is inviting people to repackage google search itself and sell it / serve with their own ads.

I tried it and contributed to searx. It didn't give the same result as Google, and it also have 10k request rate limit (per month I believe). More than that you'll have to "contact us"

Re: Updates to our web search products and Programmable Search Engine capabilities

#96

Post by throwaway290 »

echelon wrote: post #28
doublerabbit wrote: post #9

Earlier quoted context omitted.

Kagi

No wonder Kagi is angry. Google is a monopoly across several broad categories. They're also a taxation enterprise. Google Search took over as the URL bar for 91% of all web users across all devices. Since this intercepts trademarks and brand names, Google gets to tax all businesses unfairly. Tell your legislators in the US and the EU that Google shouldn't be able to sell ads against registered trademarks (+/- some ed…

what stops Kagi from indexing internet and makes them pay some guys to scrape search results from Google? one guy at Marginalia can do it and entire dev team at a PAID search engine can't?

Re: Updates to our web search products and Programmable Search Engine capabilities

#97

Post by 1718627440 »

saltysalt wrote: post #17
01jonny01 wrote: post #2

Google quietly announced that Programmable Search (ex-Custom Search) won’t allow new engines to “search the entire web” anymore. New engines are capped at searching up to 50 domains, and existing full-web engines have until Jan 1, 2027 to transition. If you actually need whole-web search, Google now points you to an “interest form” for enterprise solutions (Vertex AI Search etc.), with no public pricing and no guaran…

I built my own web search index on bare metal, index now up to 34m docs: https://greppr.org/ People rely too much on other people's infra and services, which can be decommissioned anytime. The Google Graveyard is real.

The input on the results page doesn't work, you always need to return to the start page on which the browser history is disabled. That's just confusing behaviour.

Re: Updates to our web search products and Programmable Search Engine capabilities

#98

Post by tosti »

saltysalt wrote: post #17
01jonny01 wrote: post #2

Google quietly announced that Programmable Search (ex-Custom Search) won’t allow new engines to “search the entire web” anymore. New engines are capped at searching up to 50 domains, and existing full-web engines have until Jan 1, 2027 to transition. If you actually need whole-web search, Google now points you to an “interest form” for enterprise solutions (Vertex AI Search etc.), with no public pricing and no guaran…

I built my own web search index on bare metal, index now up to 34m docs: https://greppr.org/ People rely too much on other people's infra and services, which can be decommissioned anytime. The Google Graveyard is real.

This is pretty cool. Don't let the naysayers stop you. Taking a stab at beating Google at their core product is bravery in my book. The best of luck to you!

Re: Updates to our web search products and Programmable Search Engine capabilities

#99

Post by sixhobbits »

tpetry wrote: post #72
nemosaltat wrote: post #58

Earlier quoted context omitted.

> Kagi This seems to be true, but more indirectly. From Kagi’s blog [0] which is a follow up to a Kagi blog post from last year [1]. [0]> Google: Google does not offer a public search API. The only available path is an ad-syndication bundle with no changes to result presentation - the model Startpage uses. Ad syndication is a non-starter for Kagi’s ad-free subscription model.[^1] [0]> The current interim approach (cu…

I believe they try to indirectly say they are using SerpApi or a similar product that scrapes Google search results to use them. And other big ones use it too so it must be ok... That must be the reason why they limit the searches you can do in the starter plan. Every SerpApi call costs money.

Google is also suing SerpAPI

And I can't prove correlation but they refused to index one of my domains and I think it _might_ be because we had some content on there about how to use SerpAPI

Re: Updates to our web search products and Programmable Search Engine capabilities

#100

Post by philistine »

blell wrote: post #65
johnofthesea wrote: post #33

Earlier quoted context omitted.

Better late than never. > The French index is at an advanced stage of completion, we have started creating the German language index, and the English one should start shortly. All progress is quickly integrated into the Qwant STAAN API. https://noc.social/@327ppm/115934198650900394

They can build whatever they want with lots of #hashtags and public money, but that doesn't mean they'll get 30% of French people to use it. But of course they managed to cut themselves a nice salary with EU funds, paid in part by me and you, so that's all that matters.

Oh no, someone is making money outside of the big American monopolists. Quick, the vapors!
Post reply on HN