Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Updates to our web search products and Programmable Search Engine capabilities

programmablesearchengine.googleblog.com

Post reply on HN
Print view View original

201–207 of 207 posts

Re: Updates to our web search products and Programmable Search Engine capabilities

#201

Post by 1718627440 »

saltysalt wrote: post #199
1718627440 wrote: post #186

Earlier quoted context omitted.

I don't get the problem you claim. You crawl something and get a document in whatever language the site delivers you. You know the language of that document with the lang=... attribute of the document. What results you show for a given language is under your control and not influenced by what the crawled site chose to serve to the crawler.

I'm working on the language improvements presently, but I need to clean out a lot of bad entries in my index. In essence what I am trying to say is many servers ignore "Accept-Language" so you have to rely on other means of detecting the language of the page reliably, e.g. inspecting the body content of the response. It's a non-trivial problem online.

So html lang=... is wrong, or doesn't exist?

> I am trying to say is many servers ignore "Accept-Language"

I wouldn't have expected that to be a hard rule, more like if there are multiple pages to return to have a factor, which one the user most likely wants.

Re: Updates to our web search products and Programmable Search Engine capabilities

#202

Post by aylmao »

embedding-shape wrote: post #164
aylmao wrote: post #160

Earlier quoted context omitted.

> it might take 3 years instead of 6 months, and the shutdown period would be 1 year instead of 1 month This matters much more than people (and evidently those within Google) realize

It matters more the shorter your future planning is. Neither works if you're looking forward 3-4 years, for example.

Why wouldn’t it work if you plan ahead 3-4 years?

Planning ahead doesn’t mean you never suffer from unexpected events, including depreciations. And long timelines help you as much as anyone else.

Re: Updates to our web search products and Programmable Search Engine capabilities

#204

Post by qingcharles »

bicepjai wrote: post #191

Why don’t we have something more “torrent-like” for search? Imagine a decentralized network where volunteers run crawler nodes that each fetch and extract a tiny slice of the web. Those partial results get merged into open, versioned indexes that can be distributed via P2P (or mirrored anywhere). Then anyone can build ranking, vertical search, or specialized tools on top of that shared index layer. I get that reprodu…

What's to stop someone poisoning the data, though? :(

Re: Updates to our web search products and Programmable Search Engine capabilities

#205

Post by jpadkins »

bennydog224 wrote: post #133

I built many products on Google PSE (Custom Search). Results were nowhere near as good as regular Google, but still useful. I usually needed to use another library to get the DOM content anyway. But it still was solid for grounding/checking data. RIP, another one to the Google Graveyard.

your usage was in obvious violation of the terms of service. This is why we can't have nice things.

Re: Updates to our web search products and Programmable Search Engine capabilities

#206

Post by ColinHayhurst »

direwolf20 wrote: post #181
ColinHayhurst wrote: post #78

Earlier quoted context omitted.

Excuse the self-promotion but Mojeek offers a web search API (>9 billion pages): https://www.mojeek.com/services/search/web-search-api/

Nice! I hope Kagi is licensing access from you?

They do

Re: Updates to our web search products and Programmable Search Engine capabilities

#207

Post by 01jonny01 »

01jonny01 wrote: post #2

Google quietly announced that Programmable Search (ex-Custom Search) won’t allow new engines to “search the entire web” anymore. New engines are capped at searching up to 50 domains, and existing full-web engines have until Jan 1, 2027 to transition. If you actually need whole-web search, Google now points you to an “interest form” for enterprise solutions (Vertex AI Search etc.), with no public pricing and no guaran…

Important Email Update from Google:

Dear Programmable Search Engine user,

Thank you for contacting us via the Web Search Products Interest Form. We have received your feedback and are actively reviewing the specific use cases you shared.

We are writing to share important details regarding the transition plan and the available solutions for your search needs.

1. For Unrestricted Web Search: Future Web Search Service

For partners requiring unrestricted "Search the entire web" capabilities, we are developing a new enterprise-grade Web Search Service. As you evaluate your future needs, please be aware of the commercial terms planned for this new service:

Pricing: USD $15 CPM (Cost Per Mille / 1,000 requests).

Minimum Commitment: A minimum monthly fee of USD $30,000 will apply.

We’ll release more information on this service later in 2026. Existing 'Search the entire web' engines remain functional until January 1, 2027.

2. For AI & Advanced Search: Google Vertex AI

We strongly encourage you to explore Google Vertex AI as another option for partners who need enterprise search and AI capabilities across 50 or fewer domains. Vertex AI offers powerful capabilities for:

Grounded Generation: Connecting your AI agents to your own data and/or to Google Search to provide accurate, up-to-date responses.

Custom Data Search: Building enterprise-grade search engines over your own data and specific websites.

This solution is available now and is designed to scale with your specific application needs.

Clarification on Current Service Status:

While you evaluate which path fits your business needs, please remember the timeline for your current implementations:

Existing Projects: If you have an existing Programmable Search Engine configured to "Search the entire web," your service will continue to function until January 1, 2027. You have the full year to plan your migration.

New Projects: As of January 20, 2026, new engines created in the Programmable Search Engine admin console are restricted to "Site Search Only" (specific domains only).

Later in 2026 we’ll provide you with more updates regarding the new Web Search Service and the means to express your desire to use it to power your web search experiences.

Sincerely,

Programmable Search Engine Team

Post reply on HN