Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Updates to our web search products and Programmable Search Engine capabilities

programmablesearchengine.googleblog.com

Post reply on HN
Print view View original

151–160 of 207 posts

Re: Updates to our web search products and Programmable Search Engine capabilities

#151

Post by toofy »

saltysalt wrote: post #122
tosti wrote: post #98

Earlier quoted context omitted.

This is pretty cool. Don't let the naysayers stop you. Taking a stab at beating Google at their core product is bravery in my book. The best of luck to you!

Thank you kindly! It's just for fun.

> it’s just for fun.

amazing, for real.

everything i’ve read and heard about the good internet is that it was good because sooooo many of the people did stuff for exactly that, fun.

i’ve spent some time reading through some of the old email lists from earlier internet folks, they predicted exactly what weve turned this into. reading the resistance against early adoption of cookies is incredible to see how prescient some of those people were. truly incredible.

keep having fun with it, i think it’s our only way out of whatever this thing is we have now.

Re: Updates to our web search products and Programmable Search Engine capabilities

#152

Post by Terretta »

consumer451 wrote: post #111

Dumb question: I keep seeing posts about how ~"the volume of AI scrapers is making hosting untenable." There must a ton of new full-web datasets out there, right? What are the major hurdles that prevent the owners of these datasets from providing them to third parties via API? Is it the quality of SERP, or staleness? Otherwise, this seems like a potentially lucrative pivot/side hustle?

> the volume of AI scrapers is making hosting untenable

Aside from that potential, it's also not true.

A Pentium Pro or PIII SSE with circa 1998-99 Apache happily delivers a billion hits a month w/o breaking a sweat unless you think generating pages for every visit is better than generating pages when they change.

Re: Updates to our web search products and Programmable Search Engine capabilities

#153

Post by saltysalt »

toofy wrote: post #151
saltysalt wrote: post #122

Earlier quoted context omitted.

Thank you kindly! It's just for fun.

> it’s just for fun. amazing, for real. everything i’ve read and heard about the good internet is that it was good because sooooo many of the people did stuff for exactly that, fun. i’ve spent some time reading through some of the old email lists from earlier internet folks, they predicted exactly what weve turned this into. reading the resistance against early adoption of cookies is incredible to see how prescient s…

Couldn't agree more! The early pioneers of the Internet were hackers and tinkers, I've tried to maintain the same ethos.

Re: Updates to our web search products and Programmable Search Engine capabilities

#154

Post by saltysalt »

prophesi wrote: post #141
saltysalt wrote: post #130

Earlier quoted context omitted.

I am definitely not trying to complete with Google, instead I am offering an old-school "just search" engine with no tracking, personalization filtering, or AI. It's driven by my own personal nostalgia for the early Internet, and to find interesting hidden corners of the Internet that are becoming increasingly hard to find on Google after you wade through all of the sponsored results and spam in the first few pages..…

There may be a free CS course out there that teaches how to implement a simplified version of Google's PageRank. It's essentially just the recursive idea that a page is important if important pages link to it. The original paper for it is a good read, too. Curiously, it took me forever to find the unaltered version of the paper that includes Appendix A: Advertising and Mixed Motives, explaining how any search engine…

Nice find, will review!

Re: Updates to our web search products and Programmable Search Engine capabilities

#155

Post by ipaddr »

jsnell wrote: post #94
monooso wrote: post #84

Earlier quoted context omitted.

Kagi does not use Google's search index. From their post which made the front page of HN yesterday [1]: > Google does not offer a public search API. The only available path is an ad-syndication bundle with no changes to result presentation - the model Startpage uses. Ad syndication is a non-starter for Kagi’s ad-free subscription model. [1]: https://news.ycombinator.com/item?id=46708678

They then go on to say that they pay a 3rd party company to scrape Google results (and serve those scraped results to their users). So their search engine is indeed based on unauthorized and uncompensated use of Google's index. But since they're not using/paying for a supported API but just taking what they want, they indeed are unlikely to be impacted by this API turndown.

They get results from another provider who has authorized access. Google doesn't provide search results to unauthorized requests as many on tor have experienced.

Re: Updates to our web search products and Programmable Search Engine capabilities

#156

Post by saltysalt »

dust-jacket wrote: post #142
saltysalt wrote: post #17

Earlier quoted context omitted.

I built my own web search index on bare metal, index now up to 34m docs: https://greppr.org/ People rely too much on other people's infra and services, which can be decommissioned anytime. The Google Graveyard is real.

This is mad but cool. Keep at it.

Thanks, mad is fun for me! It costs me nothing if it fails.

Re: Updates to our web search products and Programmable Search Engine capabilities

#157

Post by [deleted] »

throwaway_20357 wrote: post #4
01jonny01 wrote: post #2

Google quietly announced that Programmable Search (ex-Custom Search) won’t allow new engines to “search the entire web” anymore. New engines are capped at searching up to 50 domains, and existing full-web engines have until Jan 1, 2027 to transition. If you actually need whole-web search, Google now points you to an “interest form” for enterprise solutions (Vertex AI Search etc.), with no public pricing and no guaran…

What are some of the niche search engines build on Google's index affected by this?

[deleted]

Re: Updates to our web search products and Programmable Search Engine capabilities

#158

Post by 1718627440 »

saltysalt wrote: post #148
1718627440 wrote: post #134

Earlier quoted context omitted.

True, didn't occur to me, that I should click on the icon instead. Once I have clicked on the search icon once, enter also works. When I input a short query (single letter) it sometimes just shows a blank page, but maybe that is just HNs hug of death. Consider putting the query term more prominently in the front of the URL, so users can edit it. Also from the startpage, the URL in the URLbar isn't updated. As I alrea…

Appreciate the detailed feedback! A lot of the JS trickery and URL shenanigans I'm doing is to prevent bot spam attempts, which was a real problem in the beginning.

Sad state the web is in.

It is intended, that the page currently shows a link to the wordpress login?

Re: Updates to our web search products and Programmable Search Engine capabilities

#159

Post by saltysalt »

1718627440 wrote: post #158
saltysalt wrote: post #148

Earlier quoted context omitted.

Appreciate the detailed feedback! A lot of the JS trickery and URL shenanigans I'm doing is to prevent bot spam attempts, which was a real problem in the beginning.

Sad state the web is in. It is intended, that the page currently shows a link to the wordpress login?

It does not use WordPress.

Re: Updates to our web search products and Programmable Search Engine capabilities

#160

Post by aylmao »

embedding-shape wrote: post #146
zitterbewegung wrote: post #139

Earlier quoted context omitted.

I know that duckduckgo uses Microsoft Bing Custom search and honestly it is a much more robust system since you don't have to worry about Google axing it. https://www.customsearch.ai

Instead you worry about Microsoft axing it? Sure, it might take 3 years instead of 6 months, and the shutdown period would be 1 year instead of 1 month, but hardly either are long-term solutions.

> it might take 3 years instead of 6 months, and the shutdown period would be 1 year instead of 1 month

This matters much more than people (and evidently those within Google) realize

Post reply on HN