Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A look at search engines with their own indexes (2021)

seirdy.one

Post reply on HN
Print view View original

21–28 of 28 posts

Re: A look at search engines with their own indexes (2021)

#21

Post by reddalo »

marginalia_nu wrote: post #18
mrweasel wrote: post #10

I have been somewhat impressed by Mojeek, but it does have two obvious flaws: 1) It not really good for localized search, it might be if you're local to the US or UK. 2) No !bangs. Coming from Ecosia I frequently just do !w !maps !yt because I know where I want the answer to come from For English language searches, it completely usable, but not quite as good as Bing or Google. I really wanted to try to use Mojeek as…

Local search and location-aware search is probably Google's biggest moat against smaller search engines. Bing does it passably, but it's aguably still pretty bad. What's worse is that it's probably hard to ever get working well without the internet-scale profiling Google has access to.

> Local search and location-aware search is probably Google's biggest moat

The European Union, at least, has limited that a bit by preventing Google from linking Google Maps from their SERP.

So now, if you're in the EU, local results will display a map but you can't click on it.

Re: A look at search engines with their own indexes (2021)

#22

Post by reddalo »

marginalia_nu wrote: post #7
Waterluvian wrote: post #4

Is there some 80/20 rule for web indexing? I’m not saying having deep per-page indexing of Reddit, for example, isn’t useful. But is there any value in a breadth-focused index that is far cheaper to maintain?

Almost certainly. Internet search is above all a problem of improving the signal to noise ratio. There's an inordinate amount of documents that will never be a good search result for any query. Both in trivial cases that have barely anything to index in them, but also sign-up forms, cookie policies, redundant information (e.g. any given man page exists in dozens if not hundreds of identical copies on the web).

> cookie policies

Unless you're specifically searching for other websites' cookie policies (e.g. to understand how they work, or to do research on them, or just to plainly copy them...)

Re: A look at search engines with their own indexes (2021)

#23

Post by reddalo »

raytopia wrote: post #12

A little tangential but does anyone know if there are any modern web directories? I'm wondering because it seems like due to the amount of spam on the web there needs to be more human curation as opposed to algrothims deciding what websites are valuable or not.

Ohh I remember Google Directory, good times. They closed it down for good in 2011.

Dmoz was also closed, but it seems like there's a "new" Dmoz called Curlie [1], founded by some of the original team members.

[1] https://curlie.org

Re: A look at search engines with their own indexes (2021)

#24

Post by cyanydeez »

HeatrayEnjoyer wrote: post #8
jeffreyw128 wrote: post #5

Missed exa.ai! Embeddings-based search engine with its own index

How does an embeddings based search work? Without hallucinating bad links?

Embeddings arnt grnerative AI.

Theyre just vecotors of arbitrary.dimension and similarity is calculated by a ndimensional fnction.

Re: A look at search engines with their own indexes (2021)

#26

Post by mrweasel »

marginalia_nu wrote: post #18
mrweasel wrote: post #10

I have been somewhat impressed by Mojeek, but it does have two obvious flaws: 1) It not really good for localized search, it might be if you're local to the US or UK. 2) No !bangs. Coming from Ecosia I frequently just do !w !maps !yt because I know where I want the answer to come from For English language searches, it completely usable, but not quite as good as Bing or Google. I really wanted to try to use Mojeek as…

Local search and location-aware search is probably Google's biggest moat against smaller search engines. Bing does it passably, but it's aguably still pretty bad. What's worse is that it's probably hard to ever get working well without the internet-scale profiling Google has access to.

They might be doing it differently, but Ecosia uses Bing and have really good localized search, at least for Denmark. There is very little difference between Google and Bing these days, if anything I'd say Bing is the better search engine.

Re: A look at search engines with their own indexes (2021)

#28

Post by ldng »

mrweasel wrote: post #10

I have been somewhat impressed by Mojeek, but it does have two obvious flaws: 1) It not really good for localized search, it might be if you're local to the US or UK. 2) No !bangs. Coming from Ecosia I frequently just do !w !maps !yt because I know where I want the answer to come from For English language searches, it completely usable, but not quite as good as Bing or Google. I really wanted to try to use Mojeek as…

Perfect, localized search is the most annoying thing there is.
Post reply on HN