Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alexandria Search

alexandria.org

Post reply on HN
Print view View original

151–160 of 181 posts

Re: Alexandria Search

#151

Post by aghilmort »

zeta0134 wrote: post #68
version_five wrote: post #30

There have been a few search engines out recently. I'm curious how people evaluate them quickly. I've realized my searching is basically optimized for google and the web that has grown up around it. Also, in 1998 I wasn't as aware of what was out there as I am now. It's pretty rare (even if its possible) that I do a search and come across a completely new site that I haven't heard of before, for anything nontrivial.…

I've been wondering for a while now about building a search engine for the ad free web. That is, penalize or outright refuse to index any recognized advertising network, letting through only those sites which don't perform invasive tracking with third party services. Mostly as a curiosity: what would be left? What would rise to the top when you filter all of that out?

exploring adding that search / similar as breeze filter at https://breezethat.com/- not sure can do with curation + bing or google, may have to wait until we bring our scraper out of testing

Re: Alexandria Search

#152

Post by Seirdy »

josefcullhed wrote: post #87

Hello, My name is Josef Cullhed. I am the programmer of alexandria.org and one of two founders. We want to build an open source and non profit search engine and right now we are developing in our spare time and are funding the servers ourselves. We are indexing commoncrawl and the search engine is in a really early stage. We would be super happy to find more developers who want to help us.

Oh boy, I have too many questions. I'd appreciate any answers you're able/willing to give:

1. Do you have any plans to support the parsing of any additional metadata (e.g. semantic HTML, microformats, schema.org structured data, open graph, dublin core, etc)?

2. How do you plan to address duplicate content? Engines like Google and Bing filter out pages containing the same content, which is welcome due to the amount of syndication that occurs online. `rel="canonical"` is a start, but it alone is not enough.

3. With the ranking algorithm being open-source, is there a plan to address SEO spam that takes advantage of Alexandria's ranking algo? I know this was an issue for Gigablast, which is why some parts of the repo fell out of sync with the live engine.

4. What are some of your favorite search engines? Have you considered collaboration with any?

Re: Alexandria Search

#153

Post by thewakalix »

bricemo wrote: post #135
orlp wrote: post #58

Earlier quoted context omitted.

IMO search has had its goalpost moved. It used to be about scale, technical challenges, bandwidth, storage, etc. It is still about that, but a significantly harder challenge to solve has come up: searching in a malicious environment. SEO crap nowadays completely dominates search, Google has lost the war. Simply put, I believe that Google sucks at search, in the modern context. It is great at indexing, it has solved p…

You can just append “Reddit” or “stackoverflow”, you don’t need to write “site:Reddit.com”

No, I've seen SEO spam sites starting to take advantage of this trend by mentioning "reddit" on their pages.

Re: Alexandria Search

#154

Post by bricemo »

jstx1 wrote: post #46

Randomly picking a search that I needed for work today - searching for "pandas order by list" says that it has 44 results and it shows only 3: - a Github issue for dask - an article about panda populations - some coronavirus article that happens to have an unrelated snippet of pandas code Google obviously picks the relevant stackoverflow thread as the first response.

I searched for "pizza near me". Google shows me several pizza places within a 2 mile distance. Alexandria gave me the link to papajohns.com.

Re: Alexandria Search

#155

Post by bricemo »

gverrilla wrote: post #143

People say the problem is google, but the real problem is content. All these new search engines are searching on the same space, filled with crap.

Agreed. When people say they Google is returning poor results, I can never get an answer of what specific URL people actually want. Just general unhappiness and some mythical, vague, ideal result.

Re: Alexandria Search

#156

Post by billfruit »

pygar wrote: post #129
outcoldman wrote: post #38

Earlier quoted context omitted.

> I've realized my searching is basically optimized for google Is it just me, or I feel like Google does not provide anymore good results for me. Like every time I search something completely out of my knowledge, like "How to purchase a property in Mexico", it will give me 100+ results of some results with autogenerated content like "10 best places to buy property in Mexico". And the only way to fix that would be to…

> Is it just me, or I feel like Google does not provide anymore good results for me. I am starting to suspect that there might be nothing to find. I just don't think people (other then the tech oriented) are creating websites and running forums - and why would they? Reddit might be be only place you _can_ find that type of content. What should search engines do then? With a tiny number of exceptions, it might be that…

On a tangent, google seems to turn up lot more obnoxious Quora results rather than reddit ones. Anyone find that to be the case?

Re: Alexandria Search

#157

Post by blondin »

bricemo wrote: post #155
gverrilla wrote: post #143

People say the problem is google, but the real problem is content. All these new search engines are searching on the same space, filled with crap.

Agreed. When people say they Google is returning poor results, I can never get an answer of what specific URL people actually want. Just general unhappiness and some mythical, vague, ideal result.

google search often returns wrong, unrelated, or no result. i have been in that situation many times before and moved on to change my query.

i am (and most of us are) trying to solve my own issue. not google's.

that's why you get vague answers to your questions. it's not because it doesn't happen. it's because at that moment we care much more about solving our problem. that's what brought us to google search in the first place.

Re: Alexandria Search

#158

Post by ffhhj »

Seirdy wrote: post #130
ffhhj wrote: post #55

Earlier quoted context omitted.

> There have been a few search engines out recently I'd like to try them out, could you mention which?

I listed a bunch over at https://seirdy.one/2021/03/10/search-engines-with-own-indexe... , and I'm always adding more. I first discovered Alexandria in early February: https://git.sr.ht/~seirdy/seirdy.one/commit/935b55f10f9024ee... Around the same time, I also discovered sengine.info, Artado, Entfer, and Siik. By sheer coincidence they all were mentioned to me or decided to crawl my site within the same couple weeks.…

Great work!

Re: Alexandria Search

#159

Post by IAmEveryone »

hinkley wrote: post #144
daptaq wrote: post #97

Earlier quoted context omitted.

Might be an instance of Goodhart's law: https://en.m.wikipedia.org/wiki/Goodhart's_law If all websites try to optimise for SEO, they undermine the assumption that the evaluation of a search engine is the pure consequence of how well a site satisfies a query.

I really think than one, we are going to have to end up with search engines managing a curated list of 'roots', and two, those roots are going to end up consisting substantially of a mix of more 'human' sites and, let's be honest with ourselves, a certain amount of content that is paying for favoritism. I think it's very possible that we have effectively raised the noise floor so high that there is no signal, but als…

Oh, look! It’s the same „flaw“ people have been explaining each other for a decade. We’ve all seen it dozens of times, yet Google apparently remains in the dark.

Re: Alexandria Search

#160

Post by schemescape »

josefcullhed wrote: post #87

Hello, My name is Josef Cullhed. I am the programmer of alexandria.org and one of two founders. We want to build an open source and non profit search engine and right now we are developing in our spare time and are funding the servers ourselves. We are indexing commoncrawl and the search engine is in a really early stage. We would be super happy to find more developers who want to help us.

Very impressive work so far!

Apologies if I missed it (and solely out of curiosity), but how roughly much does hosting Alexandria Search cost (per month)? (I'm assuming you've optimized for cost to avoid spending your own money!)

I have some other questions (around crawlers, parsing, and dependencies), but I need to read the other comments first (to see if my questions have already been answered).

Post reply on HN