Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A search engine that favors text-heavy sites and punishes modern web design

search.marginalia.nu

Post reply on HN
Print view View original

81–90 of 735 posts

Re: A search engine that favors text-heavy sites and punishes modern web design

#81

Post by mda »

trutannus wrote: post #70
xipho wrote: post #50

Fascinating. I studied an "obscure" group of insects. My go-to search term to test an engine is their family name as it is a rarely used word and I know most (all?) of the major data sources that have accumulated data on it. When Wolfram Alpha added species names, I checked with the name, boring, Duck Duck, boring, Google (well we know Google isn't for search anymore, it's absolutely horrible) boring, Bing, boring...…

> well we know Google isn't for search anymore Do you suggest anything better? As far as I can tell, all the other search engines are either repackaged Bing (ie: DuckDuck), or are just as bad.

IMHO, that is a trendy claim in HN with little evidence.

Re: A search engine that favors text-heavy sites and punishes modern web design

#82

Post by potatoman22 »

mumblemumble wrote: post #41

I like it. Coincidentally, the other day I was daydreaming about a search engine that favors sites that are updated less frequently. The thought being, the kinds of labors of love that characterized the 1990s Web that I still sometimes miss are still out there, it's just harder to find them amidst the flood of SEO dreck. So perhaps they could be made discoverable again with the help of a contrarian search engine that…

I like the idea. Tangentially, I wonder how one would find the right 'penalty' for more updated sites?

Re: A search engine that favors text-heavy sites and punishes modern web design

#83

Post by zimpenfish »

Searched for my initials - got back a bunch of raw binary results (mp4, pdf, img, txz, etc.) which was disconcerting. Although it did find one reference to actual-me which is better than Google manages on the first 4 pages...

https://imgur.com/a/n2xro2Y

Re: A search engine that favors text-heavy sites and punishes modern web design

#84

Post by gibspaulding »

mumblemumble wrote: post #41

I like it. Coincidentally, the other day I was daydreaming about a search engine that favors sites that are updated less frequently. The thought being, the kinds of labors of love that characterized the 1990s Web that I still sometimes miss are still out there, it's just harder to find them amidst the flood of SEO dreck. So perhaps they could be made discoverable again with the help of a contrarian search engine that…

Million Short [1] offers an option to omit results from popular domains. It's a different approach from what you describe, but I think the goal is similar.

[1] https://millionshort.com/

Re: A search engine that favors text-heavy sites and punishes modern web design

#85

Post by marginalia_nu »

artembugara wrote: post #75
marginalia_nu wrote: post #3

Yeah so this is my project. It's very much a work in progress, but occasionally I think it works remarkably well for something I cobbled together alone out of consumer hardware and home-made code :-)

Nice, what are you using to crawl the web?

It's pretty much all bespoke.

I use external libraries for parsing HTML (JSoup) and robots.txt; but that's about it.

Re: A search engine that favors text-heavy sites and punishes modern web design

#86

Post by marcosdumay »

I've got some results where the same site is has than 70% of the links. It was a very on topic and high quality site, but still, all the results shouldn't point to the same place.

I think some grouping by site (and capping to only the few most relevant links there) would improve the engine.

Re: A search engine that favors text-heavy sites and punishes modern web design

#88

Post by yakubin »

Wikpedia links point to https://encyclopedia.marginalia.nu/> instead, which to my eyes is less readable. The justified text, done with CSS, instead of the LaTeX algorithm, looks wild. The font used for quotations is even worse (very thin).

Wikipedia is perfectly usable without JavaScript and it's one of the nicest sites out there typography-wise, so I'd reconsider this redirection.

Re: A search engine that favors text-heavy sites and punishes modern web design

#89

Post by marginalia_nu »

marginalia_nu wrote: post #37
ape4 wrote: post #32

Fast and doesn't crash when on the front page of Hacker News!

Well,... yet. Load average is at 1.2, not that bad. But the services are getting a solid workout.

The real test is in now, index server is reconstructing its index. It does this every 6 hours if there is new pages. Takes half an hour or so usually.

It's supposed to be able to handle searches at the same time, but jeepers, it's gonna have to chew through nearly 400 Gb of data while dealing with over 1 request per second.

Re: A search engine that favors text-heavy sites and punishes modern web design

#90

Post by mhh__ »

mda wrote: post #81
trutannus wrote: post #70

Earlier quoted context omitted.

> well we know Google isn't for search anymore Do you suggest anything better? As far as I can tell, all the other search engines are either repackaged Bing (ie: DuckDuck), or are just as bad.

IMHO, that is a trendy claim in HN with little evidence.

You're downvoted but in my experience I have never really been burned by this Google-decline
Post reply on HN