Live data from Hacker News

A search engine that favors text-heavy sites and punishes modern web design

search.marginalia.nu

511–520 of 735 posts

Re: A search engine that favors text-heavy sites and punishes modern web design

#513
post #50

Fascinating. I studied an "obscure" group of insects. My go-to search term to test an engine is their family name as it is a rarely used word and I know most (all?) of the major data sources that have accumulated data on it. When Wolfram Alpha added species names, I checked with the name, boring, Duck Duck, boring, Google (well we know Google isn't for search anymore, it's absolutely horrible) boring, Bing, boring...…

> well we know Google isn't for search anymore,

If you're talking about the ads, I would bear in mind Google's whole business model is basically online advertising. Search is just the vehicle to deliver those ads; I'd say Google is pretty good at throwing things back.

Re: A search engine that favors text-heavy sites and punishes modern web design

#514
post #379

Earlier quoted context omitted.

The search engine doesn't do any type of re-ordering or synonym stuff, it only tires to construct different N-grams from the search query. So if you for example compare "SDL tutorial" with "SDL tutorials". On google you'd get the same stuff, this search engine, for better or worse doesn't. This is a design decision, for now anyway, mostly because I'm incredibly annoyed when algorithms are second-guessing me. On the o…

Maybe list the synonyms under the query, so its easier to try different formulations.

Oh this sounds like it could be a really cool idea! This way it could also be subtly teaching users that the engine doesn't do automatic synonyms translation so it's worth experimenting; also kinda like giving the synonyms feature while still keeping user in full control.

Re: A search engine that favors text-heavy sites and punishes modern web design

#515

Wow, that's awesome. Great work! For a simple test, I searched "fall of the roman empire". In your search engine, I got wikipedia, followed by academic talks, chapters of books, and long-form blogs. All extremely useful resources. When I search on google, I get wikipedia, followed by a listicle "8 Reasons Why Rome Fell", then the imdb page for a movie by the same name, and then two Amazon book links, which are totall…

[deleted]

Re: A search engine that favors text-heavy sites and punishes modern web design

#516
post #322

Earlier quoted context omitted.

The context matters. I'd happily read "Top 10" lists on a website if the site itself was dedicated to that one thing. "Top 10 Prog Rock albums", while a lazy, SEO-bait title, would at least be credible if it were on a music-oriented website. But no, these stories all come from cookie-cutter "new media" blog sites, written by an anonymous content writer who's repackaged Wikipedia/Discogs info into Buzzfeed-style copy…

This got me thinking that maybe one of the other big reasons for this is that the algorithms prioritize newer pages over older pages. This produces the problem where instead of covering a topic and refining it over time, the incentive is to repackage it over and over again. It reminds me of an annoyance I have with the Kindle store. If I wanted to find a book on, let's say, Psychology, there is no option to find all-…

Amazon search clearly does not prioritize exact title matches.

Re: A search engine that favors text-heavy sites and punishes modern web design

#517

This would be awesome if the search actually worked. Typed in 'runescape' and expected a few websites left over from the early 2000s. But I got nothing, just a lot of hits to other keywords.

Huh. That's an interesting case you've found.

I think part of the problem is gold sellers were displacing all the good results. I blocked a few of them and got a few more relevant results, but it's not great.

But still, the search engine only finds two great hits. I wonder why not. Maybe there just aren't that many runescape pages around still? Or it may just be that it hasn't found anything better yet. The index is pretty shallow right now, only 20M URLs, I aim for more than double.

I honestly wasn't planning on this blowing up on HN at this stage.

Re: A search engine that favors text-heavy sites and punishes modern web design

#518
Fantastic idea and it works quite well for short phrases that I tried.

As expected I am getting a lot of early 2000s sites which is something that I miss on regular Google.

Hilariously searching for "array data structure" got me one of the top results this little tiny page: http://infolab.stanford.edu/~backrub/google.html

Re: A search engine that favors text-heavy sites and punishes modern web design

#520

Yeah so this is my project. It's very much a work in progress, but occasionally I think it works remarkably well for something I cobbled together alone out of consumer hardware and home-made code :-)

Thank you so much for creating such a useful search engine!

Is there any way that you can get an HTTP certificate?

I use an old iPhone 4S, and most of the modern web is inaccessible due to TLS. Hacker News and mbasic.facebook are two of the last sites I can use.

Usually text-based sites are more accessible, so this could be really useful to help me continue using my antique devices!

Post reply on HN