Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A search engine that favors text-heavy sites and punishes modern web design

search.marginalia.nu

Post reply on HN
Print view View original

201–210 of 735 posts

Re: A search engine that favors text-heavy sites and punishes modern web design

#201

Post by lucideer »

eterevsky wrote: post #193
snakeboy wrote: post #114

Wow, that's awesome. Great work! For a simple test, I searched "fall of the roman empire". In your search engine, I got wikipedia, followed by academic talks, chapters of books, and long-form blogs. All extremely useful resources. When I search on google, I get wikipedia, followed by a listicle "8 Reasons Why Rome Fell", then the imdb page for a movie by the same name, and then two Amazon book links, which are totall…

Imagine if you were looking for the movie.

I tend to prefer Wikipedia for movies. The exception is actor headshots if I'm trying to identify someone, which Wikipedia lacks for licensing reasons, but otherwise Wikipedia tends to be better than IMDB for most needs. Wikipedia has an IMDB link on every article anyway.

Another need I guess might be reviews, for which RT or MC are better than IMDB: not sure if either of those two will fare better than IMDB in this search engine but again Wiki has links out (in addition to good reception summaries)

Re: A search engine that favors text-heavy sites and punishes modern web design

#202

Post by zozbot234 »

MisterTea wrote: post #195
eterevsky wrote: post #193

Earlier quoted context omitted.

Imagine if you were looking for the movie.

The you'd use a different search engine. Why does everything have to be a Swiss Army knife?

Or you could just search for 'rome movie'. Though for more complex disambiguation you would need to resort to, e.g. schema.org descriptions (which are supported by most search engines, and the foundation for most "smart" search result snippets).

Re: A search engine that favors text-heavy sites and punishes modern web design

#203

Post by asjdflakjsdf »

You should monetise this with amazon affiliate links that are relevant to each search. And then use that money to keep this project going. Google is fantastic, but it has become something different from what it was, the company and the product. It is so refreshing to see a modern tool that encourages exploration of the actual world wide web.

Re: A search engine that favors text-heavy sites and punishes modern web design

#204

Post by marginalia_nu »

timvisee wrote: post #188

Cool! There do seem to be some text encoding issues though. For example: https://search.marginalia.nu/search?query=tim+visee

Yeah I think the charset detection needs work.

It understands the "Content-type: text/html;charset=utf-8" -header, and

but not

It turns out HTML has a lot of corner cases. I'm constantly marveling at how web browsers hold together as well as they do.

Re: A search engine that favors text-heavy sites and punishes modern web design

#205

Post by kwhitefoot »

adventured wrote: post #144
snakeboy wrote: post #114

Wow, that's awesome. Great work! For a simple test, I searched "fall of the roman empire". In your search engine, I got wikipedia, followed by academic talks, chapters of books, and long-form blogs. All extremely useful resources. When I search on google, I get wikipedia, followed by a listicle "8 Reasons Why Rome Fell", then the imdb page for a movie by the same name, and then two Amazon book links, which are totall…

I did a search for "George Washington" First result after Wikipedia: "Radiophone Transmitter on the U.S.S. George Washington (1920) In 1906, Reginald Fessenden contracted with General Electric to build the first alternator transmitter. G.E. continued to perfect alternator transmitter design, and at the time of this report, the Navy was operating one of G.E.'s 200 kilowatt alternators http://earlyradiohistory.us/1919w…

What were you expecting to see for British? There must be millions of pages containing that term. Anyway the first screenful from Google is unadulterated crap, advertising mixed with the usual trivia questions.

If you are going top claim something is wide of the mark then you really ought to tell us at least roughly where the mark is.

Re: A search engine that favors text-heavy sites and punishes modern web design

#206

Post by bityard »

greggturkington wrote: post #177

Wouldn't this just skew towards really old sites? The third search result for "dog" is this page on how to remove AOL Instant Messenger, published in 2002. https://sillydog.org/netscape/kb/removeaim.html No one wants to see newsletter signup popovers, but "modern web design" includes good performance and relevant content. (The search engine itself takes about 2 seconds to first contentful paint, not great.)

This search engine pretty much takes everything that Google is doing and does the opposite. For instance, Google has decided that "relevant" usually also means "recent". Thus, when searching for something on Google, you mainly get results from blogspam farms and almost never do you see anything more than a few years old.

An implication of this is that old sites tend to disappear (either into obscurity or by being taken down) because Google penalizes them in search rankings. The author of this search engine says, however:

> If a webpage has been around for a long time, then odds are it has fundamental redeeming quality that has motivated keeping it around all for that time.

I don't know that I agree 100% with this (there was lots of crap on the "old" web too), but it makes a certain amount of sense.

Re: A search engine that favors text-heavy sites and punishes modern web design

#208

Post by marginalia_nu »

burkaman wrote: post #173
klntsky wrote: post #149

Earlier quoted context omitted.

However, when searching for "haskell type inference algorithm" I get completely useless results.

That query is too long apparently. But if you shorten to "haskell type inference", I think it delivers on its promise: > If you are looking for fact, this is almost certainly the wrong tool. If you are looking for serendipity, you're on the right track. When was the last time you just stumbled onto something interesting, by the way?

The search engine doesn't do any type of re-ordering or synonym stuff, it only tires to construct different N-grams from the search query.

So if you for example compare "SDL tutorial" with "SDL tutorials". On google you'd get the same stuff, this search engine, for better or worse doesn't.

This is a design decision, for now anyway, mostly because I'm incredibly annoyed when algorithms are second-guessing me. On the other hand, it does mean you sometimes have to try different searches to get relevant results.

Re: A search engine that favors text-heavy sites and punishes modern web design

#209

Post by timvisee »

marginalia_nu wrote: post #204
timvisee wrote: post #188

Cool! There do seem to be some text encoding issues though. For example: https://search.marginalia.nu/search?query=tim+visee

Yeah I think the charset detection needs work. It understands the "Content-type: text/html;charset=utf-8" -header, and but not It turns out HTML has a lot of corner cases. I'm constantly marveling at how web browsers hold together as well as they do.

Thanks for your response! Hope you can implement this as well without too much trouble.

I wonder if you could just assume UTF-8 to be the default these days. I imagine that to fix many other cases as well.

Re: A search engine that favors text-heavy sites and punishes modern web design

#210

Post by WorldMaker »

jandrese wrote: post #160
hdjjhhvvhga wrote: post #132

Earlier quoted context omitted.

As long as few people use it, it will be great. Rest assured that the moment it becomes popular, the people who want to game it will appear.

This sort of optimization is why simple recipes are typically found at the end of a rambling pointless blog post now. Still, the best way to break SEO is to have actual competition in the search space. As long as SEO remains focused on Google there is an opportunity for these companies to thrive by evading SEO braindamage.

That sort of recipe blog hasn't happened just for SEO. It's also a bit of a "two audiences" problem: if you are coming to that food blogger from a search you certainly would prefer the recipe first and then maybe any commentary on it below if the recipe looks good. If you are a regular reader of that food blogger you are probably invested in the stories up top and that parasocial connection and the recipes themselves are sometimes incidental to why you are a regular reader.

You see some of that "two readers" divide sometimes even in classic cookbooks, where "celebrity" chefs of the day might spend much of a cookbook on a long rambling memoir. Admittedly such books were generally well indexed and had table of contents to jump right to the recipes or particular recipes, but the concept of "long personal ramble of what these recipes mean to me" is an old one in cookbooks too.

Post reply on HN