Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A search engine that favors text-heavy sites and punishes modern web design

search.marginalia.nu

Post reply on HN
Print view View original

171–180 of 735 posts

Re: A search engine that favors text-heavy sites and punishes modern web design

#171

Post by bityard »

adventured wrote: post #162
JasonFruit wrote: post #150

Earlier quoted context omitted.

Hobby project leads angry person to interesting and unexpected material; angry person remains angry. Details at six.

> Hobby project leads angry person to interesting and unexpected material; angry person remains angry. Not angry in the least. I'm thrilled someone is working on a search competitor to Google. I understand you're attempting to dismiss my pointing out the bad results by calling me angry though. You're focusing your content on me personally, instead of what I pointed out. The parent was far overhyping the results in a…

> I'm thrilled someone is working on a search competitor to Google.

Except the author goes to quite some lengths to explain that his search engine is not a competitor to Google, and is in fact exactly the opposite of Google in many ways: https://memex.marginalia.nu/projects/edge/about.gmi

Re: A search engine that favors text-heavy sites and punishes modern web design

#172

Post by Nextgrid »

hdjjhhvvhga wrote: post #132
snakeboy wrote: post #114

Wow, that's awesome. Great work! For a simple test, I searched "fall of the roman empire". In your search engine, I got wikipedia, followed by academic talks, chapters of books, and long-form blogs. All extremely useful resources. When I search on google, I get wikipedia, followed by a listicle "8 Reasons Why Rome Fell", then the imdb page for a movie by the same name, and then two Amazon book links, which are totall…

As long as few people use it, it will be great. Rest assured that the moment it becomes popular, the people who want to game it will appear.

I don't think the existing media-heavy websites are gaming Google to rank higher. It's that Google itself prefers media heavy content; they don't have to "game" anything.

I also think a search engine like this would be quite hard to game. An ML-based classifier trained on thousands of text-heavy and media-heavy screenshots should be quite robust and I think would be very hard to evade, so the "game" will become more about how identify the crawler so you can serve it a high-ranking page while serving crap to the real users, and it seems fairly easily to defeat if the search engine does a second pass using residential proxies and standard browser user agents to detect this behavior (it could also threaten huge penalties like the entire domain being banned for a month to even deter attempts at this).

Re: A search engine that favors text-heavy sites and punishes modern web design

#173

Post by burkaman »

klntsky wrote: post #149
snakeboy wrote: post #114

Wow, that's awesome. Great work! For a simple test, I searched "fall of the roman empire". In your search engine, I got wikipedia, followed by academic talks, chapters of books, and long-form blogs. All extremely useful resources. When I search on google, I get wikipedia, followed by a listicle "8 Reasons Why Rome Fell", then the imdb page for a movie by the same name, and then two Amazon book links, which are totall…

However, when searching for "haskell type inference algorithm" I get completely useless results.

That query is too long apparently. But if you shorten to "haskell type inference", I think it delivers on its promise:

> If you are looking for fact, this is almost certainly the wrong tool. If you are looking for serendipity, you're on the right track. When was the last time you just stumbled onto something interesting, by the way?

Re: A search engine that favors text-heavy sites and punishes modern web design

#174

Post by CapmCrackaWaka »

isaacgreyed wrote: post #165

A common use case, how to do random thing in programming: I searched python make a bar chart and it returned a live coding video with an AI generated text transcript and two articles which mentioned a different kind of bar. I then narrowed it down to just python bar chart, and got a blog post about scripting with a bar chart in it, this http://www.nitcentral.com/voyager4/hellyear.htm with monty python, bars, and char…

I would probably use this if I wanted to find interesting blog posts/websites about a topic I want to learn more about in general. It seems less useful for returning exact answers to specific questions.

Re: A search engine that favors text-heavy sites and punishes modern web design

#175

Post by LanceH »

klntsky wrote: post #149
snakeboy wrote: post #114

Wow, that's awesome. Great work! For a simple test, I searched "fall of the roman empire". In your search engine, I got wikipedia, followed by academic talks, chapters of books, and long-form blogs. All extremely useful resources. When I search on google, I get wikipedia, followed by a listicle "8 Reasons Why Rome Fell", then the imdb page for a movie by the same name, and then two Amazon book links, which are totall…

However, when searching for "haskell type inference algorithm" I get completely useless results.

It would be nice if we could pipe search engines.

Re: A search engine that favors text-heavy sites and punishes modern web design

#176

Post by the_other »

hdjjhhvvhga wrote: post #132
snakeboy wrote: post #114

Wow, that's awesome. Great work! For a simple test, I searched "fall of the roman empire". In your search engine, I got wikipedia, followed by academic talks, chapters of books, and long-form blogs. All extremely useful resources. When I search on google, I get wikipedia, followed by a listicle "8 Reasons Why Rome Fell", then the imdb page for a movie by the same name, and then two Amazon book links, which are totall…

As long as few people use it, it will be great. Rest assured that the moment it becomes popular, the people who want to game it will appear.

If there were a wider variety of popular search engines, with different ranking criteria, would sites begin to move away from gaming the system? Surely it would be too hard to game more than one search engine at a time?

Re: A search engine that favors text-heavy sites and punishes modern web design

#177

Post by greggturkington »

Wouldn't this just skew towards really old sites?

The third search result for "dog" is this page on how to remove AOL Instant Messenger, published in 2002.

https://sillydog.org/netscape/kb/removeaim.html

No one wants to see newsletter signup popovers, but "modern web design" includes good performance and relevant content. (The search engine itself takes about 2 seconds to first contentful paint, not great.)

Re: A search engine that favors text-heavy sites and punishes modern web design

#178

Post by hn_throwaway_99 »

snakeboy wrote: post #114

Wow, that's awesome. Great work! For a simple test, I searched "fall of the roman empire". In your search engine, I got wikipedia, followed by academic talks, chapters of books, and long-form blogs. All extremely useful resources. When I search on google, I get wikipedia, followed by a listicle "8 Reasons Why Rome Fell", then the imdb page for a movie by the same name, and then two Amazon book links, which are totall…

I had the exact opposite experience. I searched the site for "java", got a Wikipedia link first (for the island, not the programming language), and the 2nd result was to a random JEP page, and all the rest of the results were random tidbits about Java (e.g. "XZ compression algorithm in Java). Didn't get any high level results pointing to an overview of the language, getting started guides, etc.

Re: A search engine that favors text-heavy sites and punishes modern web design

#179

Post by gverrilla »

capableweb wrote: post #111
mumblemumble wrote: post #41

I like it. Coincidentally, the other day I was daydreaming about a search engine that favors sites that are updated less frequently. The thought being, the kinds of labors of love that characterized the 1990s Web that I still sometimes miss are still out there, it's just harder to find them amidst the flood of SEO dreck. So perhaps they could be made discoverable again with the help of a contrarian search engine that…

Similarly, I wish there was a recommendation engine (for web, music, movies, whatever) that can show you what is the furthest away from your existing tastes. I've learned to re-create my Spotify account once every 6 months or so, as their recommendation engine becomes a boring machine after using it daily for some months. I'd love to discover new content that is different from what I read/watch/listen to now, but it'…

Doing that takes real work and curiosity. I'm afraid an algorithm will never be able to do it, particularly if you're into niche stuff. For instance I enjoy a lot a Japanese band called The Boredoms - but few people like it, and there's only 2 of their albums available in spotify.

Re: A search engine that favors text-heavy sites and punishes modern web design

#180

Post by androceium »

marginalia_nu wrote: post #3

Yeah so this is my project. It's very much a work in progress, but occasionally I think it works remarkably well for something I cobbled together alone out of consumer hardware and home-made code :-)

Pretty neat!!!

You may already be aware of this, but the page doesn't seem to be formatted correctly on mobile. The content shows in a single thin column in the middle.

Post reply on HN