Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A search engine that favors text-heavy sites and punishes modern web design

search.marginalia.nu

Post reply on HN
Print view View original

31–40 of 735 posts

Re: A search engine that favors text-heavy sites and punishes modern web design

#31

Post by axelroze »

antupis wrote: post #26
gtmb wrote: post #6

As everything in life flows in cycle, I predict the search engine that will de-throne Google will be like Google when it started - a simple variation of page rank. No smarts, no bubble, no signals decided by over fitting to a biased engineer preference.

As dev I would love search engine which would only do search to stackoverflow github issues, documentation etc.

You can limit the search query per website in DDG (and probably in others)

Example: `rust slow compilation site:stackoverflow.com`

Re: A search engine that favors text-heavy sites and punishes modern web design

#33

Post by feikname »

muxator wrote: post #10
camillomiller wrote: post #2

Great idea, awful UI

How so? It's intuitive and super fast. I whish there were more websites with such a simple UI.

it's too "uncompact". Font size too big and could use a bit more horizontal space.

I find it comfortable to use at 60% zoom level

Re: A search engine that favors text-heavy sites and punishes modern web design

#35

Post by marginalia_nu »

santamex wrote: post #14
marginalia_nu wrote: post #3

Yeah so this is my project. It's very much a work in progress, but occasionally I think it works remarkably well for something I cobbled together alone out of consumer hardware and home-made code :-)

Which software do you use to index the sites?

I wrote it myself from scratch. I have some metadata in mariadb, but the index is bespoke.

A design sketch of the index is that it uses one file with sorted URL IDs, one with IDs of N-grams (i.e. words and word-pairs) referring to ranges in the URL file; as well as a dictionary for relating words to word-IDs; that's a GNU Trove hash map I modified to use memory map data instead of direct allocated arrays.

So when you search for two words, it translates them into IDs using the special hash map, goes to the words file and finds the least common of the words; starts with that.

Then it goes to the words file and looks up the URL range of the first word.

Then it goes to the words file and looks up the URL range of the second word.

Then it goes through the less common word's range and does a binary search for each of those in the range of the more common word.

Then it grabs the first N results, and translates them into URLs (through mariadb); and that's your search result.

I'm skipping over a few steps, but that's the very crudest of outlines.

Re: A search engine that favors text-heavy sites and punishes modern web design

#36

Post by r00t4ccess »

camillomiller wrote: post #2

Great idea, awful UI

The page isn't prompting for cookie preferences, asking to allow notifications, popping up a mailing list or coupon half way do the page, playing a full page video with sound, or load 97million lines of javascript. I'd say its pretty much perfect.

Re: A search engine that favors text-heavy sites and punishes modern web design

#39

Post by __MatrixMan__ »

gtmb wrote: post #6

As everything in life flows in cycle, I predict the search engine that will de-throne Google will be like Google when it started - a simple variation of page rank. No smarts, no bubble, no signals decided by over fitting to a biased engineer preference.

I agree, except it'll optionally accept the ID of your node in a web of trust, and it'll use a page rank customized for you.

Or you can put in two ID's and have it find sources that both parties trust.

Re: A search engine that favors text-heavy sites and punishes modern web design

#40

Post by marcos100 »

jerrre wrote: post #13
gtmb wrote: post #6

As everything in life flows in cycle, I predict the search engine that will de-throne Google will be like Google when it started - a simple variation of page rank. No smarts, no bubble, no signals decided by over fitting to a biased engineer preference.

I wouldn't say the existence of this page proves your prediction right (as it's not dethroning Google anytime soon). It's easy to forget that the goal of Google isn't to provide a useful search engine (at least not anymore), but the search engine is a by product of them wanting to show ads.

If Google isn't useful then nobody will use it.

The search engine and the ads are tightly coupled. A better search engine means it can predict with more accuracy what you are looking for and can serve you an even more targeted ad that increases the chance you'll click.

Post reply on HN