Oh, I dream of a day where there are multiple useful search engines, specialized for different purposes. You're doing God's work here. Thanks and good luck.
Kind of reminds me of the past like alta vista and dogpile.
A search engine that favors text-heavy sites and punishes modern web design
141–150 of 735 posts
Re: A search engine that favors text-heavy sites and punishes modern web design
#142curious how do you afford the infrastructure? I found that the hardest part of running a search engine.
Re: A search engine that favors text-heavy sites and punishes modern web design
#143Earlier quoted context omitted.
I'm intrigued by this experiment but I can't visualize it. What do you mean by boring results? Would combing through a library (the one with paper books) also produce boring results? What's your ideal results?
Perhaps a counter example, something that is interesting. Anecdotally. This, of all things, is the top result in my search: https://tft.brainiac.com/archive/0303/msg00037.html . Which is strange to me because I don't recognize tft.brainiac. I click, it's a list of biological relationships among Hymenoptera, including a reference to genus of the wasps I studied, presumably in a biological relationship (host/parasite)…
Yes, thanks. That helps.
If I understand it correctly, you're interested in bits and pieces of new information that's indirectly related to your object of interest. Degree 2 and 3 in Six Degrees of Kevin Bacon, so to speak. You know degree 0 like the back of your hand and you've seen almost everything closely connected. Finding novel, interesting things is getting more difficult.
Have you thought about cataloging all the related stuff you stumble upon? Something in between loose notes and what Moby Dick is to cetology.
Re: A search engine that favors text-heavy sites and punishes modern web design
#144Wow, that's awesome. Great work! For a simple test, I searched "fall of the roman empire". In your search engine, I got wikipedia, followed by academic talks, chapters of books, and long-form blogs. All extremely useful resources. When I search on google, I get wikipedia, followed by a listicle "8 Reasons Why Rome Fell", then the imdb page for a movie by the same name, and then two Amazon book links, which are totall…
I did a search for "George Washington"
First result after Wikipedia:
"Radiophone Transmitter on the U.S.S. George Washington (1920)
In 1906, Reginald Fessenden contracted with General Electric to build the first alternator transmitter. G.E. continued to perfect alternator transmitter design, and at the time of this report, the Navy was operating one of G.E.'s 200 kilowatt alternators http://earlyradiohistory.us/1919wsh.htm "
Another result in the first few:
" - VANDERBILT, GEORGE WASHINGTON
PH: (800) ###-#233 FX: (#03) 641-5###. https://www.ScottWinslow.com/manufacturer/VANDERBILT_GEORGE_... "
And just below that terrible result:
"I Looked and I Listened -- George Washington Hill extract (1954)
Although the events described in this account are undated, they appear to have occurred in late 1928. I Looked and I Listened, Ben Gross, 1954, pages 104-105: Programs such as these called for the expenditure of larger sums than NBC had anticipated. It be http://earlyradiohistory.us/1954ayl2.htm "
Dramatically worse than Google.
---
Ok, how about a search for "Rome" then? Surely it'll pull some great text results for the city or the ancient empire.
First result after Wikipedia:
"Home | Rome Daily Sentinel
Reliable Community News for Oneida, Madison and Lewis County http://romesentinel.com/"
The fourth result for searching "Rome":
"Glenn's Pens - Stores of Note
Glenn's Pens, web site about pens, inks, stores, companies - the pleasure of owning and using a pen of choice. Direcdtory of pen stores in Europe. http://www.marcuslink.com/pens/storesofnote/roma.html"
Again, dramatically worse than Google.
---
Ok, how about if I search for "British"?
First result after Wikipedia:
"BRITISH MINING DATABASE
British_Mining_Database http://www.users.globalnet.co.uk/~lizcolin/bmd.htm "
And after that:
"British Virgin Islands
Many of these photos were taken on board the Spirit of Massachusetts. The sailing trip was organized by Toto Tours. Images Copyright © Lowell Greenberg Home Up Spring Quail Gardens Forest Home Lake Hodges Cape Falcon Cape Lookout, Oregon Wahkeena http://www.earthrenewal.org/british_virgin_islands2.htm"
Again, far off the mark and dramatically worse than Google.
I like the idea of Google having lots of search competition, this isn't there yet (and I wouldn't expect it to be). I don't think overhyping its results does it any favors.
Re: A search engine that favors text-heavy sites and punishes modern web design
#145Love this! Is there any way someone could help contribute to this?
Re: A search engine that favors text-heavy sites and punishes modern web design
#146Oh, I dream of a day where there are multiple useful search engines, specialized for different purposes. You're doing God's work here. Thanks and good luck.
The Flying Spaghetti Monster wants to have a word with you.
(edit) That is a nice dream though.
Re: A search engine that favors text-heavy sites and punishes modern web design
#147Yeah so this is my project. It's very much a work in progress, but occasionally I think it works remarkably well for something I cobbled together alone out of consumer hardware and home-made code :-)
Great work. Working on an alternative search engine too. Take a look at my profile.
Re: A search engine that favors text-heavy sites and punishes modern web design
#148Earlier quoted context omitted.
Which software do you use to index the sites?
I wrote it myself from scratch. I have some metadata in mariadb, but the index is bespoke. A design sketch of the index is that it uses one file with sorted URL IDs, one with IDs of N-grams (i.e. words and word-pairs) referring to ranges in the URL file; as well as a dictionary for relating words to word-IDs; that's a GNU Trove hash map I modified to use memory map data instead of direct allocated arrays. So when you…
Good stuff. I've also been toying with doing some homegrown search engine indexing (as an exercise in scalable systems), and this is a fantastic result and great inspiration.
Definitely want to see more people doing that kind of low-level work instead of falling back to either 'use elasticsearch' or 'you can't, you're not google'.
Re: A search engine that favors text-heavy sites and punishes modern web design
#149Wow, that's awesome. Great work! For a simple test, I searched "fall of the roman empire". In your search engine, I got wikipedia, followed by academic talks, chapters of books, and long-form blogs. All extremely useful resources. When I search on google, I get wikipedia, followed by a listicle "8 Reasons Why Rome Fell", then the imdb page for a movie by the same name, and then two Amazon book links, which are totall…
However, when searching for "haskell type inference algorithm" I get completely useless results.
Re: A search engine that favors text-heavy sites and punishes modern web design
#150Wow, that's awesome. Great work! For a simple test, I searched "fall of the roman empire". In your search engine, I got wikipedia, followed by academic talks, chapters of books, and long-form blogs. All extremely useful resources. When I search on google, I get wikipedia, followed by a listicle "8 Reasons Why Rome Fell", then the imdb page for a movie by the same name, and then two Amazon book links, which are totall…
I did a search for "George Washington" First result after Wikipedia: "Radiophone Transmitter on the U.S.S. George Washington (1920) In 1906, Reginald Fessenden contracted with General Electric to build the first alternator transmitter. G.E. continued to perfect alternator transmitter design, and at the time of this report, the Navy was operating one of G.E.'s 200 kilowatt alternators http://earlyradiohistory.us/1919w…
Hobby project leads angry person to interesting and unexpected material; angry person remains angry. Details at six.