441–450 of 735 posts
For example, if you search "jehovahs witnesses", all pages from jw.org are missing.
Exactly the same thing happened when I searched "mormons" - the official website is missing and it only brings up sects/hate/conspiracies against mormons.
Earlier quoted context omitted.
Wait till you learn that modern CPUs run billions of cycles per second. With multiple cores in parallel! And they can reach transfer rates of tens of gigabytes per second to RAM, or around a terabyte per second into L3.
And then you add a single HTTP request and everything tones down to the speed of the web. Or I/O. Or DB call.
Earlier quoted context omitted.
It's very rare that I see a project on HN I can see myself using. This is one. Like others have said, the results can be a little rough. But they're rough in a way I think is much more manageable than the idiosynchrosies of more 'clever' search engines.
I think you need to approach it more like grep than google. It's a forgotten art, dealing with this type of dumb search engine. Like if you search for "How do I make a steak", you aren't going to get very good results. But a better query is "Steak Recipe", as that is at least a conceivable H1-tag.
Earlier quoted context omitted.
... is this Galileo 2021 a reference that I am not understanding?
Yup, but so far no one got it. There was such an app in the early 2000's, before Google went mainstream, and Altavista-like engines were not good: Copernic 2000. I guess I'm officially old now.
This is a pattern I see over and over again, some research group or academics show that something can be done (summaries that make sense and are true summaries, evolutionary algorithm FPGA programming, real time gaze prediction, etc) and there's a few published code repos and a bit of news, then 'poof' - no where to be seen for 15 years or more.
As everything in life flows in cycle, I predict the search engine that will de-throne Google will be like Google when it started - a simple variation of page rank. No smarts, no bubble, no signals decided by over fitting to a biased engineer preference.
As dev I would love search engine which would only do search to stackoverflow github issues, documentation etc.
Earlier quoted context omitted.
Love it, kudos! This is great for developers and others who Just Need Answers and not shopping or entertainment. If you're looking for feedback, both from a UI design and utility standpoint, you might consider "inlining" results from selected sites, e.g. Wikipedia, stacked change, etc. Having worked on search for a long time, inlining (onebox etc) is a big reason users choose Google, and that channelers fail to get t…
If Google starts showing interesting text-heavy links instead of vapid listicles and storefronts, I have accomplished everything I ever could dream of.
Earlier quoted context omitted.
Which software do you use to index the sites?
I wrote it myself from scratch. I have some metadata in mariadb, but the index is bespoke. A design sketch of the index is that it uses one file with sorted URL IDs, one with IDs of N-grams (i.e. words and word-pairs) referring to ranges in the URL file; as well as a dictionary for relating words to word-IDs; that's a GNU Trove hash map I modified to use memory map data instead of direct allocated arrays. So when you…
Beware, I got the impression straight away that some sites were censored from the results for no good reason. For example, if you search "jehovahs witnesses", all pages from jw.org are missing. Exactly the same thing happened when I searched "mormons" - the official website is missing and it only brings up sects/hate/conspiracies against mormons.