561–570 of 735 posts
Add an OpenSearch file: https://developer.mozilla.org/en-US/docs/Web/OpenSearch Maybe something like: Marginalia Marginalia Search - a search engine that favors text-heavy sites and punishes modern web design UTF-8 https://search.marginalia.nu/favicon.ico https://search.marginalia.nu/ and then add: to your page's .
Thanks everyone who suggested this.
Earlier quoted context omitted.
It's very rare that I see a project on HN I can see myself using. This is one. Like others have said, the results can be a little rough. But they're rough in a way I think is much more manageable than the idiosynchrosies of more 'clever' search engines.
I think you need to approach it more like grep than google. It's a forgotten art, dealing with this type of dumb search engine. Like if you search for "How do I make a steak", you aren't going to get very good results. But a better query is "Steak Recipe", as that is at least a conceivable H1-tag.
They used the naive approach: you searched for "steak", and they would bring the pages which included the word "steak".
The problem is that people could fool these engines by adding a long sequence like "steak, steak, steak, steak, steak, steak" to their site -- to pretend that they were the most authoritative page about steaks.
Google's big innovation was to count the referrers -- how many pages used the word "steak" to link to that particular page.
The rest is history.
Earlier quoted context omitted.
Good comparison. Reminds me of an analogy I like to make of today's web, which is it feels like browsing through a magazine store — full of top 10s, shallow wow-factoids, and baity material. I genuinely believe terrible results like this are making society dumber.
The context matters. I'd happily read "Top 10" lists on a website if the site itself was dedicated to that one thing. "Top 10 Prog Rock albums", while a lazy, SEO-bait title, would at least be credible if it were on a music-oriented website. But no, these stories all come from cookie-cutter "new media" blog sites, written by an anonymous content writer who's repackaged Wikipedia/Discogs info into Buzzfeed-style copy…
As a sufferer of Tinnitus, and having spent near 100 hours researching it, I found a few sites I had never seen offering great data and tools. Thank you
Earlier quoted context omitted.
I like this design decision. It pays you back for choosing your search terms carefully.
I’m not against a stemmer, actually, just against the aggressive concordances (?) that Google now employs, like when it shows me X in Banach spaces (the classical, textbook case) when I’m specifically searching for X in Fréchet spaces (the generalization I want to find but am not sure exists); of course Banach spaces and Fréchet spaces are almost exclusively encountered in the same context, but it doesn’t mean that o…
Very cool idea! Room for lots of improvement, keep working on it, I like the direction this is going.
Author said they threw it together on consumer hardware. How big is the index? (TB used or entries) how is it realised?
I'm pretty much interested in this since I myself am crawling some pages for my own "search index".
Oh and thx for making and posting. Added it as a keyword to firefox
Edit: Just realized that my question is a bit shallow. What I'm particular interested in is the storage before the indexing. I'm trying to store the raw html so that I can reindex everything with better algorithms, but I'm hitting many limits. It takes a few minutes getting the size of a site-directory (every site has it's own dir) and I'm at a point where I can't reasonably manage the scrape-versioning over git and I cycled through a few filesystems only to find that the metadata management kind of sucks for most of them. It's rather interesting how we store such files and I'm thinking about storing a few sites in a simple sqlite format for easy access and search. I'm thinking about a a few low overhead solutions like facebooks project haystack (implemented open source in seaweedfs) or something similar... Hopefully this gives some context to the question of storage and sites that are indexed
Earlier quoted context omitted.
Search engines whose revenue is based on advertising will ultimately be tuned to steer you to the ad foodchain. All the incentives are aligned towards and all the metrics ultimately in service of, profit for advertisers. Not in the 99% of people who can convinced to consume something by ads? Welp, screw you.
Search engines should be something you pay for. Surely search engine powerusers can afford to pay for such a service. If Google makes $1 per user per month or something, that's not too high a bar to get over.