481–490 of 735 posts
Earlier quoted context omitted.
Nice, what are you using to crawl the web?
It's pretty much all bespoke. I use external libraries for parsing HTML (JSoup) and robots.txt; but that's about it.
Earlier quoted context omitted.
PS. I added Marginalia as a search option (even the default for now) in Firefox Nightly (on Android). In case others want to, under settings for search, you can add other, then name, and then: https://search.marginalia.nu/search?query=%s
After a good amount of searching it doesn't seem possible to add Marginalia as default search in firefox (84.0b8) on Debian. I did not expect this to not be available.
Quoted from the linked site: > Convenience functions have been added, and the search engine can now perform simple calculations and unit conversions. Try 1 pint in cubic centimeters, or 50+sqrt(pi). This functionality is still under development, be patient if it doesn't work. Why would you make any ever so small effort to implement calculations? I don't get it. If your search engine enabled me to find more useful sea…
How else do you do unit conversions? I use Google because it's far easier than any other software I've tried. Mainly because it's more forgiving of errors. It knows that "34 fset in msters" is 10.3632 meter. This search engine isn't, though, so I wouldn't waste time trying to discover its unit conversion syntax rules.
Search for playwright waitForSelector and you land in pretty useless page. I'm all in for text websites, but something like playwright.dev documentation is top notch - fuzzy search being key thing.
Yeah I wasn't really planning for this to blow up like it did today. It's currently sitting at about 35% of the index size I usually aim for, so besides the stuff I can't index because it's behind CDNs, there's a lot of pages it just hasn't gotten to yet. playwright.dev is pretty low on the priority list because it has a metric crap-ton of javascript on its front page. The crawler has visited it, looked at it, and pu…
Earlier quoted context omitted.
I must congratulate you on this achievement. That's certainly a useful take on search. Nonetheless, even when coding, one should also consider thoroughly the UX and how it would be addressing the others. Saying "unsupported word" is much more sympathetic than "needs to be a word" (where you define what a "word" is, and the general user is unaware of such definition).
Fair point, I refined the phrasing a bit. > The term "𓀀" contains characters that are not currently supported
Earlier quoted context omitted.
... is this Galileo 2021 a reference that I am not understanding?
Yup, but so far no one got it. There was such an app in the early 2000's, before Google went mainstream, and Altavista-like engines were not good: Copernic 2000. I guess I'm officially old now.
Earlier quoted context omitted.
Thanks for your response! Hope you can implement this as well without too much trouble. I wonder if you could just assume UTF-8 to be the default these days. I imagine that to fix many other cases as well.
Haha! I did actually assume UTF-8 at first, but being a search engine has a lot of older websites, I sadly got a lot of encoding errors doing that, too.
Here is the second result when you search for “cat food”. It takes you to some old dudes entire family tree with full history and biographies… it even uses sub domains and everything! Crazy!
Wow, that's awesome. Great work! For a simple test, I searched "fall of the roman empire". In your search engine, I got wikipedia, followed by academic talks, chapters of books, and long-form blogs. All extremely useful resources. When I search on google, I get wikipedia, followed by a listicle "8 Reasons Why Rome Fell", then the imdb page for a movie by the same name, and then two Amazon book links, which are totall…
Cool, it appears that the trend towards JS may be causing self-selection -- if a page has a high amount of JS, it is highly unlikely to contain anything of value.
As a quick test, I searched for the name of one of my favorite game series: "Baldur's Gate" (on its own, no qualifiers, properly spelled - I would usually spell it "baldurs gate" on Google, but I decided to give this one the best chance). I search for info around video games a lot, so that's quite representative of a good chunk of my web searches, and I pretty much know the top sites Google would give me for that que…
Because in some cases it returns arguably better/more to the point results, than other search engines – for example search for “Douglas Engelbart” or “Ted Nelson”. I thought that I’ve searched everything for those two yet marginalia gave results otherwise I would have never seen,.