51–60 of 129 posts
This and the other engines seem to implement all the components of crawling, indexing, and searching strung together. Is there a reason for this? Wouldn't an option of, let's say, crawling + indexing made available separately, where others could built a search algorithm on top of, or just the crawling as a service made available. Are there stuff like these already available? Or is it just not a viable option?
Earlier quoted context omitted.
You don't need to remember it, just bookmark and tag however you like (it's anyway a waste of keystrokes to manual type the full domain for such a frequently used site like a search engine)
That is not how a large part of the citizens on the internet works. Hell, a not insignificant number of people will still "search" for Google in their address bar before they get to the actual googling
Earlier quoted context omitted.
I’m from Wales and it almost seems like a transliteration of the word “Mumble” - actual translation is “mwmial”
Welsh has the unfortunate combination of being unfamiliar to most English speakers, and not exotic enough to score diversity points.
I wondered if this approach would be feasible for a distributed crawler: https://github.com/mwmbl/mwmbl#crawling Also, your own posting appears to be missing from the index: https://mwmbl.org/?q=mwmbl+ycombinator (and, yes, another vote for changing the domain name; you can have a quirky project name, but if I can't remember the cat-walking-on-keyboard domain, I'm not going to use it)
I’m from Wales and it almost seems like a transliteration of the word “Mumble” - actual translation is “mwmial”
This and the other engines seem to implement all the components of crawling, indexing, and searching strung together. Is there a reason for this? Wouldn't an option of, let's say, crawling + indexing made available separately, where others could built a search algorithm on top of, or just the crawling as a service made available. Are there stuff like these already available? Or is it just not a viable option?
As for the rest, in order to perform well, the indexer needs to be built specifically tailored to the what the search engine is doing. Often you're scrounging for places to cram in individual bits to encode some additional piece of information about the term.
If a DBMS tries to support every use case, a search engine index does the opposite, it supports a singular use case and cuts every corner imaginable and then some to make that happen with as much resource frugality as possible.
This and the other engines seem to implement all the components of crawling, indexing, and searching strung together. Is there a reason for this? Wouldn't an option of, let's say, crawling + indexing made available separately, where others could built a search algorithm on top of, or just the crawling as a service made available. Are there stuff like these already available? Or is it just not a viable option?
There is common crawl: https://github.com/commoncrawl
But when it does find something it is very quick! So I'll give it a go.