Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mwmbl: Free, open-source and non-profit search engine

mwmbl.org

Post reply on HN
Print view View original

41–50 of 129 posts

Re: Mwmbl: Free, open-source and non-profit search engine

#41

Post by kristopolous »

marginalia_nu wrote: post #35
kristopolous wrote: post #33

Earlier quoted context omitted.

There was an effort in the early 90s to have search as a protocol so you could have a query and then select the domains you want to run it on and return an aggregate result. It was 100% abandoned and I think that's a mistake. It'd be nice to explore some of those ideas again

I think a big part of the problem is that domains in isolation don't provide the best search results. Out-of-band information like (global) anchor texts or click data makes search perform so much better. If I want to learn how to do an INNER JOIN in MariaDB, this is the authoritative source: https://mariadb.com/kb/en/join-syntax/ The problem being that INNER JOIN isn't particularly important to that page using most I…

There's more to it than that.

What if instead of crawling the php generation of database rows with a bunch of cruft, the administrator published some kind of schema with scraping and querying rules and you could alternatively make a single call to capture all of the data in a sematic schema.

You can still do all the stuff you're talking about but it could make search more coherent.

An entry for that humans and an entry for the computers.

You can't trust everybody like this sure, but say imdb, discogs, wikipedia, all of which provide database dumps anyways (eg: https://datasets.imdbws.com/). That's what I'm advocating for revisiting. Lots of legit sites such as universities, newspapers, public records offices...

You could even have a search toggle "screened sources" or whatever for the ones that make the cut

Re: Mwmbl: Free, open-source and non-profit search engine

#43

Post by 9dev »

BLKNSLVR wrote: post #7
romwell wrote: post #4

OK, the obvious question: Why go with an unpronounceable name? I mean, great that it was made, but I can't even tell people I'm using... mwumble? But it's spelled em-doubleyou-em-bee-el dot org.

It's pronounced mumble. An explanation is at the very bottom of the github Readme, quoting: > How do you pronounce "mwmbl"? > Like "mumble". I live in Mumbles, which is spelt "Mwmbwls" in Welsh. But the intended meaning is "to mumble", as in "don't search, just mwmbl!"

Marketing 101: don’t try to be clever with your brand name :)

Re: Mwmbl: Free, open-source and non-profit search engine

#44

Post by AlphaCerium »

seanthemon wrote: post #24
jw_cook wrote: post #21

Earlier quoted context omitted.

While it doesn't refute your point, the Frequently Asked Question section does give an explanation for the consonant soup: it's Welsh. > How do you pronounce "mwmbl"? Like "mumble". I live in Mumbles, which is spelt "Mwmbwls" in Welsh. But the intended meaning is "to mumble", as in "don't search, just mwmbl!"

I highly recommend grabbing something simpler to say and remember to redirect to your site. You're going to need a large amoung of inertia to get people to comfortably use an odd domain name.

Arguably, Google was probably a odd name for a search engine to people in the 90s that weren't maths-savvy.

Re: Mwmbl: Free, open-source and non-profit search engine

#45

Post by robinduckett »

mdaniel wrote: post #15

I wondered if this approach would be feasible for a distributed crawler: https://github.com/mwmbl/mwmbl#crawling Also, your own posting appears to be missing from the index: https://mwmbl.org/?q=mwmbl+ycombinator (and, yes, another vote for changing the domain name; you can have a quirky project name, but if I can't remember the cat-walking-on-keyboard domain, I'm not going to use it)

I’m from Wales and it almost seems like a transliteration of the word “Mumble” - actual translation is “mwmial”

Re: Mwmbl: Free, open-source and non-profit search engine

#46

Post by marginalia_nu »

kristopolous wrote: post #38
marginalia_nu wrote: post #37

Earlier quoted context omitted.

Question-answering is something Google pivoted toward with great enthusiasm but never quite nailed down. They'd sometimes get some questions right, but it was more of a broken clock sort of a deal.

Most implementations of this have a race towards generalities. The biggest problem used to be when seemingly the whole internet was satisfied with an answer that is extremely wrong and broken when you do it. Chatgpt can work though this without getting into a weird markov cycle maybe half the time which is great. Patterns like "Hey I tried that. It still doesn't work, can you give me another option"

ChatGPT has other failure modes. When a question doesn't have an answer written down somewhere, it really struggles. A case is something like "how do I write a parquet file in Java without using Hadoop".

This not at all trivial but quite possible[1], but ChatGPT will in 100% of the time either hallucinate APIs, disregard the instructions to not use Hadoop or give otherwise plausible but incorrect-looking answers.

The trick is that it isn't doable by simply finding the correct dependencies and API calls, you need extract and override filesystem classes from the Hadoop project to cut those ties.

[1] https://github.com/strategicblue/parquet-floor

Re: Mwmbl: Free, open-source and non-profit search engine

#47

Post by brettermeier »

AlphaCerium wrote: post #44
seanthemon wrote: post #24

Earlier quoted context omitted.

I highly recommend grabbing something simpler to say and remember to redirect to your site. You're going to need a large amoung of inertia to get people to comfortably use an odd domain name.

Arguably, Google was probably a odd name for a search engine to people in the 90s that weren't maths-savvy.

But it's a normal word, unlike "mwmbl" (I had to look it up, couldn't remember where the "b" and "w" goes after some seconds).

Re: Mwmbl: Free, open-source and non-profit search engine

#49

Post by dmurray »

robinduckett wrote: post #45
mdaniel wrote: post #15

I wondered if this approach would be feasible for a distributed crawler: https://github.com/mwmbl/mwmbl#crawling Also, your own posting appears to be missing from the index: https://mwmbl.org/?q=mwmbl+ycombinator (and, yes, another vote for changing the domain name; you can have a quirky project name, but if I can't remember the cat-walking-on-keyboard domain, I'm not going to use it)

I’m from Wales and it almost seems like a transliteration of the word “Mumble” - actual translation is “mwmial”

Welsh has the unfortunate combination of being unfamiliar to most English speakers, and not exotic enough to score diversity points.

Re: Mwmbl: Free, open-source and non-profit search engine

#50

Post by ddorian43 »

krishadi wrote: post #40

This and the other engines seem to implement all the components of crawling, indexing, and searching strung together. Is there a reason for this? Wouldn't an option of, let's say, crawling + indexing made available separately, where others could built a search algorithm on top of, or just the crawling as a service made available. Are there stuff like these already available? Or is it just not a viable option?

There is common crawl: https://github.com/commoncrawl
Post reply on HN