Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mwmbl: Free, open-source and non-profit search engine

mwmbl.org

Post reply on HN
Print view View original

31–40 of 129 posts

Re: Mwmbl: Free, open-source and non-profit search engine

#31

Post by eviks »

mdaniel wrote: post #15

I wondered if this approach would be feasible for a distributed crawler: https://github.com/mwmbl/mwmbl#crawling Also, your own posting appears to be missing from the index: https://mwmbl.org/?q=mwmbl+ycombinator (and, yes, another vote for changing the domain name; you can have a quirky project name, but if I can't remember the cat-walking-on-keyboard domain, I'm not going to use it)

You don't need to remember it, just bookmark and tag however you like (it's anyway a waste of keystrokes to manual type the full domain for such a frequently used site like a search engine)

Re: Mwmbl: Free, open-source and non-profit search engine

#32

Post by xenodium »

If keen on some minor feedback (specially for mobile), you can likely cut down on landing page text:

From:

    MWMBL

    [Search on mwmbl...]

    Welcome to mwmbl, the free, open-source and non-profit search engine.

    You can start searching by using the search bar above!

    Find more on

    [Github] [Wiki]
To:

    MWMBL

    [Search on mwmbl...]

    A free, open-source and non-profit search engine.

    [Github] [Wiki]

Re: Mwmbl: Free, open-source and non-profit search engine

#33

Post by kristopolous »

marginalia_nu wrote: post #30
mdtrooper wrote: post #28

I remember https://yacy.net/ but the big problem of this project was java and had not implementations in others languages. I mean it as imagine torrent was only in perl.

YaCy's big problem is that distributed search is a bad idea that will never perform well. Search is as fast as the data is local.

There was an effort in the early 90s to have search as a protocol so you could have a query and then select the domains you want to run it on and return an aggregate result.

It was 100% abandoned and I think that's a mistake. It'd be nice to explore some of those ideas again

Re: Mwmbl: Free, open-source and non-profit search engine

#34

Post by kristopolous »

marginalia_nu wrote: post #29
evolve2k wrote: post #13

Love that you folks are working on this. We desperately need more diversity in search options. Much is at stake in this arena.

I'd love to see more competition in search. Feels like everyone right now gets tripped up on trying to emulate Google, which is a trap even if you succeed. Nobody is going to out-Google Google. ChatGPT's recent huge success in performing a specific tasks previously within the domain of Google by doing something other than they are is a good example of this.

Chatgpt is so much more useful for things that are specific and complex than Google is.

Google used to be good at it but it's now utterly befuddled by specificity and returns such garbage that I had given up.

But the form of "I'm doing this, I'm seeing this and I'm wondering if X is possible" chatgpt is solid on that - basically a personal stack overflow

Re: Mwmbl: Free, open-source and non-profit search engine

#35

Post by marginalia_nu »

kristopolous wrote: post #33
marginalia_nu wrote: post #30

Earlier quoted context omitted.

YaCy's big problem is that distributed search is a bad idea that will never perform well. Search is as fast as the data is local.

There was an effort in the early 90s to have search as a protocol so you could have a query and then select the domains you want to run it on and return an aggregate result. It was 100% abandoned and I think that's a mistake. It'd be nice to explore some of those ideas again

I think a big part of the problem is that domains in isolation don't provide the best search results. Out-of-band information like (global) anchor texts or click data makes search perform so much better.

If I want to learn how to do an INNER JOIN in MariaDB, this is the authoritative source: https://mariadb.com/kb/en/join-syntax/

The problem being that INNER JOIN isn't particularly important to that page using most IR measures of importance, it's also primarily in a -block which is typically further de-prioritized. To learn that this is an important link, you need to look outside of mariadb.com.

Re: Mwmbl: Free, open-source and non-profit search engine

#37

Post by marginalia_nu »

kristopolous wrote: post #34
marginalia_nu wrote: post #29

Earlier quoted context omitted.

I'd love to see more competition in search. Feels like everyone right now gets tripped up on trying to emulate Google, which is a trap even if you succeed. Nobody is going to out-Google Google. ChatGPT's recent huge success in performing a specific tasks previously within the domain of Google by doing something other than they are is a good example of this.

Chatgpt is so much more useful for things that are specific and complex than Google is. Google used to be good at it but it's now utterly befuddled by specificity and returns such garbage that I had given up. But the form of "I'm doing this, I'm seeing this and I'm wondering if X is possible" chatgpt is solid on that - basically a personal stack overflow

Question-answering is something Google pivoted toward with great enthusiasm but never quite nailed down. They'd sometimes get some questions right, but it was more of a broken clock sort of a deal.

Re: Mwmbl: Free, open-source and non-profit search engine

#38

Post by kristopolous »

marginalia_nu wrote: post #37
kristopolous wrote: post #34

Earlier quoted context omitted.

Chatgpt is so much more useful for things that are specific and complex than Google is. Google used to be good at it but it's now utterly befuddled by specificity and returns such garbage that I had given up. But the form of "I'm doing this, I'm seeing this and I'm wondering if X is possible" chatgpt is solid on that - basically a personal stack overflow

Question-answering is something Google pivoted toward with great enthusiasm but never quite nailed down. They'd sometimes get some questions right, but it was more of a broken clock sort of a deal.

Most implementations of this have a race towards generalities.

The biggest problem used to be when seemingly the whole internet was satisfied with an answer that is extremely wrong and broken when you do it.

Chatgpt can work though this without getting into a weird markov cycle maybe half the time which is great.

Patterns like "Hey I tried that. It still doesn't work, can you give me another option"

Re: Mwmbl: Free, open-source and non-profit search engine

#39

Post by DandyDev »

eviks wrote: post #31
mdaniel wrote: post #15

I wondered if this approach would be feasible for a distributed crawler: https://github.com/mwmbl/mwmbl#crawling Also, your own posting appears to be missing from the index: https://mwmbl.org/?q=mwmbl+ycombinator (and, yes, another vote for changing the domain name; you can have a quirky project name, but if I can't remember the cat-walking-on-keyboard domain, I'm not going to use it)

You don't need to remember it, just bookmark and tag however you like (it's anyway a waste of keystrokes to manual type the full domain for such a frequently used site like a search engine)

That is not how a large part of the citizens on the internet works. Hell, a not insignificant number of people will still "search" for Google in their address bar before they get to the actual googling

Re: Mwmbl: Free, open-source and non-profit search engine

#40

Post by krishadi »

This and the other engines seem to implement all the components of crawling, indexing, and searching strung together. Is there a reason for this? Wouldn't an option of, let's say, crawling + indexing made available separately, where others could built a search algorithm on top of, or just the crawling as a service made available. Are there stuff like these already available? Or is it just not a viable option?
Post reply on HN