Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Didyougogo – An Altavista slayer

didyougogo.com

Post reply on HN
Print view View original

21–30 of 131 posts

Re: Show HN: Didyougogo – An Altavista slayer

#21

Post by misterman0 »

spchampion2 wrote: post #7

I searched for "Cnn" and got 0 results. I searched for "Amazon" and got a five random results, including the IMDB page for "Rambo, Part 2." If this were really like AltaVista, I'd get 3 trillion results and have to use advanced Boolean logic to cut that down to the most useful 7,000 - so I guess having no results is sort of easier...

My boolean logic is here: [1] I'm sure it has flaws.

Since the index had only five or six entries a couple of hours ago I set the matching to be wide instead of narrow. I'm also experimenting with loading the model with phrases, phrases and words or words only. I might have f-ed up the query parsing because of that. Remember, this is 0.1, fresh out of the press.

Searching the tree is here: [2]

Tokenization is here: [3]

[1] https://github.com/kreeben/resin/blob/master/src/Sir.Store/R...

[2] https://github.com/kreeben/resin/blob/master/src/Sir.Store/V...

[3] https://github.com/kreeben/resin/blob/master/src/Sir.Store/L...

Re: Show HN: Didyougogo – An Altavista slayer

#22

Post by misterman0 »

tcmb wrote: post #10

Kudos for your courage to make your great ambitions public from the start. 1. Does the site do any crawling on its own, or is the public index only fed from submissions? 2. It appears Umlaut/Unicode handling needs some work: When I search for "Käse" (German for 'cheese'), I get the response "0 results for 'Käse' in 'www' (0 ms)". At this point I'm not sure if there's actually 0 results or if it was actually sear…

Thanks!

1. You may submit a page. When I have a little more capacity that just 1 CPU/1 GB RAM I will also crawl.

2. I'll look into it. Thank you.

Re: Show HN: Didyougogo – An Altavista slayer

#24

Post by misterman0 »

gunkaaa wrote: post #5

I love it - well done. As always, the question is how it scales.

I was just talking to someone about scaling so I'm reusing what I said:

Scaling out technically and socially seems a little bit related. I want to scale out like this: a public search server (node) knows about other public nodes and the semantic topics their data carries. When a node cannot sufficiently answer a query it can reach out to other nodes by looking up a map of topic/list of nodes. Sharding by table/collection can also be solved the same way. That way, people owning public nodes can create queries that span tables they don't even host. They can build analytics using _their_ data _and_ the world's data. That's super-powerful.

Re: Show HN: Didyougogo – An Altavista slayer

#26

Post by misterman0 »

jhabdas wrote: post #13

Sounds too good to be true. What's the catch?

The catch is: this is 0.1 alpha software. I need a small team and some server capacity to get rolling. I need people to submit URLs. And a few hundred queries per second. That would scare the living shit out of big league search engines and might wake up some investor wanting to throw money at this.

Re: Show HN: Didyougogo – An Altavista slayer

#28

Post by misterman0 »

jhabdas wrote: post #11
spchampion2 wrote: post #7

I searched for "Cnn" and got 0 results. I searched for "Amazon" and got a five random results, including the IMDB page for "Rambo, Part 2." If this were really like AltaVista, I'd get 3 trillion results and have to use advanced Boolean logic to cut that down to the most useful 7,000 - so I guess having no results is sort of easier...

Great movie. I think it's available on Amazon...

Yes, indeed. I used the following titles to test my boolean logic:

[0] First Blood

[1] Rambo: First Blood Part II

[2] Rambo III

Query:

title:first blood

-title:rambo

Result:

[0] First Blood

(the best one in the bunch)

Re: Show HN: Didyougogo – An Altavista slayer

#29

Post by kps »

misterman0 wrote: post #22
tcmb wrote: post #10

Kudos for your courage to make your great ambitions public from the start. 1. Does the site do any crawling on its own, or is the public index only fed from submissions? 2. It appears Umlaut/Unicode handling needs some work: When I search for "Käse" (German for 'cheese'), I get the response "0 results for 'Käse' in 'www' (0 ms)". At this point I'm not sure if there's actually 0 results or if it was actually sear…

Thanks! 1. You may submit a page. When I have a little more capacity that just 1 CPU/1 GB RAM I will also crawl. 2. I'll look into it. Thank you.

Altavista ran on an AlphaServer 8400¹: maximum 14 × 612MHz CPUs, 28G RAM.

¹ https://en.wikipedia.org/wiki/AlphaServer#Turbolaser_Family

Post reply on HN