Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: DeepHN – Full-text search of 30M Hacker News posts and linked webpages

deephn.org

Post reply on HN
Print view View original

61–70 of 118 posts

Re: Show HN: DeepHN – Full-text search of 30M Hacker News posts and linked webpages

#61

Post by ehmmmmmmmm »

abcc8 wrote: post #45

I'm curious how this site is compliant with copyright law. I've thought of working on similar projects, but the possibility of being sued has dissuaded me.

It's probably not compliant. As with all good things, if you're in the Land of the Free (TM), eventually people will hunt you down if you try to do good things.

Alternatively, base your site out of a country which gives you the freedom to do things like this. Build it in China and you'll be applauded instead of sued. Copyright suits for petty things like this would laughed off when it's something actually useful.

Re: Show HN: DeepHN – Full-text search of 30M Hacker News posts and linked webpages

#62

Post by clukic »

exdsq wrote: post #7

It’s fun to see Bitcoin posts from 2008-2010. I wonder how many bought and are now retired because of them. Seeing recommendations to use MtGox is pretty funny too https://news.ycombinator.com/item?id=1998144

I read about Bitcoin here over a decade ago. Took the time to setup a miner, and let it run for a week. I think I generated a few hundred coins worth less than a dollar total. Figured the electricity cost way more than that and deleted it all.

Re: Show HN: DeepHN – Full-text search of 30M Hacker News posts and linked webpages

#63

Post by NicoJuicy »

ehmmmmmmmm wrote: post #61
abcc8 wrote: post #45

I'm curious how this site is compliant with copyright law. I've thought of working on similar projects, but the possibility of being sued has dissuaded me.

It's probably not compliant. As with all good things, if you're in the Land of the Free (TM), eventually people will hunt you down if you try to do good things. Alternatively, base your site out of a country which gives you the freedom to do things like this. Build it in China and you'll be applauded instead of sued. Copyright suits for petty things like this would laughed off when it's something actually useful.

China's monopolies are far worse than in other countries.

Founders there have no choice but to sell or get copied by the platform that you're building on. It's baba or tenscent.

Re: Show HN: DeepHN – Full-text search of 30M Hacker News posts and linked webpages

#67

Post by Bender »

We're sorry but seekstorm-deephn doesn't work properly without JavaScript enabled. Please enable it to continue.

Could the folks with JS disabled get a small semi-static page with the top interesting HN stats, and a message that say something like "Here are some simple stats. For the full interactive experience, please enable javascript"?

Re: Show HN: DeepHN – Full-text search of 30M Hacker News posts and linked webpages

#68

Post by notional »

dang wrote: post #56
myth_drannon wrote: post #9

Earlier quoted context omitted.

Earlier post 2009: https://news.ycombinator.com/item?id=599852 "Well this is an exceptionally cute idea, but there is absolutely no way that anyone is going to have any faith in this currency." Written by co-founder/CEO of Pachyderm and first employee of RethinkDB none the less.

People love to pick out the comments of the past that seem most wrong now, because—let's be honest—it makes us feel superior. But if you go back and look at that original 3-comment Bitcoin thread, what you find is that those 3 comments span an entire spectrum: elated, skeptical, and bewildered: https://news.ycombinator.com/item?id=599852 . The first cryptocurrency thread on HN—about 9 months earlier—was even more int…

Ho boy looking up Xinjiang then reading this comment was a trip. Any ways Dropbox is pretty great and seeing the differences in sentiment towards TSLA stock compared to Bitcoin over the years has been great. Can't pick all the winners.

Re: Show HN: DeepHN – Full-text search of 30M Hacker News posts and linked webpages

#69

Post by wolfgarbe »

freediver wrote: post #53
wolfgarbe wrote: post #51

Earlier quoted context omitted.

Crawling speed is between 100...1000 pages per second. We crawled about 4 million linked unique web pages The pricing for an Index like DeepHN would be $99/month: While we are indexing 30 million Hacker news posts, for DeepHN we are combining a single HN story with all its comments and its linked webpage into a single SeekSorm document. So that we index just 4 million SeekStorm documents. Yes, it would be possible to…

What is being used as a crawler and is it integrated with Seekstorm? The same article referenced here https://deephn.org/?q=how+to+be+productive&filter=%7B%22hash... contains the phrase 'well-defined' Any idea why doesn't the article surface when searching for this ?

The crawler is a part of SeekStorm.

"Well-defined": Just a guess: We are doing key text extraction, i.e. we try not to index boilerplate stuff an and menu items. As "well-defined" is within a short list item, it might be accidentally skipped.

So, that is not yet perfect, and considering the diversity in web page structure it probably never will. But we will try to improve.

Re: Show HN: DeepHN – Full-text search of 30M Hacker News posts and linked webpages

#70

Post by wolfgarbe »

Bender wrote: post #67

We're sorry but seekstorm-deephn doesn't work properly without JavaScript enabled. Please enable it to continue. Could the folks with JS disabled get a small semi-static page with the top interesting HN stats, and a message that say something like "Here are some simple stats. For the full interactive experience, please enable javascript"?

Good idea!
Post reply on HN