Live data from Hacker News

Ask HN: Where can I host a large data-set (35GB) in public?

news.ycombinator.com

11–13 of 13 posts

Re: Ask HN: Where can I host a large data-set (35GB) in public?

#11
post #10
post #4

How big is it if you gzip or bz2 it? I've found compression to be pretty effective for datasets like this.

Good suggestion. gzip reduced one file from 2332528640 to 129440511

Bzip may do yet better, though it will take longer. 7z may be worth a try too. You're getting into the range where you could just toss the whole thing up somewhere without much hassle. If you're not trying to make money, I'd just throw the whole data set out there on a torrent. Why take the time to write an access service when you don't even know what people are going to want to do with it? First thing a serious researcher is likely to do anyhow is just scrape the whole thing off your website, at much greater expense to you.

Re: Ask HN: Where can I host a large data-set (35GB) in public?

#13

Would infochimps.com be useful for this?

Yes, we'd be love to host this dataset for free. Send an email to upload@infochimps.com and we can arrange the transfer.

Sorry! Accidentally clicked the "down" button on your comment and there's no way to undo it.
Post reply on HN