Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: BestSFBooks, Mashup of the Best SF/Fantasy Books

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 26 posts

Re: Show HN: BestSFBooks, Mashup of the Best SF/Fantasy Books

#12

Post by gurgeous »

natbro wrote: post #3

OK, I have to admit at first I thought there was something wrong with the site because I didn't recognize enough books... but after reading some excerpts of books off the lists, I'm psyched -- turns out I just haven't been finding the good books for a long time, so now I can. Suggests: * show me excerpts on-page (if possible from amzn?) * allow community +1/-1 on books and generate lists based on top-rated by site us…

Thanks Nat - I fixed the like button. Turns out that you have to specify the href param when using the iframe, unlike twitter.

Re: Show HN: BestSFBooks, Mashup of the Best SF/Fantasy Books

#14

Post by bgraves »

gurgeous wrote: post #9
bgraves wrote: post #6

"As always, data acquisition and cleanup was the hardest part." I'm most interested in this piece of the project. What were your particular tools and methodologies? How long did it take you, once you identified your data sources? Any interesting stumbling blocks or problems that were solved along the way?

I've written a lot of data tools as part of Urbanspoon and subsequent startups. I like to collect publicly available data, clean it up, normalize it, and then release it in a more useful way. Hot tips for crawling data: - Cache pages locally while you work on the indexing - Nokogiri is awesome - Don't be afraid to use regular expressions - Initially, put data into a spreadsheet (not the db). That way it can be checke…

Thanks! I've been doing some scraping projects lately and really like it a lot. There's a pretty steep learning curve, but it gets easier and easier as you go along, I think.

1. Caching pages is definitely a great idea while debugging. Especially if the data source has a request limit :)

2. I've never heard of Nokogiri, but it looks like BeautifulSoup for Ruby. I've found that Python has worked for everything I need so far, but thanks for the reference.

3. I suck so bad at regex, but using it more will help me climb that mountain.

4. One tip I've used is writing out the "INSERT INTO TABLE..." statements along with the scraped results. I definitely use CSV (and Google Refine) for general clean up and spot checking.

5. You should write a 'Data Scraping One-liners Explained' ebook :)

Re: Show HN: BestSFBooks, Mashup of the Best SF/Fantasy Books

#16

Post by phil »

gurgeous wrote: post #9
bgraves wrote: post #6

"As always, data acquisition and cleanup was the hardest part." I'm most interested in this piece of the project. What were your particular tools and methodologies? How long did it take you, once you identified your data sources? Any interesting stumbling blocks or problems that were solved along the way?

I've written a lot of data tools as part of Urbanspoon and subsequent startups. I like to collect publicly available data, clean it up, normalize it, and then release it in a more useful way. Hot tips for crawling data: - Cache pages locally while you work on the indexing - Nokogiri is awesome - Don't be afraid to use regular expressions - Initially, put data into a spreadsheet (not the db). That way it can be checke…

Yes, repleceng ell vewels weth the letter E clerle empreves dete qelete ;)

Re: Show HN: BestSFBooks, Mashup of the Best SF/Fantasy Books

#18

Post by 100k »

Cool! Very nice.

I'm sort of slowly working my way through the double winners of the Hugo and Nebula (kind of a lifetime goal, I guess).

It took me a while to find the "Hall of Fame" for books, but that is what immediately wanted from a site like this (I don't care so much about the year-by-year rankings). Maybe make it more prominent?

EDIT: also, I think there's a big difference between nominations and winning. Would be cool to sort based on actually won awards, not just nominations.

Post reply on HN