Show HN: BestSFBooks, Mashup of the Best SF/Fantasy Books

I'm releasing BestSFBooks today:

http://www.bestsfbooks.com

I was inspired by this HN post from a few weeks ago:

http://news.ycombinator.com/item?id=2978027

BestSFBooks ranks science fiction/fantasy books according to how many awards they've won or been nominated for. I included all the big ones (Hugo, Nebula, etc.) and also more obscure awards that I like such as SF Site Editor's choice. Once I had all the awards in there it was easy to start creating a New Book list based on award winning authors.

I could tell the app was working as soon as I saw the two book lists on the home page. They're excellent!

The stack is virtually identical to the stuff I used to build PickHealthInsurance. BestSFBooks is built with Rails 3.1.1 (HAML, Sass, CoffeeScript). It's hosted on Heroku and MongoHQ. I used the Twitter Bootstrap CSS toolkit, which I continue to find hugely innovative and useful. As always, data acquisition and cleanup was the hardest part.

I've wanted to build something like this for quite a while. As my time becomes more valuable, I'm becoming less tolerant of bad books. For a laugh, check out the "prototype" that I created ten years ago - http://www.gurge.com/amd/top100

Please send feedback!

OK, I have to admit at first I thought there was something wrong with the site because I didn't recognize enough books... but after reading some excerpts of books off the lists, I'm psyched -- turns out I just haven't been finding the good books for a long time, so now I can.

Suggests:

* show me excerpts on-page (if possible from amzn?)

* allow community +1/-1 on books and generate lists based on top-rated by site users

* commenting, facebook or disquss, on each book

(little issue: Facebook "like" button on main page and book pages doesn't seem to be working -- dunno if that's facebook's problem not yours)

This fairly reeks of awesome. If the ones I don't recognize are as great as the ones I do, I'd say you have a damn fine site there!

This is fantastic. I think you should follow natbro's lead on a few things - allow disquss commenting and show excerpts of the book.

Also I have a series to add: Harry Turtledove's "Darkness" series. It is like 5 books, if I remember correctly. I'm at work...

I've written a lot of data tools as part of Urbanspoon and subsequent startups. I like to collect publicly available data, clean it up, normalize it, and then release it in a more useful way.

Hot tips for crawling data:

  - Cache pages locally while you work on the indexing
  - Nokogiri is awesome
  - Don't be afraid to use regular expressions
  - Initially, put data into a spreadsheet (not the db).
    That way it can be checked in and diffed.

I also have a lot of subtle tricks for cleaning up messy data. For example, to see if two similar authors refer to the same person, I have a method that converts an author name to an author key. The key is just like the name, only it's been uppercased, apostrophes removed, etc. Plus weird stuff like this:

  # replace all vowels with the letter E
  s = s.gsub(/[AEIOUY]+/, "E")

It's little things like this hack that make a big difference in data quality.

(edit: formatting)