Goodreads offloads DynamoDB tables to S3 and queries them with Athena
1–10 of 110 posts
Re: Goodreads offloads DynamoDB tables to S3 and queries them with Athena
#2First lesson - it's taken Goodreads so long and so many users to actually start moving to microservices.
Re: Goodreads offloads DynamoDB tables to S3 and queries them with Athena
#3Weren't Goodreads bought by Amazon a couple of years ago? If so, they might've been pushed to do the move (to microservices, s3, etc) to comply with corporate guidelines/policy not because there wasn't a better/more efficient way to scale.
Re: Goodreads offloads DynamoDB tables to S3 and queries them with Athena
#4Hey! I’m the author of this post. I’m pretty chuffed to see this here. Happy to answer any questions.
Re: Goodreads offloads DynamoDB tables to S3 and queries them with Athena
#5Weren't Goodreads bought by Amazon a couple of years ago? If so, they might've been pushed to do the move (to microservices, s3, etc) to comply with corporate guidelines/policy not because there wasn't a better/more efficient way to scale.
Yeah 2013. You get goodreads info inside your profile section of the kindle apps.
Re: Goodreads offloads DynamoDB tables to S3 and queries them with Athena
#6We did something similar at a previous position I had, except we set up an amazon lambda function that was triggered on every insert or update to a DynamoDB table. The lambda function flattened the updated record and inserted it into our redshift cluster, which gave us a real-time ETL pipeline for our DynamoDB data. That allowed us to report on our DynamoDB data just like our relational data.
Re: Goodreads offloads DynamoDB tables to S3 and queries them with Athena
#7Why is the author’s HN comment dead?
Joe feeneys HN account is indeed iamawalrus; he wasn’t impersonating anyone. Further his comment just said:
“Hey! I’m the author of this post. I’m pretty chuffed to see this here. Happy to answer any questions.”
I want to hear what he has to say.
Re: Goodreads offloads DynamoDB tables to S3 and queries them with Athena
#8I use Goodreads as my main book tracker and reviewer. My impression is that Amazon bought it for whatever reason, and then ignored it. It seems like there's so much more potential to a social community around books than what Goodreads offers. The UI also never fails to disappoint. One of the core user features, searching for books, has all sorts of weird quirks. I'll use a very specific query that should guarantee a hit, and nothing comes up. Or, the book that I want will come up for a second, and right as I'm going to select it, the results completely change and the book I want has vanished.
Re: Goodreads offloads DynamoDB tables to S3 and queries them with Athena
#9Why is the author’s HN comment dead? Joe feeneys HN account is indeed iamawalrus; he wasn’t impersonating anyone. Further his comment just said: “Hey! I’m the author of this post. I’m pretty chuffed to see this here. Happy to answer any questions.” I want to hear what he has to say.
Unsure, but I vouched for it so it's undead now.
Re: Goodreads offloads DynamoDB tables to S3 and queries them with Athena
#10What I don't get about Athena is what happens after you've put the data in Athena? Fine, you've got SQL and tabular data, but the type of BI I've had to do usually has a graph or some other visual representation at the end rather than a table. There's only so much data you can import into Excel from a CSV that Athena produces. Usually I find periscope/cluvio to be much better tools for this and then you need to go to redshift. So why bother with all the data pipeline to Athena? Does anyone use this as well as Periscope/Cluvio and can chip in?