Ask HN: Can Dynamo DB Handle Google?

#1

Post by iamAtom » Sun, Aug 23, 2020, 2:52 PM UTC

Hypothetical question lets say if google migrate their data store to Dynamo DB. Is AWS powerful enough to handle google scale ?

Re: Ask HN: Can Dynamo DB Handle Google?

#2

Post by hilalh » Sun, Aug 23, 2020, 7:46 PM UTC

To what Google product are you actually referring? Search, Gmail, Calendar? All of their products? How are you going to design the tables? Do you have any clues regarding what's going on with the current setup?

Re: Ask HN: Can Dynamo DB Handle Google?

#3

Post by iamAtom » Mon, Aug 24, 2020, 1:40 AM UTC

hilalh wrote: post #2

To what Google product are you actually referring? Search, Gmail, Calendar? All of their products? How are you going to design the tables? Do you have any clues regarding what's going on with the current setup?

Let's say just search engine.

Re: Ask HN: Can Dynamo DB Handle Google?

#4

Post by hilalh » Mon, Aug 24, 2020, 5:55 AM UTC

iamAtom wrote: post #3

hilalh wrote: post #2

To what Google product are you actually referring? Search, Gmail, Calendar? All of their products? How are you going to design the tables? Do you have any clues regarding what's going on with the current setup?

Let's say just search engine.

It is possible to build a crawler and store its data on DynamoDB, but you have to consider multiple issues when dealing with scale such as Google. E.g How much of the data is going to stay in DynamoDB and what is going to be offloaded to say S3 for example? What are you using DynamoDB for? Storing crawled info? Storing meta data of crawled sites? Scheduling your crawling? I would suggest you to look up crawling with Python / Node.js & DynamoDB, it might better answer your questions.