Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon DynamoDB – a Fast and Scalable NoSQL Database Service from AWS

allthingsdistributed.com

Post reply on HN
Print view View original

111–120 of 135 posts

Re: Amazon DynamoDB – a Fast and Scalable NoSQL Database Service from AWS

#111

Post by werner »

Yrlec wrote: post #83
peripitea wrote: post #73

Earlier quoted context omitted.

Can you explain your use case a bit more? I'm having a hard time imagining something that does ~430M DB writes/day but can't easily afford to pay $120 for those writes.

Remember that the throughput is per item, not per query. For instance we have an indexed query that returns ~1500 rows each time. Just doing that query a couple of times per second would create that kind of throughput requirement.

The amount of consumed read units by a query is not necessarily proportional to the # of items. It is equal to the cumulative size of processed items, rounded up to the next kilobyte increment. For example if you have a query returning 1,500 items of 64 bytes each, then you’ll consume 94 read units, not 1,500.

Re: Amazon DynamoDB – a Fast and Scalable NoSQL Database Service from AWS

#113

Post by werner »

nosequel wrote: post #89

https://img.skitch.com/20120118-etwh2gwu52isnn47fw2jpcir5p.p... You could buy a lot of Riak or Cassie for that.

5000 reads per sec of 64Kb items, would make you stream 2.5 Gbits/sec using consistent reads and 1Gbits/sec writes, moving close to 1.5TB each hour. At the end of the month you have read well over 800 TB and updates 160 TB... That is a substantial application you have in mind... :-)

Re: Amazon DynamoDB – a Fast and Scalable NoSQL Database Service from AWS

#114

Post by mattyb »

aphyr wrote: post #104
PanMan wrote: post #53

Earlier quoted context omitted.

For one, it responds quicker than Riak: Riak has (cold) response times of about 300ms, while this service claims single-digit ms response times. Also setting up Riak is not exactly trivial, and using this service outsources that hassle.

Riak has many shortcomings, but I wouldn't describe latency or installation as primary concerns. Our cold response times have a 99% bound of 8ms, and median of 5ms on commodity SSDs. Installation is handled by apt and is trivial to automate.

What are some of Riak's shortcomings?

Re: Amazon DynamoDB – a Fast and Scalable NoSQL Database Service from AWS

#115

Post by balakk »

vyrotek wrote: post #61
balakk wrote: post #32

Earlier quoted context omitted.

Can you elaborate how this is different and not comparable? Azure's table service offers the same automatic partition management, unlimited per-table scalability, composite keys, range queries, and availability guarantees. The linked paper goes into more details.

Besides the points I already complained about before... How about 200ms response times even when performing a query using the Row & Partition Keys. I'm not sure if by composite keys you were referring to something other than the RK & PK because those are the only indexes you get.

ATS response times within the Azure data center are pretty impressive in my experience.

Your partition keys can be composite, have a look here:

http://blogs.msdn.com/b/windowsazurestorage/archive/2010/11/...

I agree with your other pain points - in terms of not being able to get counts, secondary indices etc. However, you can easily simulate some of those - maintain your own summary tables, indices and so on. These ought to emerge as platform features pretty soon though. It's not perfect, but its feature set is close to Dynamo.

As for Mongo DB, I guess this service has been built from ground-up to provide the availability guarantees and automatic partition management features. I don't know if Mongo provides those. You could run Mongo yourself on Azure if you wanted to; there's even a supported solution done recently.

Re: Amazon DynamoDB – a Fast and Scalable NoSQL Database Service from AWS

#116

Post by balakk »

rojabuck wrote: post #60
mattcole wrote: post #37

Earlier quoted context omitted.

How is this vastly different to Azure Tables?

The cost per transaction, performance & ease. Reads per $0.01 = (50.60).60 = 180000 Writes per $0.01 = (10.60).60 = 36000 Assuming that you hit your usage is at 100% capacity then from a read prospective DynamoDB is half the price. Writes are much more expensive but many applications are heavily read oriented. DynamoDB claims single digit millisecond reads, azure tables does not (from my experience.) Azure tables hav…

Can you please explain your math? AFAIK Azure txns are not paid by the hour - they are a flat cost of $.01 per 10000 storage txns. If you do batched GETs and PUTs you make only 550 txns (55000/100 entites/batch).

http://www.windowsazure.com/en-us/home/tour/storage/

I agree that Dynamo's provisioned throughput capacity is a very useful feature though. Azure does not provide any such performance guarantee; the throughput limit is also a guideline as far as i know, not an absolute barrier.

Re: Amazon DynamoDB – a Fast and Scalable NoSQL Database Service from AWS

#117

Post by goosmurf »

Ataraxy wrote: post #95
peripitea wrote: post #73

Earlier quoted context omitted.

Can you explain your use case a bit more? I'm having a hard time imagining something that does ~430M DB writes/day but can't easily afford to pay $120 for those writes.

A small mobile marketing company jumping into the wild wild west of real time bidding. It would be used more so for logging impression requests to be used later for further analysis. Our bidder would need to be able to handle upwards of 5000 bid requests per second. Though these requests can be throttled down, naturally the more data we can collect the better. This also doesn't include the associated costs with query…

$3k/month = $36k/year

Consider this cost relative to the cost of a trustworthy ops person, plus the capex & opex of running your own reliable & scalable DB.

Re: Amazon DynamoDB – a Fast and Scalable NoSQL Database Service from AWS

#118

Post by vyrotek »

balakk wrote: post #115
vyrotek wrote: post #61

Earlier quoted context omitted.

Besides the points I already complained about before... How about 200ms response times even when performing a query using the Row & Partition Keys. I'm not sure if by composite keys you were referring to something other than the RK & PK because those are the only indexes you get.

ATS response times within the Azure data center are pretty impressive in my experience. Your partition keys can be composite, have a look here: http://blogs.msdn.com/b/windowsazurestorage/archive/2010/11/... I agree with your other pain points - in terms of not being able to get counts, secondary indices etc. However, you can easily simulate some of those - maintain your own summary tables, indices and so on. These o…

Hmm, I guess when I think about composite keys I think of ways to indicate a specific field/column as being part of the key. Data duplication along with string concatenation aren't really an elegant way to do it. If I remember right you also can't update the key values once the record has been saved. This is coming from a big SQL guy though :)

Re: Amazon DynamoDB – a Fast and Scalable NoSQL Database Service from AWS

#119

Post by spidaman »

ridruejo wrote: post #102
toddmorey wrote: post #92

Earlier quoted context omitted.

Open source really helps here. Amazon are innovative, but they are not the only place innovation is happening. In fact, here's a pretty good writeup (if a wee biased) on how the new offering compares to to the open source Cassandra project: http://www.datastax.com/dev/blog/amazon-dynamodb

Cassandra is a great project, as it is Hadoop, MySQL, etc. The issue I am raising is that it is not so much which project is better on a feature basis, but the fact that Amazon is able to offer it as a service, in a scalable way that no other vendor is able to do (with the exception of Google and, on a good day, Microsoft). Most other "traditional" cloud vendors, such as Rackspace, do not have anything remotely compa…

I also found it interesting that the storage media is specified and it's SSDs. Solid state will be hugely disruptive for hosted services, I've been hoping for an instance-by-the-hour service backed by SSDs and I'll surmise from this announcement that it won't be long before that shows up on the EC2 menu. Gimme :)
Post reply on HN