Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Azure Cosmos DB: Microsoft's Cloud-Born Globally Distributed Database

muratbuffalo.blogspot.com

Post reply on HN
Print view View original

91–100 of 137 posts

Re: Azure Cosmos DB: Microsoft's Cloud-Born Globally Distributed Database

#91

Post by jbreiding »

manigandham wrote: post #72
equasar wrote: post #39

Earlier quoted context omitted.

I second this, Azure Table Storage is an amazing product, but would be a more powerful one if they enable indexing on all columns and also any automatic backup options, which are only available on CosmosDB Table, which makes me thing that Azure wants to push me to use CosmosDB if I want those features, since they don't have a roadmap for this for the Azure Table Storage.

Azure Table storage is a key/value store similar to Hbase, Cassandra, GCP Bigtable, AWS DynamoDB and others. It's a very stable and scalable design which is why everyone has some version of it. There's no way to add indexes to this data model, it's all done by just writing your data in another table using the key you want and the value pointing to the original table's row. Perhaps Azure could offer secondary indexes…

Having the azure table storage be indexed into azure search with an azure function as the indexer delivers a pretty powerful alternative that isn't exactly batteries included, but close.

Re: Azure Cosmos DB: Microsoft's Cloud-Born Globally Distributed Database

#92

Post by jbreiding »

chupa-chups wrote: post #37

We used it also. Ran into the issue that collection (?) names are limited to 64 chars(1) (which wasn't documented anywhere), triggering obscure errors while everything was fine from the Azure Frontend. Also even simple queries using an index (and a uniform document structure) cost a comparatively huge amount of RUs compared to a competitor. Icing on the cake was the non-support for paging at that time, and a total fu…

Any reason you wouldn't use resource groups and db/collections for segregation instead of encoding into the collection name? I believe all of that hirearchy ends up in the resource URI if you really need it encoded there.

Re: Azure Cosmos DB: Microsoft's Cloud-Born Globally Distributed Database

#93

Post by rnehme »

kuzehanka wrote: post #3

After trying CosmosDB I put it down pretty quickly. I can't say I'd recommend it to anyone. Shockingly poor perf. Is CosmosDB really that bad or did we have it misconfigured? We're not sure, the docs didn't help us understand. Random failures when connecting. Random errors (or worse, no errors but unexpected results) when querying. Largely undocumented. Painful interop with non-azure proprietary offerings. Almost zer…

Hi, I am from Azure Cosmos DB Engineering Team.

Please feel free to send us the details of your workload that suffered from poor performance to AskCosmosDB@microsoft.com (it is a direct way to reach the engineering team and it does not cost anything), and we can help determine why and provide with the recommendations.

For best practices and performance optimizations, the following links will help:

https://docs.microsoft.com/en-us/azure/cosmos-db/sql-api-que...

https://docs.microsoft.com/en-us/azure/cosmos-db/performance...

https://docs.microsoft.com/en-us/azure/cosmos-db/performance...

https://docs.microsoft.com/en-us/azure/cosmos-db/performance...

Also, the following conceptual docs will help:

https://docs.microsoft.com/en-us/azure/cosmos-db/partitionin...

https://docs.microsoft.com/en-us/azure/cosmos-db/partition-d...

https://docs.microsoft.com/en-us/azure/cosmos-db/request-uni...

We are doing a full refresh of our docs (rewriting a lot of the older content and adding new technical content). We welcome feedback on how we can make the docs better and invite everyone to contribute via public PR/Github. If there are any topics that are missing in our docs or are not easily discoverable, please let us know. We will make sure to address it.

We are committed to building a great community. As a relatively young database, we have ton of work ahead, but we are super committed to making sure we make developers successful.

Re: Azure Cosmos DB: Microsoft's Cloud-Born Globally Distributed Database

#94

Post by rnehme »

llcoolv wrote: post #46
maltalex wrote: post #24

Earlier quoted context omitted.

To the brave, I recommend trying to make a copy of a CosmosDb collection. The only solution that doesn't require spinning up a VM, reading all the data and writing it back again is the Azure Data Factory [0]. Which according to Azure's own benchmarks [1] manages to copy only 2MBps between two CosmosDb instances each with 100,000 RUs ("Resource Units"). That's more than half a day for a small 100GB collection. And in…

I am sorry to have to argue, but the multi-region writes are calculated N + 1, where N is the number of regions. So it would actually set you back $17520 [0]. And yes, I know that even their own documentation and calculator conflict on this, but they do charge the higher number. 0. https://docs.microsoft.com/bg-bg/azure/cosmos-db/optimize-co...

Hi, I am from Azure Cosmos DB Engineering Team.

We are in the process of updating the Azure pricing calculator to provide various options (e.g., # of regions, single vs. multi-master, etc.). The updates to the calculator are coming shortly.

In the meantime, please feel free to ping us at AskCosmosDB@microsoft.com, if you need any help with cost estimates or have any questions.

Re: Azure Cosmos DB: Microsoft's Cloud-Born Globally Distributed Database

#95

Post by rnehme »

m_fayer wrote: post #15
kuzehanka wrote: post #3

After trying CosmosDB I put it down pretty quickly. I can't say I'd recommend it to anyone. Shockingly poor perf. Is CosmosDB really that bad or did we have it misconfigured? We're not sure, the docs didn't help us understand. Random failures when connecting. Random errors (or worse, no errors but unexpected results) when querying. Largely undocumented. Painful interop with non-azure proprietary offerings. Almost zer…

Same experience here. Also found it to be hideously expensive while setting all dials to minimum for experimentation and exploration. There may be a way to configure it for an affordable trial run, but it didn't motivate me to stay and figure it out.

Hi, I am from Azure Cosmos DB Engineering Team.

For development (before going into production) with Azure Cosmos DB, we suggest a number of options (also documented here: https://docs.microsoft.com/en-us/azure/cosmos-db/optimize-de...):

[1] Cosmos DB emulator (Cosmos DB Local version) - https://docs.microsoft.com/en-us/azure/cosmos-db/local-emula.... The Azure Cosmos DB Emulator provides a local environment that emulates the Azure Cosmos DB service for development purposes. Using the Azure Cosmos DB Emulator, you can develop and test your application locally, without creating an Azure subscription or incurring any costs. When you're satisfied with how your application is working in the Azure Cosmos DB Emulator, you can switch to using an Azure Cosmos DB account in the cloud. Support for Cosmos DB emulator on Linux is coming.

[2] We provide Try Cosmos DB for free experience - https://azure.microsoft.com/try/cosmosdb/. There is Try Azure Cosmos DB for free experience which gives you a time-limited full-service experience with Azure Cosmos DB. It is designed for trying out Cosmos DB, doing a tutorial, a demo, doing a quick start, a lab without requiring an Azure account or a credit card.

[3] We provide a free tier (Cosmos DB is a part of free account) - https://azure.microsoft.com/free. In Azure, we support a free tier with 12 month access to all popular services in Azure. As a part of free tier specifically for Cosmos DB you get 5GB in storage and 400RUs.

Thanks so much!

Re: Azure Cosmos DB: Microsoft's Cloud-Born Globally Distributed Database

#96

Post by rnehme »

tasubotadas wrote: post #30
kuzehanka wrote: post #3

After trying CosmosDB I put it down pretty quickly. I can't say I'd recommend it to anyone. Shockingly poor perf. Is CosmosDB really that bad or did we have it misconfigured? We're not sure, the docs didn't help us understand. Random failures when connecting. Random errors (or worse, no errors but unexpected results) when querying. Largely undocumented. Painful interop with non-azure proprietary offerings. Almost zer…

You also forgot to mention that it is crazy expensive if you have lots of collections

Hi, I am from Azure Cosmos DB Engineering Team.

@llcoolv has responded and the response is correct.

For scenarios where you create a lot of collections/tables, we recommend to provision throughput on a database and share the provisioned throughput among all the containers – to save on costs. You can change the throughput at database exactly like how you will change the throughput at container level. It is the same API, it is supported via CLI, SDK, Rest API and Portal as well.

The entry point for database level throughout is 400RUs $24 per month and you can add any number of collections/tables that can share this throughput. You can scale up and down in the increments of 100RUs (or $6/month).

Please see the following links for details:

[1] https://docs.microsoft.com/en-us/azure/cosmos-db/set-through...

[2] https://docs.microsoft.com/en-us/azure/cosmos-db/how-to-prov...

[3] https://docs.microsoft.com/en-us/azure/cosmos-db/how-to-prov...

Please see the sample here for moving data:

[4] https://github.com/Azure/azure-documentdb-dotnet/tree/master...

Thanks!

Re: Azure Cosmos DB: Microsoft's Cloud-Born Globally Distributed Database

#97

Post by rnehme »

dtrailin wrote: post #85
ignoramous wrote: post #33

This post is interesting but one or two MS employees did reveal some key details about CosmosDB in the original announcement thread: https://news.ycombinator.com/item?id=14308814 (edited) > There are many significant differences in capabilities, and design approaches between other systems (CockroachDB and Spanner) and Cosmos DB. At a very high level differences are at two levels - the design of the database engine an…

The TLA+ specs where eventually published [0], although I wonder how up to date they are to current state of the DB since they haven't been updated for a while. [0] https://github.com/Azure/azure-cosmos-tla

Hi, I am from Azure Cosmos DB Engineering Team.

The TLA+ specs are fairly up to date.

Thanks.

Re: Azure Cosmos DB: Microsoft's Cloud-Born Globally Distributed Database

#98

Post by dharmashuklaMS »

suff wrote: post #11

FTA: "say datacenter automatic failover component, in one part of the territory. Getting this component right may take months of your time. But it is OK. You are building a new street in one of the suburbs, and this adds up to the big picture." Wait, what?! Driving metaphor aside... 'MONTHS of your time' to implement failover? People, this is WHY the cloud was invented, so we didn't have to spend months re-inventing.…

Hi, I am from Azure Cosmos DB Engineering Team.

The article is referring to the implementation details of automatic failover mechanisms inside of Cosmos DB service.

For the customers, automatic failover is available as a turnkey capability. Customers do not need to spend any time to implement automatic failover.

Thanks.

Re: Azure Cosmos DB: Microsoft's Cloud-Born Globally Distributed Database

#99

Post by dharmashuklaMS »

he0001 wrote: post #2

Do they guarantee that if a write is done one region, when completed upon return, is available when read in another region?

Hi, I am from the Azure Cosmos DB engineering team.

All five consistency levels (strong, bounded staleness, session, consistent prefix, and eventual) are available for your Cosmos account, regardless of the number of regions it is associated with.

To answer your specific question, yes, if you choose strong consistency, you will read the latest write in any of the regions associated with your Cosmos database.

If you choose bounded staleness consistency, you can configure the number of versions (k) or time (t) up to which you are willing to tolerate the staleness for your reads in return for the write to get ack’d immediately (after it is quorum committed in the local region).

The documents below describe the consistency levels and tradeoffs: https://docs.microsoft.com/en-us/azure/cosmos-db/consistency... https://docs.microsoft.com/en-us/azure/cosmos-db/consistency...

Thanks!

Re: Azure Cosmos DB: Microsoft's Cloud-Born Globally Distributed Database

#100

Post by jjirsa »

Hey CosmosDB team in the comments:

Building distributed databases at scale is hard af, few people know what it’s like to run hundreds of thousands of databases. Don’t be discouraged by random negative posts, remember you’re doing something most of the world can’t do.

Post reply on HN