Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The trouble with Cassandra as an object storage metadata database

blog.min.io

Post reply on HN
Print view View original

51–60 of 82 posts

Re: The trouble with Cassandra as an object storage metadata database

#51

Post by kerblang »

kerblang wrote: post #15

The most basic and trivial way that you're going to get burned by cassandra is that you have to divide your primary key into two parts: partition key columns, and clustering key columns. Partition keys must be in every "where" clause; only clustering keys are optional. Okay, I'll just not bother with partition keys, right? Except partition keys determine which "partition" your data goes in. So if your only partition…

Replying to myself because I can't seem to find an edit button: If you're interested in Cassandra, the free online DataStax training is highly recommended. You'll learn all the gotchas ahead of time instead of after you went to production, and you really, really don't want the latter.

Re: The trouble with Cassandra as an object storage metadata database

#52

Post by anonymousDan »

nemothekid wrote: post #47
anonymousDan wrote: post #35

I don't quite get what they are referring to when they talk about 'metadata' here. Are they talking mostly about something internal to the database, something to do with the schema, or some kind of additional data used to enrich a particular object?

Minio is an s3 clone, so I assume you metadata they mean details about the object you uploaded. So if the actual data is a video file, the metadata would be the headers, filesize, what host/drive the file is stored on. Essentially when you ask for minio.io/my/video/file some service has to transform that to 192.168.0.1/customer/drive/ahe123.blob

Ahhh,I see thanks.

Re: The trouble with Cassandra as an object storage metadata database

#53

Post by mgkimsal »

jeffbee wrote: post #40
manigandham wrote: post #39

Earlier quoted context omitted.

The issue seems to be a misunderstanding of what Cassandra is. It's an (advanced/nested) key-value database, so of course you need to specify the key to do anything.

The misunderstanding is understandable, since the Apache Cassandra site fails to state what Cassandra is, even directly below the heading "What is Cassandra?"

Well... it is "the right choice when you need scalability and high availability without compromising performance". I'm not sure how much more description you need! :)

Re: The trouble with Cassandra as an object storage metadata database

#54

Post by llarsson »

acscott wrote: post #42
omginternets wrote: post #28

This is a restatement of the Cassandra docs paired with the usual misunderstanding of CAP. An AP system is not a choice of "I'll have availability and partition tolerance, please". It's the choice of availability given a partition. The whole point of CAP is that when partitions occur, there is a forced choice -- this is why it's incorrect to ask for a CA system. The Minio team are manifestly excellent engineers, but…

Do you mean AP is the choice of losing consistency given a partition?

That's how it is usually expressed, yes.

Re: The trouble with Cassandra as an object storage metadata database

#55

Post by jnwatson »

francoisdevlin wrote: post #5

A tangent - I love using minio in dev & test for their s3 simulators. Being able to throw away a bucket and start from scratch, and having everything self contained in my docker-compose command is a real blessing. Has anyone ever used min.io for production stuff? What are the pros & cons over vanilla s3?

My n=1 indirect experience as a consumer of someone else setting it up. The devops team had a lot of trouble keeping Minio up and running. Performance was not great.

The main pro and con is that it is self-hosted.

Re: The trouble with Cassandra as an object storage metadata database

#56

Post by omginternets »

acscott wrote: post #42
omginternets wrote: post #28

This is a restatement of the Cassandra docs paired with the usual misunderstanding of CAP. An AP system is not a choice of "I'll have availability and partition tolerance, please". It's the choice of availability given a partition. The whole point of CAP is that when partitions occur, there is a forced choice -- this is why it's incorrect to ask for a CA system. The Minio team are manifestly excellent engineers, but…

Do you mean AP is the choice of losing consistency given a partition?

Oh duh... You were pointing out a typo in my comment.

I'm leaving my original reply below because what the hell.

~~~~

Yes, exactly.

Think of it this way: you don't actually have any control over partitioning, therefore partitions are a given. So CAP is expressed as such: given a partition, choose between consistency and availability.

EDIT: I personally find PACELC [0] less confusing, and more nuanced than CAP, but they basically say the same thing.

[0] https://en.wikipedia.org/wiki/PACELC_theorem

Re: The trouble with Cassandra as an object storage metadata database

#59

Post by koolba »

jimbokun wrote: post #24
kerblang wrote: post #15

The most basic and trivial way that you're going to get burned by cassandra is that you have to divide your primary key into two parts: partition key columns, and clustering key columns. Partition keys must be in every "where" clause; only clustering keys are optional. Okay, I'll just not bother with partition keys, right? Except partition keys determine which "partition" your data goes in. So if your only partition…

Best of summary of the Cassandra data model I've seen: HashMap > So you can lookup data by the ParititionKey, then perform range queries on the ClusterKey to filter data belonging to that partition. If your access patterns look like that, Cassandra could be a great fit. For anything else, you probably want to consider a different data store.

Ha! That's pretty succinct.

Though HashMap should be just "Map" and the value of the top level map is not a set, it's another map:

Map>

Re: The trouble with Cassandra as an object storage metadata database

#60

Post by noncoml »

kerblang wrote: post #15

The most basic and trivial way that you're going to get burned by cassandra is that you have to divide your primary key into two parts: partition key columns, and clustering key columns. Partition keys must be in every "where" clause; only clustering keys are optional. Okay, I'll just not bother with partition keys, right? Except partition keys determine which "partition" your data goes in. So if your only partition…

> Your partitions need to be Is that a typo? 300MB sounds ridiculously low..
Post reply on HN