Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Torus: A distributed storage system by CoreOS

coreos.com

Post reply on HN
Print view View original

161–170 of 187 posts

Re: Torus: A distributed storage system by CoreOS

#161

Post by jsmthrowaway »

mrmondo wrote: post #122

Watch out, next we'll be seeing CoreOS writing their own encryption protocols. As someone that's designed and built storage clusters from the ground up - take it from me: storage is not as easy as it seems if you care about your data and performance at scale. The number of people I've seen using CoreOS and then moving away from it is quite alarming, I feel like CoreOS will become the Ubuntu of the container world.

They're all-in on Kubernetes, which is quite apparent given (a) who's funding them and (b) product direction, including this one. They don't have the resources or backing to go after Docker; notice that changed? Docker has something like 15x the funding and is pulling away in a lot of ways, so CoreOS hitched on the Kubernetes wagon.

Better bet for destiny: Google is letting them build stuff off the books by just funding them, and at some point they'll get quietly bought with all their stuff added to Kubernetes, and that'll be that.

Re: Torus: A distributed storage system by CoreOS

#162

Post by notacoward »

zxcvcxz wrote: post #76

It's threads like these where you start to wonder just how many marketing teams are arguing with each other in the comments. I take the FUD with a grain of salt: there is a lot of financial incentive to create FUD, where there is financial incentive and little to no regulations a market will naturally arise. It's going to get worse as the bubble pops and companies become more and more desperate.

I don't think it's marketing teams. There are at least a half dozen CoreOS employees/contributors commenting here, plus a couple of known allies. On the "other side" there's Bryan, me, and one person who seems to be an ex-employee. AFAIK none of them are in marketing, or even in coordination with marketing. Certainly, I often get flak from my company's official mouthpieces for saying things that conflict with their preferred talking points. They wish they could control what I say.

Mostly I think this is a matter of people naturally standing up for their friends and colleagues, which is a wonderful thing, vs. people who have specific concerns about The Right Way to do either technical or non-technical things. No coordination or collusion is necessary. You can see exactly the same thing happen for every company or project that's discussed here, or on Twitter, or wherever. Try criticizing a YC portfolio company some time. Not all of the people arguing with you will admit their affiliations, and of course the downvotes are all anonymous anyway. Just remember, the more skin someone has in the game, the more tempted they'll be to cross that vague line into astroturf. All the rest of us can do is admit our affiliations and biases, and hope that people will get their heads out of the ad hominem gutter enough to reach rational conclusions about the facts being presented.

Re: Torus: A distributed storage system by CoreOS

#163

Post by jedisfullofit »

jsmthrowaway wrote: post #102

I love that I know who this is at CoreOS by the choice of username. Pick better throwaways if you're going to blow up a thread like this. (ideal0227 upthread is also a CoreOS employee and one of the etcd developers.) On topic, yes. It is quite trivial to lose data with etcd and pretty much everybody I know who runs it has experienced problems. Try backing it up and restoring. etcd is compsci great yet operationally b…

While we're talking about who is who -- the parent poster is Jed Smith; he's a sysadmin and disgruntled former CoreOS employee. I'd take what he says with more than a grain of salt.

Re: Torus: A distributed storage system by CoreOS

#164

Post by [deleted] »

jedisfullofit wrote: post #163
jsmthrowaway wrote: post #102

I love that I know who this is at CoreOS by the choice of username. Pick better throwaways if you're going to blow up a thread like this. (ideal0227 upthread is also a CoreOS employee and one of the etcd developers.) On topic, yes. It is quite trivial to lose data with etcd and pretty much everybody I know who runs it has experienced problems. Try backing it up and restoring. etcd is compsci great yet operationally b…

While we're talking about who is who -- the parent poster is Jed Smith; he's a sysadmin and disgruntled former CoreOS employee. I'd take what he says with more than a grain of salt.

[deleted]

Re: Torus: A distributed storage system by CoreOS

#165

Post by jbooth »

betawaffle wrote: post #159
jbooth wrote: post #158

Earlier quoted context omitted.

Yeah, but they have to read several data/parity blocks, and then rewrite all parity blocks plus one data block, for any write to a given block. This creates big difficulties for both consistency and performance, and fixes for consistency make the performance worse (and vice versa). Google's filesystem could use reed-solomon because they're append-only, making consistency a non-issue and performance can be fixed by bu…

Torus is append-only too. We also plan to support something more like what Facebook's paper describes, where they have extra parity (xor) to support more efficient local repair.

How? I thought you're exporting a block device, not a filesystem? You can't append to a block device, and certainly every filesystem out there expects block devices to be random-writable, right?

Re: Torus: A distributed storage system by CoreOS

#166

Post by betawaffle »

jbooth wrote: post #165
betawaffle wrote: post #159

Earlier quoted context omitted.

Torus is append-only too. We also plan to support something more like what Facebook's paper describes, where they have extra parity (xor) to support more efficient local repair.

How? I thought you're exporting a block device, not a filesystem? You can't append to a block device, and certainly every filesystem out there expects block devices to be random-writable, right?

The "interface" we're exporting is very different from the underlying storage. The block device interface we currently provide supports random writes just fine, but the underlying storage we use (which involves memory-mapped files) is append-only. Once written, blocks are only ever GC'd, not modified.

Re: Torus: A distributed storage system by CoreOS

#167

Post by jbooth »

betawaffle wrote: post #166
jbooth wrote: post #165

Earlier quoted context omitted.

How? I thought you're exporting a block device, not a filesystem? You can't append to a block device, and certainly every filesystem out there expects block devices to be random-writable, right?

The "interface" we're exporting is very different from the underlying storage. The block device interface we currently provide supports random writes just fine, but the underlying storage we use (which involves memory-mapped files) is append-only. Once written, blocks are only ever GC'd, not modified.

Ok, so that plus a little MVCC can make you consistent, but you've still got the read-many-to-write-one thing from the perspective of your block device interface, right? And block devices, if I'm remembering right, don't leave you any room to buffer pending writes.

Re: Torus: A distributed storage system by CoreOS

#168

Post by stevelandiss »

betawaffle wrote: post #166
jbooth wrote: post #165

Earlier quoted context omitted.

How? I thought you're exporting a block device, not a filesystem? You can't append to a block device, and certainly every filesystem out there expects block devices to be random-writable, right?

The "interface" we're exporting is very different from the underlying storage. The block device interface we currently provide supports random writes just fine, but the underlying storage we use (which involves memory-mapped files) is append-only. Once written, blocks are only ever GC'd, not modified.

So if I were to run a database on this, wit a lot of overwrites, the storage would grow infinitely?

Secondly, this implies you are remapping the LBA (offsets) all the time, perhaps taking what would be sequential access and turning it into random? That sounds pretty painful.

Re: Torus: A distributed storage system by CoreOS

#169

Post by dang »

jsmthrowaway wrote: post #102

I love that I know who this is at CoreOS by the choice of username. Pick better throwaways if you're going to blow up a thread like this. (ideal0227 upthread is also a CoreOS employee and one of the etcd developers.) On topic, yes. It is quite trivial to lose data with etcd and pretty much everybody I know who runs it has experienced problems. Try backing it up and restoring. etcd is compsci great yet operationally b…

> I love that I know who this is at CoreOS by the choice of username

You can't comment like this here. It's a breach of civility and took the conversation in a needlessly personal direction. Please don't do this again.

We detached this subthread from https://news.ycombinator.com/item?id=11818589 and marked it off-topic.

Re: Torus: A distributed storage system by CoreOS

#170

Post by dang »

jedisfullofit wrote: post #163
jsmthrowaway wrote: post #102

I love that I know who this is at CoreOS by the choice of username. Pick better throwaways if you're going to blow up a thread like this. (ideal0227 upthread is also a CoreOS employee and one of the etcd developers.) On topic, yes. It is quite trivial to lose data with etcd and pretty much everybody I know who runs it has experienced problems. Try backing it up and restoring. etcd is compsci great yet operationally b…

While we're talking about who is who -- the parent poster is Jed Smith; he's a sysadmin and disgruntled former CoreOS employee. I'd take what he says with more than a grain of salt.

Personal attacks are not allowed on HN, nor single-purpose accounts. Novelty accounts for personal attacks are right out, so we've banned this one.
Post reply on HN