Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Seagate just reinvented the disk interface using Ethernet

speakingofclouds.com

Post reply on HN
Print view View original

41–50 of 122 posts

Re: Seagate just reinvented the disk interface using Ethernet

#41

Post by asharp »

noonespecial wrote: post #11

I really like the "its just a server that takes a 4k key and stores and retrieves a 1M value" approach. I'm not so keen on the physical drive "repurposing" the standard pinout of existing hardware unless they are prepared to gracefully fall back to the old block device standard if it gets plugged into a "muggle" device. This has real promise so long as it stays as radically open as they are claiming it will be. When…

Key value stores are useful, and they are especially useful in this form factor. On the other hand, you now have a very large black box that you have to somehow navigate in order to create a workable system. Given that this is likely an arm core running linux on the inside, I would have considered a slightly more open approach to be 'Here's a working KV store using backing db X and here's how to reflash it if it doesn't quite work for you'.

Re: Seagate just reinvented the disk interface using Ethernet

#42

Post by asharp »

mullr wrote: post #2

The technical details: https://developers.seagate.com/display/KV/Kinetic+Open+Stora... The important, actual TLDR: "Kinetic Open Storage is a drive architecture in which the drive is a key/value server with Ethernet connectivity."

Also look at https://github.com/Seagate/Kinetic-Preview

Re: Seagate just reinvented the disk interface using Ethernet

#43

Post by vidarh »

If this means 10gbps ethernet switches finally comes down in price, awesome...

Otherwise this will be hampered by the fact that the 6Gbps of SATA III is already too slow to take maximum advantage of many SSD devices (hence OCZ experiments with effectively extending PCIe over cables to the devices.

Re: Seagate just reinvented the disk interface using Ethernet

#44

Post by rythie »

stingraycharles wrote: post #32
rythie wrote: post #26

I don't understand why this is branded as a Ethernet protocol when it's a IP protocol

It communicates over ethernet, which is what matters. So it doesn't matter that much, and people understand what it means -- in the same way it doesn't matter you just said "IP protocol", which expands to "internet protocol protocol".

But IP runs over more than just ethernet or is it limited only run on ethernet?

Re: Seagate just reinvented the disk interface using Ethernet

#45

Post by buyx »

konstruktor wrote: post #6

"Über alles" will be primarily associated with Hitler's anthem of the Third Reich by most native speakers of German who know some history. Not a good choice for a title.

Since the article was written in English, for an English speaking audience, who consider generally "über alles" to not be particularly offensive anymore (it has come to mean something like "all conquering"). I don't think it was a poor choice of title at all, especially since it sums up the tone of his article.

It's quite unfortunate that the far-right in Germany has since appropriated the phrase, but it's been appropriated differently by English-speakers. Censoring people based on a usage that is foreign to them is a little harsh.

Re: Seagate just reinvented the disk interface using Ethernet

#46

Post by vidarh »

rurounijones wrote: post #25

As a counterpoint: A slightly less gushing article with some good comments (yes, even on El Reg) http://www.theregister.co.uk/2013/10/22/seagate_letting_apps... Comments along the lines of "Backups? Snapshots? RAID? How they handling this then?"

How are the handling this today? You can treat an existing HD as a key-value store where the keys is the location on disk and the value is a sector of binary data. Conceptually there's no difference.

The answer is: If you need those capabilities to offer up a traditional file system, you do as you do today: you layer it on top.

But many systems don't, because they already re-implement reliability measures on top of hard drives, as we want systems that are reliably available in the case of server failure too.

E.g. consider something like Sheepdog: https://github.com/sheepdog/sheepdog Sheepdog is a cluster block device solution with automatic rebalancing and snapshots. It implements this on top of normal filesystems by storing "objects" on any of a number of servers, and uses that abstraction to provide all the services. Currently sheepdog requires the sheep daemon to run on a set of servers that can mount a file system on the disks each server is meant to use. With this system, you could possibly dispense with the filesystem, and have the sheep daemons talk directly to a number of disks that are not directly attached.

For sheepdog RAID is not really recommended, as sheepdog implements redundancy itself (and you can specify the desired number of copies of each "block device" ), and it also provides snapshots, copy on write, extensive caching and support incremental snapshot based backups of the entire cluster in one go.

So in other words, there are applications that can make very good use of this type of arrangement without any support for raid etc. at the disk level. And for applications that can't, a key value store can trivially emulate a block device - after all sheepdog emulates a block device on top of object storage on top of block devices...

You could also potentially reduce the amount of rebalancing needed in the case of failures, by having sheep daemons take over the disks of servers that die if the disks are still online and reachable.

The biggest challenge is going to be networking costs - as I mentioned elsewhere, SSDs are already hampered by 6Gbps in SATA III, and 10GE switches are ludicrously expensive still.

Re: Seagate just reinvented the disk interface using Ethernet

#47

Post by dmpk2k »

perlpimp wrote: post #9

Radical simplification and IMO this is great. Remains to be seen how this will fare in comparison with RAID. I'd wager that google would be very interested, if they already not doing something like that in their data centers. Nerdy me likes idea of POE hub and bunch of drives doing their own thing. Also pretty good time to start writing stuff to support this into Linux kernel and developing support apps. my 2c

I'd wager that google would be very interested, if they already not doing something like that in their data centers.

I wonder about that.

It's usually a lot cheaper to move computation to data, rather than data to computation. The model that Seagate is presenting here strikes me as wrong, because it assumes very fat pipes (or specialized topologies) for any non-trivial app. At the scale Google operates at, I just don't see this happening.

That, and I have a healthy distrust of networks. Instead of having a box with an OSS OS and dumb drives with small(er) closed firmware blobs, now you have the OS, all the network devices and their closed firmware blobs, and drives with large(r) closed firmware blobs, just to access your data. A lot more can go wrong. A lot more byzantine things can go wrong. Drives are dodgy lying sacks of fecal matter as is; this looks like it'll make things much worse.

The model Seagate presents could be useful for data that is rarely accessed, but I'm not really sold on that either.

Re: Seagate just reinvented the disk interface using Ethernet

#48

Post by arnehormann »

rythie wrote: post #44
stingraycharles wrote: post #32

Earlier quoted context omitted.

It communicates over ethernet, which is what matters. So it doesn't matter that much, and people understand what it means -- in the same way it doesn't matter you just said "IP protocol", which expands to "internet protocol protocol".

But IP runs over more than just ethernet or is it limited only run on ethernet?

IP is not limited to Ethernet. See http://en.wikipedia.org/wiki/Osi_layer Ethernet is Layer 1, IP is Layer 3.

Re: Seagate just reinvented the disk interface using Ethernet

#49

Post by sneak »

This seems like a reinvention of Coraid's ATAoE, which has the added benefit of already being in the mainline kernel, good server/target support (vblade), hardware products shipping now, a lack of IP/TCP overhead, and a dead-simple protocol.

http://aoetools.sourceforge.net/

Re: Seagate just reinvented the disk interface using Ethernet

#50

Post by chris_wot »

arnehormann wrote: post #48
rythie wrote: post #44

Earlier quoted context omitted.

But IP runs over more than just ethernet or is it limited only run on ethernet?

IP is not limited to Ethernet. See http://en.wikipedia.org/wiki/Osi_layer Ethernet is Layer 1, IP is Layer 3.

IEEE 802.3 exists in layer 2 also. Each frame holds source and destination MAC addresses (and a lot more besides).
Post reply on HN