231–240 of 252 posts
Earlier quoted context omitted.
Except it is actually a job title at all sorts of places.
And if you google it, you'll see plenty of discussion around why that's controversial. https://www.google.com/search?q=devops+job+title&ie=utf-8&oe...
Earlier quoted context omitted.
Yes, a 1PB+ failure domain only made sense because Magic Pocket is very good at automatic cluster management and repair.
Surely it's the other way round? That is, the better your software can handle repair, the smaller your failure domain can be? A larger failure domain would only make sense if you wanted to minimize the compute required per unit stored?
Earlier quoted context omitted.
To be fair to Backblaze this level of storage density is really only possible with recent advances in disk technology (higher densities, SMR storage, etc). Also not everyone wants to be packing a petabyte into a box. At that level of density you need to invest a lot of effort in replication strategies, tooling, network fabric etc to handle failures with high levels of availability/durability.
Is dropbox using SMR? (I never used dropbox. Do they offer incremental backup? Otherwise I would have thought they would need to be able to do random writes, unless SMR is so cheap that it is economical to keep some dead data when a user uploads a new version of a file)
Earlier quoted context omitted.
The difference to get even higher is our system supports host-managed SMR disks that come in 10T and 14T sizes.
What interface and protocol do your drives have/use? Is it SATA, SAS or a custom interface/protocol?
Earlier quoted context omitted.
10 TB drives can be gotten off the shelf -- http://www.hgst.com/products/hard-drives/ultrastar-he10 -- and 14 TB drives are probably in the state of availability for large customers; it's not unusual for drive makers to make available drives coming down the pipe to certain customers.
True, it's just that they also tend to shout about upcoming drives and boast that they are previewing to select customers. Case in point, HGST marketing is in full swing on that He10 but good luck finding any stock in a mainstream retailer.
Earlier quoted context omitted.
Good questions, let me try to tackle them one by one. > The article makes a brief mention of Go causing issues with RAM usage. Was this due to large heap usage, or was it a problem of GC pressure/throughput/latency? If the former, what were some of the core problems that could not be further optimized in Go? The reasons for using rust were many, but memory was one of them. Primarily, for this particular project, the…
Did you guys use a custom allocator for rust? And if so how did it differ from jemalloc and how could it be compared to C++ allocators like tbb::scalable_allocator?
We also tweak jemalloc pretty heavily for our workload.
Earlier quoted context omitted.
Lazy answer but we'll blog about this in the next month. On a high level it's variable-sized blocks packed into 1GB extents, which are then aggregated into volumes and erasure coded across a set of disks on different machines/racks/rows/etc. We also replicate cross-country in addition to this. Live writes are written into non-erasure-coded volumes and encoded in the background. The volume metadata on the disks contai…
Are live writes replicated in real time, or are they locally staged (which is what Facebook does, I believe) Also, do you mimic the eventually consistent behaviour of AWS, or do you offer a stronger form of consistency?
It looks like the servers they make use of are purchasable, part of the Dell DSS series - the DSS7000: http://downloads.dell.com/manuals/common/dss%207000_dss%2075... 90x 3.5in disks, 2x compute nodes with 2x E5-2600v3 CPUs.
Unrelated to the content of the article--I've never seen the oft talked about "anti-adblocker interstitial" before. I was surprised to find that the website blocks viewing of the article entirely because I'm running Adblock. It's an interesting subject. I'll never click on or be persuaded by ads, so they're not really gaining anything by showing me ads. I'm essentially worthless traffic to them no matter how they cut…
The word "cloud" loses all meaning in this article. "The irony is that in fleeing the cloud, Dropbox is showing why the cloud is so powerful. It too is building infrastructure so that others don’t have to. It too is, well, a cloud company." Wait ... so using AWS is "cloud", having your own servers is "cloud" too. Everything is cloudy!
The second aspect is paying on an incremental basis for someone else to provide the pools developed in the first part. But that's more or less a trivial extension once you've got the first.