11–20 of 76 posts
Also, considering saving money on hardware costs is a key factor in Backblaze staying competitive, they must be saving money elsewhere and/or have other competitive advantages. Otherwise releasing this information would be akin to publishing a restaurant's 'secret sauce'.
What's the RAID config of these boxes? 45 drives, 4 controllers, and a 16TB volume limitation from ext4. Please share if you know! --edit-- I know it's RAID6 from the article. What I'm wondering is, how many drives in an array? How many arrays per box?
At the lowest level there are three RAID groups in each pod. Each RAID group is made of 15 drives configured in software RAID 6 with 2 parity drives. This means you can lose 2 dives and the data is entirely safe and intact. If 3 or more drives completely fail simultaneously (not just pop out of the RAID group or power down, but where that drive is lost forever, like it will never power up again) you will lose at leas…
Regardless though, super impressed by the work into rolling your own hardware, hope you guys continue to do well.
Does anyone know if Backblaze will ever support Linux? I've wanted to use their service for a while now, but their lack of Linux support has being a big turn down, and I don't think they have made any change in their statements regarding this 'issue'.
Explanation about the "GUI" comment above -> the Backblaze backup client was simultaneously written from the ground up compiling on Mac OS, Windows, and Linux. The same tree and the same source compiles on all three on EVERY SVN CHECKIN. There is one exception, which is the GUI is an extremely simple stand alone process entirely natively written to match the host OS. On Mac OS it is in Objective C in the beautiful Apple GUI layout editor, on Windows we use Visual Studio and C++ and Win32. The firm rule is these GUIs are ONLY allowed to edit one or two simple XML files, and all the real encryption, compression, transmission is done by other cross platform processes. On Linux we configure the XML file with "vi". :-) The X-Windows GUI has not even been started.
It's 135TB worth of drives, but with RAID don't you see a far less useable amount? Also, considering saving money on hardware costs is a key factor in Backblaze staying competitive, they must be saving money elsewhere and/or have other competitive advantages. Otherwise releasing this information would be akin to publishing a restaurant's 'secret sauce'.
On the other hand, I read this article and I read their last one, when it too shot to the top of HN. Geeks go nuts for hardware porn, and are a great audience to sell these kinds of services to. So the marketing benefits are probably quite substantial.
No ECC?
The ECC RAM absolutely does find and corrects problems (we see them in the logs). However, just to be absolutely clear we would not need ECC RAM -> Backblaze checksums EVERYTHING on an end-to-end basis (mostly we use SHA-1). This is so important I cannot stress this highly enough, each and every file and portion of file we store has our own checksum on the end, and we use this all over the place. For example, we pass over the data every week or so reading it, recalculating the checksums, and if a single bit has been thrown we heal it up either from our own copies of the data or ask the client to re-transmit that file or part of that file.
At the large amount of data we store, our checksums catch errors at EVERY level - RAM, hard drive, network transmission, everywhere. I suppose consumers just do not notice when a single bit in one of their JPEG photos has been flipped -> one pixel gets every so slightly more red or something. Only one photo changes out of their collection of thousands. But at our crazy numbers of files stored we see it (and fix it) daily.
Does anyone know if Backblaze will ever support Linux? I've wanted to use their service for a while now, but their lack of Linux support has being a big turn down, and I don't think they have made any change in their statements regarding this 'issue'.
We would like to, we just haven't had time to get it done yet. It runs internally, but is lacking an installer and a GUI, and we would need to prioritize and choose one or more Linux distributions to launch with. Ubuntu is an obvious choice (we focus more on desktop backup than on servers). But some people also ask for CentOS and a few others. It bums me out the Linux community has not solved binary compatibility any…
XML is a pain, but holding up release due to lack of a gui is sorta silly. Plus I imagine, like me, most multi-server users would be rolling out a standard xml config anyway without more than a simple string substitution on a per server basis.
In fact, the only GUI I have running is on my desktop.
Backblaze should really consider selling their pods - it would basically free money for them because whatever inventory they don't sell they can use for their service, and I'm sure there's lots of businesses that would pay good money for a cheaper alternative to storage servers from Dell or HP.
Also, most businesses buy from the Dells or the HPs because they don't have the in-house expertise to manage a more bare-bones box (or more likely just don't want to). The companies that have the need/capability to manage more raw storage could just take the plans and build them out anyways I would imagine.