Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

135TB for $7,384 - Backblaze Pod 2.0

zdnet.com

Post reply on HN
Print view View original

31–40 of 76 posts

Re: 135TB for $7,384 - Backblaze Pod 2.0

#31

Post by brianwski »

rdoherty wrote: post #11

It's 135TB worth of drives, but with RAID don't you see a far less useable amount? Also, considering saving money on hardware costs is a key factor in Backblaze staying competitive, they must be saving money elsewhere and/or have other competitive advantages. Otherwise releasing this information would be akin to publishing a restaurant's 'secret sauce'.

Anybody who builds their own pod is welcome to use RAID or not, and which RAID you choose will affect your final numbers. At Backblaze, we configure it as RAID 6 groups, each group is 15 drives which includes 2 parity drives. So you are down to 13/15 = 86.67 percent of the raw unformatted space BEFORE the overhead of ext4 which adds a little tiny bit extra. So after formatting in our datacenter we are left with about 116 TBytes of storage space for customer files. On the other hand, we use lossless compression on the customer's files before transmitting to the pod, so it can sometimes appear like we are fitting more than 116 TBytes of customer data on a pod, if that makes sense.

Re: 135TB for $7,384 - Backblaze Pod 2.0

#33

Post by brianwski »

cakeface wrote: post #29
shapoopy wrote: post #28

Earlier quoted context omitted.

I don't know how many other folks feel this way, but I would kill for a GUI-less version of your client. I'd love to configure XML files, or some (any!) analog. I've been looking for an off-site backup solution for a (nerdy and technically competent) home user for years, and yours is the only one that I could afford (poor recent BA here). Please?! I would pay $10 a month (probably more, really) for that. Even if you…

It sounds like those XML files are already there when you install the windows or mac version. Why don't you just open them up and edit them by hand?

Honestly, the XML files are pretty simple. The one main one the GUI writes out is called bzinfo.xml and is found on any Mac system at /Library/Backblaze/bzdata/bzinfo.xml and on any Windows Vista or later system at C:\ProgramData\Backblaze\bzdata\bzinfo.xml

Backblaze is designed to be used with absolutely no configuration (for many users they have no idea where their Outlook.pst file is and we don't think they should have to know), and the only way we could figure out how to make this work was to backup EVERYTHING on your system unless you explicitly exclude it. So bzinfo.xml is basically a flat list of excluded folders you do not want backed up. There is also a throttle in there if you don't want Backblaze to utterly destroy your network uplink, and a few other small settings. It's pretty straightforward.

With that said, we really pride ourselves on easy to use software, so it goes against everything in our DNA to release software with NO GUI at all, but maybe we'll give that a serious thought. If you are using Linux, you probably aren't the average Mom & Pop user. :-)

Re: 135TB for $7,384 - Backblaze Pod 2.0

#34

Post by brianwski »

db48x wrote: post #30
brianwski wrote: post #18

Earlier quoted context omitted.

Yes, the RAM is ECC. Here is a link: http://www.crucial.com/store/partspecs.aspx?imodule=CT25672B... The ECC RAM absolutely does find and corrects problems (we see them in the logs). However, just to be absolutely clear we would not need ECC RAM -> Backblaze checksums EVERYTHING on an end-to-end basis (mostly we use SHA-1). This is so important I cannot stress this highly enough, each and every file and portion of fi…

Merkle Trees FTW, but it sounds like you've basically reimplemented ZFS. Have you tried using ZFS on one of these?

Earlier we were totally interested in ZFS, as it would replace RAID & LVM as well (and ZFS gets great reviews). But (to my understanding) native ZFS is not available on Linux and we're not really looking to switch to OpenSolaris.

ANOTHER option down this line of thinking is switching to btrfs, but we haven't played with it yet.

By the way, at Backblaze I've felt like we have had to implement several things that I would have guessed would be standard "off the shelf". One example: When a customer wants to download a restore file with a web browser, are you aware there are no checksums for over-the-network transfers other than the built in (completely unacceptable) 16-bit TCP checksum? You are virtually guaranteed to have an undetected corruption within 30 Gbytes of download, which is basically what we like to call "a totally average customer restore". So Backblaze had to write our own custom reliable, restartable "downloader". It boggles my mind that the whole internet is throwing undetected errors on HTTP downloads and nobody cares to fix the protocol?!! Where the heck is Google, Apple, Facebook, Microsoft, or defining a standard for web browser downloads larger than a few GBytes?

Re: 135TB for $7,384 - Backblaze Pod 2.0

#35

Post by brianwski »

nantes wrote: post #26
brianwski wrote: post #14

Earlier quoted context omitted.

We would like to, we just haven't had time to get it done yet. It runs internally, but is lacking an installer and a GUI, and we would need to prioritize and choose one or more Linux distributions to launch with. Ubuntu is an obvious choice (we focus more on desktop backup than on servers). But some people also ask for CentOS and a few others. It bums me out the Linux community has not solved binary compatibility any…

Any chance of releasing the XML file specs so someone in the community can write an OSS GUI?

See my other post on this. It is extremely simple XML anybody would understand instantly. In fact, some people have already scripted changes to the XML files running on Mac to increase the "throttle" at night and decrease the "throttle" during the day. We'll be giving the Linux client some more thought over the next few days.

Re: 135TB for $7,384 - Backblaze Pod 2.0

#36

Post by barrkel »

brianwski wrote: post #14
alfet wrote: post #7

Does anyone know if Backblaze will ever support Linux? I've wanted to use their service for a while now, but their lack of Linux support has being a big turn down, and I don't think they have made any change in their statements regarding this 'issue'.

We would like to, we just haven't had time to get it done yet. It runs internally, but is lacking an installer and a GUI, and we would need to prioritize and choose one or more Linux distributions to launch with. Ubuntu is an obvious choice (we focus more on desktop backup than on servers). But some people also ask for CentOS and a few others. It bums me out the Linux community has not solved binary compatibility any…

I like the way CrashPlan works. I use it to back up both Windows and Solaris. The GUI is Java, but I only run it on Windows; the GUI communicates over TCP to the Solaris machine. This seems to me the best compromise.

Similarly, the CrashPlan service itself is written in Java and for binary compatibility, only relies on a little binary library loaded via JNI to accommodate the differences between Linux and Solaris, not to mention Windows and Mac. It doesn't even support Nexenta (OpenSolaris kernel + Debian userland) directly out of the box, but I was able to hand-combine the bits from Linux and Solaris installs to get something working: the moral being, for OSes targeted towards more technically adept users, you may not need as much work as you think, if your app is self-contained enough.

Re: 135TB for $7,384 - Backblaze Pod 2.0

#37

Post by brianwski »

ComputerGuru wrote: post #32

I wanted to use Backblaze after hearing people rave about it, but on Lion it completely breaks the network stack somehow during upload/backup..

Yikes, what? We all run Lion here at Backblaze and it works flawlessly for us, and our internal stats show THOUSANDS of Lion customers happily backing up. If you can provide more details of your setup we would LOVE to fix that for you. Since Mac customers make up more than half our customer base, we're completely, 100 percent dedicated to Lion. Send us info or contact us at our company name at twitter or facebook or use the help links off the homepage.

As a side note -> Backblaze only does one thing -> HTTPS POST. In user space. We do NOT extend the kernel, we do not have drivers (at all), we simply read files (we don't even write to your files), compress and encrypt them in RAM, and push them through the completely common HTTPS. It is unusual for Backblaze to cause any problems except a sluggish network or a hiccup in a skype phone call.

Re: 135TB for $7,384 - Backblaze Pod 2.0

#38

Post by zerosanity »

brianwski wrote: post #34
db48x wrote: post #30

Earlier quoted context omitted.

Merkle Trees FTW, but it sounds like you've basically reimplemented ZFS. Have you tried using ZFS on one of these?

Earlier we were totally interested in ZFS, as it would replace RAID & LVM as well (and ZFS gets great reviews). But (to my understanding) native ZFS is not available on Linux and we're not really looking to switch to OpenSolaris. ANOTHER option down this line of thinking is switching to btrfs, but we haven't played with it yet. By the way, at Backblaze I've felt like we have had to implement several things that I wou…

We're using FreeBSD's ZFS support and haven't had any hiccups to complain about.

Re: 135TB for $7,384 - Backblaze Pod 2.0

#39

Post by ebiester »

brianwski wrote: post #33
cakeface wrote: post #29

Earlier quoted context omitted.

It sounds like those XML files are already there when you install the windows or mac version. Why don't you just open them up and edit them by hand?

Honestly, the XML files are pretty simple. The one main one the GUI writes out is called bzinfo.xml and is found on any Mac system at /Library/Backblaze/bzdata/bzinfo.xml and on any Windows Vista or later system at C:\ProgramData\Backblaze\bzdata\bzinfo.xml Backblaze is designed to be used with absolutely no configuration (for many users they have no idea where their Outlook.pst file is and we don't think they should…

To a linux user, editing an xml file is easy to use. And since it's a pretty reasonable default to backup /home, you're likely to be pretty safe with defaults anyway.

Call it alpha, see how much demand there is, and the extra money you bring in might be the motivation to finish up the pretty. :)

Re: 135TB for $7,384 - Backblaze Pod 2.0

#40

Post by db48x »

brianwski wrote: post #34
db48x wrote: post #30

Earlier quoted context omitted.

Merkle Trees FTW, but it sounds like you've basically reimplemented ZFS. Have you tried using ZFS on one of these?

Earlier we were totally interested in ZFS, as it would replace RAID & LVM as well (and ZFS gets great reviews). But (to my understanding) native ZFS is not available on Linux and we're not really looking to switch to OpenSolaris. ANOTHER option down this line of thinking is switching to btrfs, but we haven't played with it yet. By the way, at Backblaze I've felt like we have had to implement several things that I wou…

Actually, zfs-on-linux is doing pretty well. It's not production ready yet, of course.

As for HTTP, I agree with you entirely! There is actually a standard that lets you put a hash (SHA-1 or SHA-256 or something) on an http anchor link, and the browser will verify it when the download finishes. It hasn't really gained very wide adoption though. Personally, I think something akin to Bittorrent is a better solution, since it doesn't have to redownload the entire file when it detects an error. It's ironic that our videos often have better data integrity than the graphics driver installer that I just downloaded, or the browser that I downloaded yesterday.

Post reply on HN