Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The AeroFS Private Cloud

blog.aerofs.com

Post reply on HN
Print view View original

51–60 of 69 posts

Re: The AeroFS Private Cloud

#51

Post by bestham »

barrkel wrote: post #37
HorizonXP wrote: post #5

So I'm actually setting up a private cloud for my company right now. It's a company of 1, so the requirements are quite extensive. :-P I'm building a FreeNAS server with 6x3TB hard drives in a RAID-Z2 config. My goal is to allow my Mac to use it for Time Machine backups, but to also use AeroFS as my file sync mechanism when I'm both in the office and on the road. Hopefully it works out smoothly. I'll have to figure o…

Check out the recovery probabilities of raidz2 vs raid10. You're winning ~17% more space at the cost of a substantial performance hit, and almost no difference in reliability, depending on how you estimate HDD failure rates. Raidz2 is not fast when everything is working well, and if any of your drives develop a speed problem, the whole array will be reduced to that speed. I have a 11x4T raidz2, and will be migrating…

While you might be right about the small space advantage, I would feel more safe if the position of the failing drive (and the next failing drive) does not matter at all.

If a drive develops a speed problem, I'll replace it.

Re: The AeroFS Private Cloud

#52

Post by hemancuso »

So AeroFS had been in beta for like 4 years so they could do serverless peer to peer dropbox. And now they are launching a private cloud server product. Can't help but worry that these guys don't have any clear product they are committed to.

Re: The AeroFS Private Cloud

#53

Post by yapcguy »

mindslight wrote: post #21
nostromo wrote: post #17

Earlier quoted context omitted.

Why not just use BTSync for syncing? I use it for this purpose and it's fantastic. For backup you can use one of many solutions to archive to S3. I wouldn't recommend Time Machine.

Why not use Unison? It only does pairwise synchronization, but is Free software and extremely mature. (sigh, it seems the tech community has become blind to anything that isn't heavily blogvertised.)

Unison is no longer under active development, right? Would love to use it but going forward, is there a big enough community to fix bugs, add new features etc?

http://www.cis.upenn.edu/~bcpierce/unison/status.html

Re: The AeroFS Private Cloud

#56

Post by prostoalex »

rdl wrote: post #30

Silicon Valley startups might not think it's a big deal, but being able to run entirely on a private network (either "behind a firewall", or an entirely disconnected network) is pretty huge. Without AeroFS, your choices today kind of suck, especially for 10-500 person companies (or bigger companies where your corporate option sucks or isn't available). Dropbox doesn't work if you care about security. You're left with…

BitTorrent Sync? Or the idea of discovery by passing secrets around doesn't really fit small business model.

Re: The AeroFS Private Cloud

#57

Post by barrkel »

bestham wrote: post #51
barrkel wrote: post #37

Earlier quoted context omitted.

Check out the recovery probabilities of raidz2 vs raid10. You're winning ~17% more space at the cost of a substantial performance hit, and almost no difference in reliability, depending on how you estimate HDD failure rates. Raidz2 is not fast when everything is working well, and if any of your drives develop a speed problem, the whole array will be reduced to that speed. I have a 11x4T raidz2, and will be migrating…

While you might be right about the small space advantage, I would feel more safe if the position of the failing drive (and the next failing drive) does not matter at all. If a drive develops a speed problem, I'll replace it.

You might feel more safe, but you're not as safe as you feel.

Suppose that once one drive has failed, the chance of another drive failing (at least an URE, but potentially worse) while reading it entirely (as necessary during a scrub / resilver) is 10%. I don't think this is very implausible, as RAID HDD failures are often correlated, and not only because it's inconvenient to distribute HDD purchases.

The chance of successfully recovering in a mirrored scenario is 90%. You only need to read one drive.

In raidz1 with 6 drives, your probability is 0.9^5 - you need to get lucky 5 times, because you need to read every other drive fully to recover. That brings you down to less than 60%. Raidz1 with 5 drives is 0.9^4 - a bit over 65%.

Raidz2 for 6 drives combines these two stats. The probability of one failing disk while reading is 1-0.9^5. But you can recover from that. And the probability of that recovery is 0.9^4. So your consecutive probability of failures is (1-0.9^5)(1-0.9^4). Subtract that from 1 and you get your probability of getting lucky.

Work out the numbers, and you'll find it's only 86%. It's less than mirroring.

There are a couple of simplifications in the maths above (don't forget, a second drive failure means another drive to resilver, and a restart to the procedure), and a big assumption based on reliability %. But frankly, unless you think HDD failure is already fairly rare - and it's not - mirroring is usually ahead on safety, and the lead only increases with number of disks in the array.

I used to think like you do, that I'd feel more safe if a single disk failed, knowing I could still tolerate another failure. That's why my array is currently raidz2. But I worked out the numbers, and it changed my mind. And that's why my next array is going to be raid10.

Not to mention that it's hard to get much more than 150M/sec for any realistic I/O with raidz2. I have over 1G/sec of potential HDD bandwidth. Unless I'm doing a scrub, I can't touch the potential performance. And of course all disks act like they're on the same spindle, there is zero random I/O advantage for raidz.

Re: The AeroFS Private Cloud

#58

Post by zentrus »

Overall I think their approach for easy installation and configuration is a good one. I struggle with these same issues at my job--we sell products with a complex application stack to customers that often have no system administrators. The only issue I see here is with the upgrade path. Particularly for a product that is meant for file storage, I can't imagine downloading a 1TB backup file and uploading it again every time there is a new release.

Re: The AeroFS Private Cloud

#59

Post by yurisagalov »

zentrus wrote: post #58

Overall I think their approach for easy installation and configuration is a good one. I struggle with these same issues at my job--we sell products with a complex application stack to customers that often have no system administrators. The only issue I see here is with the upgrade path. Particularly for a product that is meant for file storage, I can't imagine downloading a 1TB backup file and uploading it again ever…

The Appliance actually does not store any file data on it, so your appliance upgrades likely won't include 1TB backup files :)

The (optional) AeroFS Team Server is what would store file data in the company if you wanted to, but many of our customers actually just end up using the direct peer-to-peer syncing without a team server.

Re: The AeroFS Private Cloud

#60

Post by phlo »

barrkel wrote: post #37
HorizonXP wrote: post #5

So I'm actually setting up a private cloud for my company right now. It's a company of 1, so the requirements are quite extensive. :-P I'm building a FreeNAS server with 6x3TB hard drives in a RAID-Z2 config. My goal is to allow my Mac to use it for Time Machine backups, but to also use AeroFS as my file sync mechanism when I'm both in the office and on the road. Hopefully it works out smoothly. I'll have to figure o…

Check out the recovery probabilities of raidz2 vs raid10. You're winning ~17% more space at the cost of a substantial performance hit, and almost no difference in reliability, depending on how you estimate HDD failure rates. Raidz2 is not fast when everything is working well, and if any of your drives develop a speed problem, the whole array will be reduced to that speed. I have a 11x4T raidz2, and will be migrating…

Could you elaborate a bit on your CrashPlan setup? I have a somewhat similar setup and have been trying to back up my NAS (Synology DS1812+) to CrashPlan for about half a year now. I was able to start the backup, but the transfer rate quickly dropped to about 1.2 Mbps or lower, and having backed up about 1.5 TB the CrashPlan client now won't even run a backup.

Are you using Crashplan+ or any of the Pro/Enterprise plans? What sort of transfer rates do you get? And were you able to complete a full backup of that 11x4T array?

Post reply on HN