51–60 of 62 posts
Earlier quoted context omitted.
This is not true. Read up on mirrored vdevs. http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-...
50% storage efficiency and 87.5% survival is not an option for us. We store medical images by the petabyte and drives fail much too fast at scale.
Earlier quoted context omitted.
This is not true. Read up on mirrored vdevs. http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-...
50% storage efficiency and 87.5% survival is not an option for us. We store medical images by the petabyte and drives fail much too fast at scale.
Earlier quoted context omitted.
That is not correct: one can easily increase the size of the pool by adding larger devices. As soon as the last device is replaced, the pool instantaneously possesses the upgraded capacity. No complicated grow commands, no filesystem expansions. It JustWorks(SM). zpool set autoreplace=on pool_name zpool set autoexpand=on pool_name properties must be set on the pool for this to work, either before or after the pool up…
That is a lot of work compared to just adding a disk and rebalancing. No need to replace every disk in a vdev just to increase disk utilization to the entire disk when using Swift. No fear of losing a pool when losing a vdev.
There is never any fear of losing a pool when losing a vdev, as ZFS will let one simulate the entire thing by letting one use files as disks:
% mkfile -v 128m /var/tmp/d0
/var/tmp/d0 134217728 bytes
% mkfile -v 128m /var/tmp/d1
/var/tmp/d1 134217728 bytes
% sudo zpool create testpool0 mirror /var/tmp/d0 /var/tmp/d1
Password:
% zpool status
pool: testpool0
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
testpool0 ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
/var/tmp/d0 ONLINE 0 0 0
/var/tmp/d1 ONLINE 0 0 0
errors: No known data errors
% zfs list
NAME USED AVAIL REFER MOUNTPOINT
testpool0 988K 79.0M 952K /Volumes/testpool0
% sudo zpool set autoexpand=on testpool0
% mkfile -v 192m /var/tmp/d2
/var/tmp/d2 201326592 bytes
% mkfile -v 192m /var/tmp/d3
/var/tmp/d3 201326592 bytes
% sudo zpool replace testpool0 /var/tmp/d0 /var/tmp/d2
% sudo zpool replace testpool0 /var/tmp/d1 /var/tmp/d3
% zfs list
NAME USED AVAIL REFER MOUNTPOINT
testpool0 1008K 143M 952K /Volumes/testpool0
% sudo zpool destroy testpool0
Running process: '/usr/sbin/diskutil' 'unmount' '/Volumes/testpool0'
Unmount successful for /Volumes/testpool0
% rm /var/tmp/d[0-9]Earlier quoted context omitted.
That is a lot of work compared to just adding a disk and rebalancing. No need to replace every disk in a vdev just to increase disk utilization to the entire disk when using Swift. No fear of losing a pool when losing a vdev.
"Increase disk utilization when using Swift"? What you wrote makes no sense. There is never any fear of losing a pool when losing a vdev, as ZFS will let one simulate the entire thing by letting one use files as disks: % mkfile -v 128m /var/tmp/d0 /var/tmp/d0 134217728 bytes % mkfile -v 128m /var/tmp/d1 /var/tmp/d1 134217728 bytes % sudo zpool create testpool0 mirror /var/tmp/d0 /var/tmp/d1 Password: % zpool status p…
It says "Fault tolerance / degraded performance
Be careful here. Keep in mind that if any single vdev fails, the entire pool fails with it. There is no fault tolerance at the pool level, only at the individual vdev level!"
Earlier quoted context omitted.
That is a lot of work compared to just adding a disk and rebalancing. No need to replace every disk in a vdev just to increase disk utilization to the entire disk when using Swift. No fear of losing a pool when losing a vdev.
"Increase disk utilization when using Swift"? What you wrote makes no sense. There is never any fear of losing a pool when losing a vdev, as ZFS will let one simulate the entire thing by letting one use files as disks: % mkfile -v 128m /var/tmp/d0 /var/tmp/d0 134217728 bytes % mkfile -v 128m /var/tmp/d1 /var/tmp/d1 134217728 bytes % sudo zpool create testpool0 mirror /var/tmp/d0 /var/tmp/d1 Password: % zpool status p…
Earlier quoted context omitted.
This is about a completely different type of VM application, and a completely different kind of storage architecture. This is about everything being on the storage box, not just any application state. The two nodes export NFS shares, which host the root filesystems of the nodes being run on the VM servers. Most likely, the two storage heads will be existing on the same subnet as the VMWare boxes, thus making a networ…
> This is about a completely different type of VM application, and a completely different kind of storage architecture. I understand it completely, and I vehemently disagree with everything about it. Providing networked block storage for virtual disks on top of NFS is a recipe for disaster and performance degradation. Just because "everyone" does it doesn't make me like it any more. Using redundant local disk (prefer…
SAN storage and VmWare (ESXi) have been around forever.
VmWare will gladly provision VMs over any SAN storage. Live migration has been available for more than 10 years (that's called VMotion and it requires any of paid edition).
Prefer local storage? Put any sort of local storage on the server, VmWare will gladly provision VM on it.
What's the most important thing in a storage... THE BACKUP!
For backups, there are snapshots capabilities well integrated into VmWare (cold snapshots, live snapshots, incremental snapsnots, whatever you name it).
All of this has existed for more than ten years. Standard protocols (iSCSI, FiberChannel, other) + robust enclosures (all vendors have SAN) + battle tested software and drivers (VmWare).
A couple servers and some TB of storage is a nothing fancy. The cost of the setup is shared between the servers + the labor + many SAS hard drives + the enclosure (a very little part).
It's really hard to take seriously an overly complex setup that is trying to imitate that over cheap untested hardware and software (NFS? seriously?).
Not only it is a recipe for disaster in production but there isn't even any cost savings to be done.
The cost of complexity and labors will be going wild with all the configuration. The hardware costs won't go down (still need many drives and servers). Good luck debugging that and retrieving data when things will go wrong.
Well. I suppose it's fun for home labs and that's about it :D
Earlier quoted context omitted.
"Increase disk utilization when using Swift"? What you wrote makes no sense. There is never any fear of losing a pool when losing a vdev, as ZFS will let one simulate the entire thing by letting one use files as disks: % mkfile -v 128m /var/tmp/d0 /var/tmp/d0 134217728 bytes % mkfile -v 128m /var/tmp/d1 /var/tmp/d1 134217728 bytes % sudo zpool create testpool0 mirror /var/tmp/d0 /var/tmp/d1 Password: % zpool status p…
I didn't say "increase disk utilization with Swift", I said ZFS does not use the whole disk when you add a new large disk unless you replace the entire vdev worth of disks. Swift treats each disk individually and uses weighting to deal with heterogeneous disk sizes which is much cleaner.
Swift is also how many orders of magnitude more complex?
Earlier quoted context omitted.
"Increase disk utilization when using Swift"? What you wrote makes no sense. There is never any fear of losing a pool when losing a vdev, as ZFS will let one simulate the entire thing by letting one use files as disks: % mkfile -v 128m /var/tmp/d0 /var/tmp/d0 134217728 bytes % mkfile -v 128m /var/tmp/d1 /var/tmp/d1 134217728 bytes % sudo zpool create testpool0 mirror /var/tmp/d0 /var/tmp/d1 Password: % zpool status p…
Did you not read the other guys link above? http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-... It says "Fault tolerance / degraded performance Be careful here. Keep in mind that if any single vdev fails, the entire pool fails with it. There is no fault tolerance at the pool level, only at the individual vdev level!"
Please do not assume that everyone here comes from GNU/Linux and knows next to nothing or half-true anecdotes about (Open)ZFS. For example, I learned what and how in ZFS directly from the ZFS development team at Sun Microsystems: Roch Burbonnais, Adam Leventhal, Eric Shrock, and Jeff Bonwick.
Earlier quoted context omitted.
> This is about a completely different type of VM application, and a completely different kind of storage architecture. I understand it completely, and I vehemently disagree with everything about it. Providing networked block storage for virtual disks on top of NFS is a recipe for disaster and performance degradation. Just because "everyone" does it doesn't make me like it any more. Using redundant local disk (prefer…
I really don't understand that fight. SAN storage and VmWare (ESXi) have been around forever. VmWare will gladly provision VMs over any SAN storage. Live migration has been available for more than 10 years (that's called VMotion and it requires any of paid edition). Prefer local storage? Put any sort of local storage on the server, VmWare will gladly provision VM on it. What's the most important thing in a storage...…
Earlier quoted context omitted.
I really don't understand that fight. SAN storage and VmWare (ESXi) have been around forever. VmWare will gladly provision VMs over any SAN storage. Live migration has been available for more than 10 years (that's called VMotion and it requires any of paid edition). Prefer local storage? Put any sort of local storage on the server, VmWare will gladly provision VM on it. What's the most important thing in a storage...…
Huh? I'm not sure I understand.