Live data from Hacker News

From Proxmox to FreeBSD and Sylve in our office lab

iptechnics.com

41–50 of 86 posts

Re: From Proxmox to FreeBSD and Sylve in our office lab

#41
post #28
post #8

Sylve looks like a decent project with a promising future but this article really doesn't explain why they picked it over Proxmox at all. They explain a lot of things but I can't see the advantage over prox other than they wanted to use it.

OP here. One thing we mentioned in the blog but probably didn’t emphasize enough is how deeply ZFS is integrated into the UI. With Sylve, you rarely need to touch the CLI. Snapshots, datasets, ZVOLs, even flashing images directly to ZVOLs, it’s all handled from the UI in a straightforward way. That tight ZFS integration also lets us build more flexible backup workflows. You can back up VMs, jails, or entire datasets…

Do you have any opinions on how this works vs doing iSCSI to some other storage system using ZFS? That's how I've been handling Proxmox on the backend, and have mixed feelings. The GUI leaves a very great deal to be desired in honestly curious ways, have to touch the CLI a lot even for super basic networking or auth stuff, and of course neither side has the same insight to the data structures in question. Either you've got to do ZVOL instances and thus manual effort or scripting, or you give Proxmox a single big blob then let it manage that with LVM but that means the storage side can't give any granular help on snapshots and the like. It still can deal with data integrity and backups and storage redundancy and all that but no further, and some increased overhead. But on the other hand, I do feel like a really firm separation of concerns isn't without value. Having native support though is an interesting alternative I hadn't really considered.

Re: From Proxmox to FreeBSD and Sylve in our office lab

#42

Earlier quoted context omitted.

Nested virtualization can be very handy in both the lab and in production. In the lab, you can try out a new hosting platform by running one atop the other. IE: Proxmox on VMWare, Hyper-V on KVM. This lets you try things out without needing fresh bare metal hardware. In prod, let's say you run workloads in Firecracker VMs. You have plenty of headroom on your existing hardware. Nested virtualization would allow you to…

Perhaps I'm misunderstanding, but wouldn't that case be covered by simply putting some vms under a vnet and others on another vnet and make them talk to each other? I can't also understand what you mean by "fresh bare metal hardware". In either case you don't need bare metal, being a top level vm or a nested one.

If you're evaluating VM hosts (proxmox, hyper-V, vmware, etc...) You need to have support for nested virtualization all the way down. Otherwise, if you want to evaluate a VM infrastructure, you need to start with bare-metal. Really, you just need to make sure that your top level support nested virtualization, but I understand their point.

However, the point about firecracker VMs in place of containers I think is really a good use-case. Firecracker can provide a better isolation environment, so it would be great to be able to run Firecracker VMs for workloads, which would require that the host (and the VM host above) support nested virtualization.

Re: From Proxmox to FreeBSD and Sylve in our office lab

#44
post #23

Earlier quoted context omitted.

Regardless of the number of drives available, you gain an advantage when your file system can leverage snapshots to roll backwards or forwards. There are other Linux-native filesystems that can provide this capability too, but many admins prefer ZFS because the full range of capabilities is unparelleled.

Perhaps I'm missing your point, but proxmox+lxc on zfs storage works fine in proxmox? If just looks like any other storage in proxmox and on commandline you've got all the usual zfs tools

I think it comes down to the standard argument against ZFS on linux -- uncertainty. It works *now*. Will it continue to work? Will any upstream changes in the Linux kernel cause issues with the ZFS modules bolted on top?

It is unlikely for there to be issues with ZFS and Linux. It's too common now, but it's not included in the main Linux tree, so it's not explicitly tested.

So, it's a low risk, but not zero risk.

More to the point here, when working with FreeBSD, ZFS is a first-class citizen (moreso even), so working with it *should* be more integrated with a FreeBSD solution than Proxmox, but how much more (and is that meaningful) is probably a qualitative feel than quantitative fact.

Re: From Proxmox to FreeBSD and Sylve in our office lab

#45
post #35

What does Sylve provide that proxmox doesn't? Or better, how does it do it better than proxmox? This isn't to say that proxmox is the best thing since sliced bread, I'm curious as to what makes sylve better, is it the API?

OP here. It’s less about Sylve doing something Proxmox can’t do, and more about a bunch of QoL improvements that come from us being heavy Proxmox users and building what we felt was missing. A few concrete things: ZFS-first UX: Not just "ZFS as storage”, but everything built around it. Snapshots, clones, ZVOLs, replication, all cleanly exposed in the UI without dropping to CLI. Simple backups without extra infra: Any…

Sounds awesome! Looking forward to trying it out.

Re: From Proxmox to FreeBSD and Sylve in our office lab

#46
post #28
post #8

Sylve looks like a decent project with a promising future but this article really doesn't explain why they picked it over Proxmox at all. They explain a lot of things but I can't see the advantage over prox other than they wanted to use it.

OP here. One thing we mentioned in the blog but probably didn’t emphasize enough is how deeply ZFS is integrated into the UI. With Sylve, you rarely need to touch the CLI. Snapshots, datasets, ZVOLs, even flashing images directly to ZVOLs, it’s all handled from the UI in a straightforward way. That tight ZFS integration also lets us build more flexible backup workflows. You can back up VMs, jails, or entire datasets…

I did actually notice the ZFS gui which is indeed something lacking in proxmox which doesn't default to ZFS in the installer. However once you do install it using ZFS it actually makes use of it pretty well and the user does not need to mess with the zfs cli tools much. Obviously it would be nice to have a GUI for all zfs operations too. Then again even TrueNAS refers you back to the cli for SOME operations.

On proxmox ZFS syncs do not require proxmox backup server, which actually has its own format which is very efficient in speed and disk space, but you do either need something like sanoid/syncoid or use of the shell.

Re: From Proxmox to FreeBSD and Sylve in our office lab

#47
post #11

Earlier quoted context omitted.

Sometimes unification can be an advantage. I run Proxmox at home, but now that I have been drinking the NixOS koolaid over the past 2 years, all of my homelab problems suddenly look like Nix-shaped nails.

Well it looks like we might soon be able to have the benefits of NixOS while also having bhyve (and presumably Sylve): https://github.com/nixos-bsd/nixbsd

https://github.com/SaumonNet/proxmox-nixos

Looks like Nix will eat the world soon. :)

Re: From Proxmox to FreeBSD and Sylve in our office lab

#49
post #47

Earlier quoted context omitted.

Well it looks like we might soon be able to have the benefits of NixOS while also having bhyve (and presumably Sylve): https://github.com/nixos-bsd/nixbsd

https://github.com/SaumonNet/proxmox-nixos Looks like Nix will eat the world soon. :)

Damn this is crazy!

Re: From Proxmox to FreeBSD and Sylve in our office lab

#50

Wow, TIL! https://github.com/AlchemillaHQ/Sylve Also: https://www.youtube.com/watch?v=wo4oD5UON30

There was a bit of discussion over the weekend (70 points, 21 comments) https://news.ycombinator.com/item?id=47557392

Not sure why this copy made the SCP

Post reply on HN