It's a trade-off. If you have no disk, the disk can't fail, but the network can, and the remote PXE server can, and the remote SAN can. You can get into a state where you have to pray no servers reboot. Intermittent errors can be real annoying when it makes provisioning fail. (used to work a server farm that'd do server rebuilds over PXE, and ran a few diskless cluster projects) An alternative is you use a RAID array…
A network or PXE server can fail regardless, so this are things that always have to be taken into consideration and in those instances then you address those issues. With this type of setup you do not need a remote SAN as it would defeat the purpose of not having external storage that could store logs. Mullvad has servers all over the world, so a temporary failure in one location will not bring down their entire infr…
Mullvad: Diskless infrastructure using stboot in beta
91–100 of 135 posts
Re: Mullvad: Diskless infrastructure using stboot in beta
#92I've been following Mullvad for a long time and my impression (from countless reviews and comments here on HN) has been quite positive. But here's what I don't understand: Why are the servers located in Sweden, a country that's known for online surveillance[0] like no other country in the EU? From the Wikipedia article[1]: > The law permits the signals intelligence agency, National Defense Radio Establishment, to mon…
Thank you for asking. > Why are the servers located in Sweden, We have 762 servers spread across 38 countries. Less than 10% of our servers are located in Sweden [0]. > a country that's known for online surveillance My cofounder and I started Mullvad as a protest against the growing mass surveillance of Sweden as well as other countries. Our intent was direct political action through entrepreneurship. Incorporating t…
Re: Mullvad: Diskless infrastructure using stboot in beta
#93I've been following Mullvad for a long time and my impression (from countless reviews and comments here on HN) has been quite positive. But here's what I don't understand: Why are the servers located in Sweden, a country that's known for online surveillance[0] like no other country in the EU? From the Wikipedia article[1]: > The law permits the signals intelligence agency, National Defense Radio Establishment, to mon…
Thank you for asking. > Why are the servers located in Sweden, We have 762 servers spread across 38 countries. Less than 10% of our servers are located in Sweden [0]. > a country that's known for online surveillance My cofounder and I started Mullvad as a protest against the growing mass surveillance of Sweden as well as other countries. Our intent was direct political action through entrepreneurship. Incorporating t…
Re: Mullvad: Diskless infrastructure using stboot in beta
#94It's a trade-off. If you have no disk, the disk can't fail, but the network can, and the remote PXE server can, and the remote SAN can. You can get into a state where you have to pray no servers reboot. Intermittent errors can be real annoying when it makes provisioning fail. (used to work a server farm that'd do server rebuilds over PXE, and ran a few diskless cluster projects) An alternative is you use a RAID array…
A network or PXE server can fail regardless, so this are things that always have to be taken into consideration and in those instances then you address those issues. With this type of setup you do not need a remote SAN as it would defeat the purpose of not having external storage that could store logs. Mullvad has servers all over the world, so a temporary failure in one location will not bring down their entire infr…
They can still use local disks to provision the OS over a network but boot from local storage, and prevent writing to disks from the booted OS (hell, they can completely remove the disk drivers from the kernel!). It just doesn't make sense to ditch the drives from a reliability standpoint. They're going to have a big outage one day just because they didn't want to deal with drives.
Re: Mullvad: Diskless infrastructure using stboot in beta
#95Earlier quoted context omitted.
my grocer down the road is a nice fella and has tasty vegetables from sustainable sources, but he might get bought up by a big supermarket chain, so I'm not going to buy from him
your grocer does not have the ability to retroactively change the food you've bought from them. A VPN provider can "accidentally" enable logging prior to the sale
Re: Mullvad: Diskless infrastructure using stboot in beta
#96A bit tangential to the main post, but I'd to share a recent positive experience with Mullvad: I am a regular user of Mullvad and recently wanted to try a different VPN, that only provides Wireguard configs (i.e. no native app). I used the default setup. For some reason, my internet connection was flaky, and when it disconnected and reconnected, my traffic leaked. That never happened to me with Mullvad as the app com…
On linux you can create a network namespace exposing only the wireguard network device, so that applications in that namespace cannot leak traffic. Setting this up, however, is quite fiddly in my experience.
https://github.com/VTimofeenko/wireguard-namespace-service/b...
Re: Mullvad: Diskless infrastructure using stboot in beta
#97Earlier quoted context omitted.
Or disable usb in bios entirely
Disabling USB in BIOS only disables the emulation of classic PS2 keyboards and IDE storage so that old OSes or bootloaders without USB stacks can work with modern equipment. As soon as the OS kernel initializes the PCI bus, USB will work again - however they could go and remove the xHCI modules from the kernel and image.
As a result they can absolutely disable USB entirely by never exposing those parts of the device tree to Linux.
Re: Mullvad: Diskless infrastructure using stboot in beta
#98The next step: dedicated VPN over FPGAs
Re: Mullvad: Diskless infrastructure using stboot in beta
#99Earlier quoted context omitted.
A network or PXE server can fail regardless, so this are things that always have to be taken into consideration and in those instances then you address those issues. With this type of setup you do not need a remote SAN as it would defeat the purpose of not having external storage that could store logs. Mullvad has servers all over the world, so a temporary failure in one location will not bring down their entire infr…
It's not just a temporary failure, it's potentially the entire AZ going down hard. High Availability network boot without local storage is very difficult/expensive. They can still use local disks to provision the OS over a network but boot from local storage, and prevent writing to disks from the booted OS (hell, they can completely remove the disk drivers from the kernel!). It just doesn't make sense to ditch the dr…
The point of not using local disks is again fairly straightforward, to show that they do not have a stateful storage medium to write logs to. Whether it significantly helps or not is beside the point, they have determined that it helps provide assurance to their customers and additionally showcases a feature for auditors.
Network booting loads the OS into RAM, so even if there was a network outage they'd have to restart the severs to cause a problem. From what I know of most VPN solutions though, again a network outage would only affect the group of servers at that data center which isn't their entire operation.
Re: Mullvad: Diskless infrastructure using stboot in beta
#100For people wondering how the hell a user can audit the server is diskless or whatever, the goal appears to be using TPM to provide remote attestation for all code in the boot path. See https://www.system-transparency.org/ .
Correct! Thank you for highlighting that. Here are some additional details for those interested. We intend to make use of TPM for remote attestation of the current boot chain, reproducible builds to provide a strong link from source code to build artifacts, and a transparency log for a historical record of previously used boot chains, artifacts, WireGuard server keys, and related signatures. As dtx1 mentioned elsewhe…