Live data from Hacker News

FreeBSD SSH Hardening

gist.github.com

111–120 of 121 posts

Re: FreeBSD SSH Hardening

#111
post #95
post #8

Relevant: If your SSH server is public, you can give its address to https://sshcheck.com/ and it will report any weak spots in your config.

I can recommend similar, but locally running tool, `ssh-audit`: https://github.com/jtesta/ssh-audit Disclaimer: I'm not associated in any way

Links to a publicly accessible https://www.ssh-audit.com

Re: FreeBSD SSH Hardening

#112
post #50

Earlier quoted context omitted.

What would you do if your Wireguard tunnel dies? That's the one thing that's prevented me from actually doing this.

The same thing that happens if the SSH daemon dies, I guess? FWIW, I’ve been using Wireguard for a while (probably ~2 years?) as an always-on VPN for multiple mobile devices, and also as a reverse tunnel to pinhole service access inside a LAN. The Wireguard config and daemon has been rock solid. The only time it’s failed is when I messed up the AllowedIPs, but that failure occurs at configuration time. It has never c…

That's a good point.

I guess I'll give it a try for some time.

Re: FreeBSD SSH Hardening

#113
post #7

I always heard that FreeBSD has unparalleled networking Does it mean that it'd be worth picking FreeBSD over Linux for my C# crud app if it had to handle a lot of requests/sec? (let's ignore db for the moment)

As with all things, you would really need to benchmark the system, preferably with real load, both ways to know for sure. But that takes a lot of time, especially if you're going to put in the time to tweak both systems. People can do amazing stuff with enough time in both FreeBSD and Linux. I honestly think most server applications wouldn't be held back by either OS. You need your application to be really lightweigh…

> when I upgrade, everything will keep working

FreeBSD doesn't promise compatibility between major versions, it's a breaking change, e.g. in FreeBSD 12 they revamped some system types and like half of system interface became incompatible with FreeBSD 11.

Re: FreeBSD SSH Hardening

#114
post #93

Earlier quoted context omitted.

Hi toast0 IIRC, I saw a presentation by someone (Rick?) where at your previous employer - you guys slimmed FreeBSD down to be unbelievably minimal such that only 2-3 total services ran on the entire server. Was that done for performance reason? Or for hardening reasons? If someone wanted to do that today with FreeBSD: would you recommend it and how would you go about doing it (NanoBSD)?

Probably Rick, he did a bunch of presentations. We ran a pretty vanilla FreeBSD. IIRC, we disabled locales, and had 5-10 patches depending on exactly when and what bottlenecks we were running into. We didn't go out of our way to turn off daemons, we just didn't start anything we didn't need. Out of the box, you get sshd, crond, getty, ntpd, syslogd, our system activity report, and that's really all you need to admini…

Thanks so much. Really appreciate the insight.

Would you use FreeBSD for new messaging/web-service today or Linux?

(I ask given you have first hand knowledge of how FreeBSD performed vs Linux after your organization made the migration)

Re: FreeBSD SSH Hardening

#115
post #84

Earlier quoted context omitted.

> None of the algorithms enabled by default has any known security issue. For the tinfoil among us, jump on the post-quantum key exchange train, there's little overhead, it still uses traditional elliptic curve crypto, best of both worlds. The jump-off: sntrup4591761x25519-sha512@tinyssh.org

Note that the above post-quantum key exchange method was removed in OpenSSH 8.5 (released March 3, 2021) in favor of a newer one: sntrup761x25519-sha512@openssh.com So if you add the previous one to your server config, sshd may fail to start after upgrading to 8.5 or newer (8.7 is the most recent release). I started using sntrup4591761x25519-sha512@tinyssh.org about two years ago, and the new one when OpenSSH 8.5 cam…

Thanks for the update, guessing this is the round 2 submission?

Despite being sidelined in NIST's post quantum standardisation project it's interesting to see their Streamlined NTRU Prime algorithm still being the main real world adoption out there right now. I'd be interested if anyone knows of a more utilised post-quantum algo irl.

Re: FreeBSD SSH Hardening

#116
post #93

Earlier quoted context omitted.

Probably Rick, he did a bunch of presentations. We ran a pretty vanilla FreeBSD. IIRC, we disabled locales, and had 5-10 patches depending on exactly when and what bottlenecks we were running into. We didn't go out of our way to turn off daemons, we just didn't start anything we didn't need. Out of the box, you get sshd, crond, getty, ntpd, syslogd, our system activity report, and that's really all you need to admini…

Thanks so much. Really appreciate the insight. Would you use FreeBSD for new messaging/web-service today or Linux? (I ask given you have first hand knowledge of how FreeBSD performed vs Linux after your organization made the migration)

Personally, I would use FreeBSD. But, to be honest, that's 99% because of familiarity and only 1% because I hope the performance is better. Some of the familiarity is just knowing how the system works and how to do routine tasks easily. Some of it is knowing what manual pages will give me the answers I need and being familiar with the writing so I can look it up. But also, I know a lot of important sysctls to adjust behaviors and where to look if there's something that needs adjusting that I think might have a knob. And, I'm fairly handy at digging through the kernel now that I've done it enough. The Linux kernel is organized differently, and I'd need to develop that skill/knowledge. Getting changes upstreamed is a challenge and I've had some success with FreeBSD, but Linux kernel would be new processes, and some things that are really core to a usable system (for example ifconfig/ip) are developed seperately, so have their own processes, so potentially more processes to learn.

Mostly our software ran fine in Linux and the things that bothered me the most were mostly artifacts of how Facebook runs their servers. That said, some things in Linux bother me: why does ss exist as a faster netstat instead of just making netstat faster? I don't find the new init framework compelling on servers (I don't hotplug anything, and I don't think init should restart crashed server processes) and it actively hinders me on desktop/laptop, so it's frustrating that the ecosystem has embraced it (but I'm trying to avoid an offtopic rant, email in profile if you want a rant).

We had a couple migration issues, but nothing serious. epoll and kqueue are similar, but not similar enough that we didn't need to tweak some things in Erlang to make it work better, but the chat machines in FB have much less ram because we had to select from FB's menu of server hardware and more nodes with less ram in each makes the most sense on their menu, so I can't say for sure if epoll scales as well as kqueue. I understand epoll takes more syscalls to do the same work (for network I/O anyway), so that's probably worse, but it likely doesn't make a big difference, but then there's also io_uring which I haven't evaluated and might be better than either epoll or kqueue. I think Linux's default memory overcommit behavior is problematic for a single process server, but it's hard to disable it because the ecosystem has formed around it. Tools allocate huge memory because 'it doesn't matter, it only really allocates when I write to it', and like, I understand how that's useful, but it's impossible for a program to handle out of memory when the trigger is a write to allocated space; alloc failures are hard to handle, but with careful design you can get a fairly consistent crash report most of the time.

Re: FreeBSD SSH Hardening

#117
post #39

Earlier quoted context omitted.

I looked into this for a project a couple of years ago (to boot VMs from minimal customized ISO) and ended up using mfsbsd instead. https://github.com/mmatuska/mfsbsd

That's interesting, this appears to be created by a core FreeBSD developer. https://mfsbsd.vx.sk I'm not able to find a lot of documentation on it. How does it compare to NanoBSD (and why did you choose it over Nano)?

mfsbsd builds a custom live CD using an easily-customized Makefile. NanoBSD expects multiple partitions, some of which are mutable, so is more oriented toward booting small systems from flash.

For my purposes I needed an application server that would reboot to a known-good state, so I embedded the application code inside a mfsbsd live CD.

Re: FreeBSD SSH Hardening

#118
post #7

Earlier quoted context omitted.

As with all things, you would really need to benchmark the system, preferably with real load, both ways to know for sure. But that takes a lot of time, especially if you're going to put in the time to tweak both systems. People can do amazing stuff with enough time in both FreeBSD and Linux. I honestly think most server applications wouldn't be held back by either OS. You need your application to be really lightweigh…

> when I upgrade, everything will keep working FreeBSD doesn't promise compatibility between major versions, it's a breaking change, e.g. in FreeBSD 12 they revamped some system types and like half of system interface became incompatible with FreeBSD 11.

It depends on the specifics, but if you install the -compat packages and enable the compat kernel flags, broadly speaking old software continues to work across major versions. Anyway, just because I expect it to happen, doesn't mean it happens (and I acknowledged that).

Things I've run into include when they made the CPU masks bigger, old binaries couldn't set CPU affinity (even when the old code had a mask big enough for the current CPU; we had a local patch for this, since upstream didn't seem interested in making it right).

Updating to 13 at home with the approved process (install kernel, reboot, install userland, reboot (maybe), do ports/pkg update), resulted in bad network configuration because 12.x ifconfig didn't fully work with 13.0 kernel, but I believe that's been addressed; and I never hit similar issues in prod because we would install both the new kernel and new userland before rebooting, because redundant servers means you can live dangerously (if the first server fails to start, you can improve the upgrade process on the next ones). This one wasn't terrible for me, but I can imagine someone having a very negative experience and holding a grudge if they had a less accessible machine and specific network configs that made the machine unreachable after the first reboot, my machine was accessible but misbehaving and I've got easy access to the console I rolled back and later did a more cautious update.

FreeBSD 11 (+/- 1) changed page inactivity policies in an unexpected way and resulted in undesired swapping for our use cases and becauae it was a slow process it wasn't obvious. That release also negatively impacted our heavy disk I/O systems in a way I didn't have time to get to the bottom of, accurate benchmarking was time consuming and user impacting and migration was in progress, so those systems were forbidden to upgrade. They didn't need the positive networking changes because of their use case, so it was acceptable.

Re: FreeBSD SSH Hardening

#119
post #116

Earlier quoted context omitted.

Thanks so much. Really appreciate the insight. Would you use FreeBSD for new messaging/web-service today or Linux? (I ask given you have first hand knowledge of how FreeBSD performed vs Linux after your organization made the migration)

Personally, I would use FreeBSD. But, to be honest, that's 99% because of familiarity and only 1% because I hope the performance is better. Some of the familiarity is just knowing how the system works and how to do routine tasks easily. Some of it is knowing what manual pages will give me the answers I need and being familiar with the writing so I can look it up. But also, I know a lot of important sysctls to adjust…

That's super interesting and insightful to hear. I really appreciate the time you spent detailing out your prespetive!

I too love FreeBSD (and don't have much in-depth Linux experience). You hit on a topic I'm also interested in as well, which is io_uring. A week doesn't go by that it seems I bump into an article/post about it. Unforunately, I haven't found any solid comparison yet on kqueue vs io_uring (or at least - aren't written by someone in the Linux community who doesn't understand FreeBSD well ... which can be frustrating).

Nonetheless, again - really appreciate your comment.

Re: FreeBSD SSH Hardening

#120
post #84

Earlier quoted context omitted.

> None of the algorithms enabled by default has any known security issue. For the tinfoil among us, jump on the post-quantum key exchange train, there's little overhead, it still uses traditional elliptic curve crypto, best of both worlds. The jump-off: sntrup4591761x25519-sha512@tinyssh.org

Note that the above post-quantum key exchange method was removed in OpenSSH 8.5 (released March 3, 2021) in favor of a newer one: sntrup761x25519-sha512@openssh.com So if you add the previous one to your server config, sshd may fail to start after upgrading to 8.5 or newer (8.7 is the most recent release). I started using sntrup4591761x25519-sha512@tinyssh.org about two years ago, and the new one when OpenSSH 8.5 cam…

Do you compile your own OpenSSH? From what I can tell, the distributions have not really caught up yet.

Bullseye is still on 8.4. OpenSSH on Windows appears to be 8.1.

Post reply on HN