Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: SadServers – Test your Linux troubleshooting skills

sadservers.com

Post reply on HN
Print view View original

111–120 of 133 posts

Re: Show HN: SadServers – Test your Linux troubleshooting skills

#111

Post by paulfurtado »

encryptluks2 wrote: post #106
spiffytech wrote: post #97

Earlier quoted context omitted.

Containers have a history of escape vulnerabilities, for reasons like sharing a kernel with the host and other containers. VMs are designed from the ground up to isolate guests, rather than focusing on application deployment. Firecracker is the modern container alternative in untrusted compute scenarios, with Fly.io even converting container images into Firecracker VMs.

User mapping is now a standard feature in Kubernetes, so escape vulnerabilities aren't so much an issue anymore. Additionally, you can use gVisor.

User namespaces have resulted in multiple new container breakout CVEs in the last year. Some guides actually recommend disabling user namespaces because they are still somewhat new and perilous.

Re: Show HN: SadServers – Test your Linux troubleshooting skills

#112

Post by paulfurtado »

BossingAround wrote: post #84
apawloski wrote: post #13

Based on your architecture diagram it looks like you're spinning up an instance per-user? As you're probably finding now, you will hit AWS limits quickly. You might instead want to have a smaller pool of (larger) servers that you run co-resident VMs on with https://firecracker-microvm.github.io/ . That will avoid account limits and also keep your AWS costs more predictable.

Why not spin up containers instead of VMs? Seems to me containers would fit much better than VMs.

If the goal of the test is to debug a sad linux server, containers are going to severely limit what ways the server can be sad in, isn't it?

Re: Show HN: SadServers – Test your Linux troubleshooting skills

#114

Post by hunter2_ »

jacooper wrote: post #109
georgyo wrote: post #108

Earlier quoted context omitted.

If it is hosted on AWS the bandwidth of distributing the images is likely more than the cost of the compute.

Cloudflare exists

I thought Cloudflare only ensures high usage of the free tier for "web"-ish responses, which doesn't even include .txt files. But I suppose this use case is several orders of magnitude away from that of EasyList, at least in request rate.

Re: Show HN: SadServers – Test your Linux troubleshooting skills

#115

Post by nneonneo »

NovemberWhiskey wrote: post #98
spiffytech wrote: post #97

Earlier quoted context omitted.

Containers have a history of escape vulnerabilities, for reasons like sharing a kernel with the host and other containers. VMs are designed from the ground up to isolate guests, rather than focusing on application deployment. Firecracker is the modern container alternative in untrusted compute scenarios, with Fly.io even converting container images into Firecracker VMs.

> Containers have a history of escape vulnerabilities Generally agreed, but for this use-case do we care?

I haven’t gotten any of the challenges to load, but if you’re going to simulate a sysadmin it would make sense to give you high privileges (or even root) on the box. The more privileged you are inside a container, the more attack surface you expose.

Re: Show HN: SadServers – Test your Linux troubleshooting skills

#116

Post by nobody9999 »

mewse-hn wrote: post #34

Completed the first challenge and it was a lot of fun - spoiler I've never had to use the 'lsof' command before.

>Completed the first challenge and it was a lot of fun - spoiler I've never had to use the 'lsof' command before.

I've been waiting a while for the "sad server" to come up for me and read the scenario (saint john) whilst waiting.

lsof was the first thing that came to mind after reading the scenario.

I guess that once I actually get a "sad server" I'll make it "happy" quickly :)

Re: Show HN: SadServers – Test your Linux troubleshooting skills

#117

Post by encryptluks2 »

paulfurtado wrote: post #111
encryptluks2 wrote: post #106

Earlier quoted context omitted.

User mapping is now a standard feature in Kubernetes, so escape vulnerabilities aren't so much an issue anymore. Additionally, you can use gVisor.

User namespaces have resulted in multiple new container breakout CVEs in the last year. Some guides actually recommend disabling user namespaces because they are still somewhat new and perilous.

What are the ones in the last year? They provide security benefits as well. I mean, you could say the Linux kernel is also dangerous and the Windows kernel and pretty much anything that has ever had a CVE. You can also limit it to specific users too if that is a major concern.

Re: Show HN: SadServers – Test your Linux troubleshooting skills

#118

Post by loa_in_ »

teddyh wrote: post #65
Timja wrote: post #19

Earlier quoted context omitted.

Depends on how the broken program writes to the log. If it does while true; do echo hello >> bad.log; done Then renaming bad.log will not solve the challenge.

Replace it with a symlink to /dev/null! Or /dev/full if we feel like it. (Yes, these are bad solutions, since the instructions explicitly said to stop the process which is writing.)

It will still keep writing to an open inode

Re: Show HN: SadServers – Test your Linux troubleshooting skills

#119

Post by Sebguer »

deeblering4 wrote: post #6

> It's also my not-so-secret hope that a sophisticated enough version of SadServers could be used by tech companies (or for companies that carry on job interviews on their behalf) to automate or facilitate the Linux troubleshooting interview section. Yup, that's what I was afraid of.

Already exists. I can't remember the name, but the infra company that I used to work for used one of these as part of their interview loop.

Re: Show HN: SadServers – Test your Linux troubleshooting skills

#120

Post by cyphar »

paulfurtado wrote: post #111
encryptluks2 wrote: post #106

Earlier quoted context omitted.

User mapping is now a standard feature in Kubernetes, so escape vulnerabilities aren't so much an issue anymore. Additionally, you can use gVisor.

User namespaces have resulted in multiple new container breakout CVEs in the last year. Some guides actually recommend disabling user namespaces because they are still somewhat new and perilous.

You're talking about creating new user namespaces inside a container, not running a container in a user namespace. Running a container in a user namespace is strictly a security improvement over running it in the host user namespace.

Also, all container runtimes automatically block unshare(CLONE_NEWUSER) with seccomp already (unless they've disabled seccomp, which I'm not sure if Kubernetes still does).

Post reply on HN