Earlier quoted context omitted.
I haven't fully grokked this yet, but one trick I've used in the past to get around limits is AWS Organizations, creating a sub-account per property. A bit more setup but can keep things cleaner administratively.
AWS will raise limits if you ask. Increasing EC2 instance limits is usually a quick turn around.
Show HN: SadServers – Test your Linux troubleshooting skills
61–70 of 133 posts
Re: Show HN: SadServers – Test your Linux troubleshooting skills
#62>Practice for your next SRE/DevOps interview. Are SREs and DevOps tasked with administration of operating systems?
Re: Show HN: SadServers – Test your Linux troubleshooting skills
#63Based on your architecture diagram it looks like you're spinning up an instance per-user? As you're probably finding now, you will hit AWS limits quickly. You might instead want to have a smaller pool of (larger) servers that you run co-resident VMs on with https://firecracker-microvm.github.io/ . That will avoid account limits and also keep your AWS costs more predictable.
trollface.jpg
Re: Show HN: SadServers – Test your Linux troubleshooting skills
#64I'd suggest integrating https://bellard.org/jslinux/ and running the VM in the browser if you can - then you can scale without running out of resources.
Thanks, I've been looking at WASM, for ex https://github.com/snaplet/postgres-wasm/tree/main/packages/... , it would certainly simplify everything to "download a fat file".
Re: Show HN: SadServers – Test your Linux troubleshooting skills
#65Can't get to the first problem because of HN hug but anyway there are fake ways to "solve" it like renaming the logfile (what they test for solved is provided).
Depends on how the broken program writes to the log. If it does while true; do echo hello >> bad.log; done Then renaming bad.log will not solve the challenge.
(Yes, these are bad solutions, since the instructions explicitly said to stop the process which is writing.)
Re: Show HN: SadServers – Test your Linux troubleshooting skills
#66Very cool! This reminds me of the ops challenge @ Slack. I'm not sure if they still do this, but the SRE/platform infra interview used to involve a VM running a malfunctioning LAMP stack. You'd get SSH access to the VM, then submit a diagnostic report of what was broken (and how you fixed it). Reminded me of how Red Hat used to run their certification test (RHCE). I probably still have the live CDs for my RHCE laying…
Usually a simple combination of immutable files, SELinux policies, and types in configuration files were enough for most of the challenges. Though now and again you'd find they'd given you a server with packages removed, or not yet installed.
Re: Show HN: SadServers – Test your Linux troubleshooting skills
#67Earlier quoted context omitted.
Why are you afraid of this? My org has run a hands-on technical exam with a stack of linux admin basics (I won't enumerate them here because people do their research) but they are based on real problems we've had and the feedback is overwhelmingly "this was one of the best technical interviews I've ever had." We ask the engineer who is proctoring the interview to think about the following question: Would you want to…
If you give the person you're interviewing access to the same tools they'd have in a regular day on the job (Google, manpages, etc.), I'd say that's a fair and probably relatively enjoyable interview. Rejecting someone because they can't recall the correct netstat syntax doesn't seem like good hiring practice, but I assume in good faith that's not what you meant :)
And usually you only use a fairly small set of tools that often, in any job, and which set will depend on the employer, how things are set up, and what exactly you're doing.
Oh and somehow I get "-r" versus "-R" for "recursive" wrong almost every time, even for commands I type almost daily, unless I check first. It's weird. If tools could get on the same damn page about which means "recursive", that'd be great.
TL;DR I do have a pretty good idea what I'm doing, but look like an absolute idiot if anyone watches me do it. Much worse, even, if I know they're watching and we're not in some kind of relatively high-trust relationship (so, definitely not in an interview setting).
Re: Show HN: SadServers – Test your Linux troubleshooting skills
#68My only feedback is that this is unrealistic because today developers wouldn’t try to debug something, they’d just destroy the instance, push a commit and hope it fixed something infra related then recreate it. Why would you need to understand how something works? Just use containers. /s
If most developers can't debug a VM, then anyone who can will be able to charge a premium. If you have a proficiency in ops, remember that the next time you negotiate a compensation package. [Edited my compensation numbers to avoid down votes - yikes]
My guess is trying to sell high end services as a "principal software engineer" isn't going to be enough to justify that cash comp to a lot of people hiring.
Re: Show HN: SadServers – Test your Linux troubleshooting skills
#69Earlier quoted context omitted.
Why are you afraid of this? My org has run a hands-on technical exam with a stack of linux admin basics (I won't enumerate them here because people do their research) but they are based on real problems we've had and the feedback is overwhelmingly "this was one of the best technical interviews I've ever had." We ask the engineer who is proctoring the interview to think about the following question: Would you want to…
Yeah, we did this at a previous employer. One example, is we had them ssh, download & extract a tarball (the Linux source, but the content doesn't matter). Sometimes, they'd gunzip to stdout. The reaction tells you a lot "lol whoopsie " followed by a quick fix: person knows what they're doing. "uh… what is going on? did I break it?" followed with general cluelessness… maybe not. That did occasionally break tmux, thou…
Re: Show HN: SadServers – Test your Linux troubleshooting skills
#70Commenting to give this a try later, I've routinely been the person to get these kinds of gremlins escalated I've long wanted for some sort of mock, "things are broken - I want to see how you think" approach for sysad
188 points by fduran 3 hours ago | unvote | flag | hide | past | favorite | 68 comments
If you click 'favorite' it will save it to your favorites list. This is a publicly visible list - yours is https://news.ycombinator.com/favorites?id=bravetraveler and mine is https://news.ycombinator.com/favorites?id=shagie which makes it easy to get a bookmark type style functionality within HN.As I tend to favorite less often than I comment, it makes it easier to find those things I want to find again.