Live data from Hacker News

I got hacked: My Hetzner server started mining Monero

blog.jakesaunders.dev

371–380 of 422 posts

Re: I got hacked: My Hetzner server started mining Monero

#371
post #269

Earlier quoted context omitted.

I keep reading comments by podman fans asking to drop Docker and yet every time I have tried to use podman it failed on me miserably. IMHO it would be better if podman was not designed and sold as a docker drop in replacement but its own thing.

That sucks, I never had any problem running a Dockerfile in podman. I don't know what I do differently, but I would as a principle filter out any container that messes with stuff like docker in docker. Podman doesn't need these kind of shenegians. Also the Docker Compose tool is a well-know exception to the compatibility story. (There is some unofficial podman compose tool, but that is not feature complete and quadle…

[deleted]

Re: I got hacked: My Hetzner server started mining Monero

#372
post #269

Earlier quoted context omitted.

I keep reading comments by podman fans asking to drop Docker and yet every time I have tried to use podman it failed on me miserably. IMHO it would be better if podman was not designed and sold as a docker drop in replacement but its own thing.

That sucks, I never had any problem running a Dockerfile in podman. I don't know what I do differently, but I would as a principle filter out any container that messes with stuff like docker in docker. Podman doesn't need these kind of shenegians. Also the Docker Compose tool is a well-know exception to the compatibility story. (There is some unofficial podman compose tool, but that is not feature complete and quadle…

A side-effect of running rootless and daemonless is that containers stop on user log out, and I can't believe how all this is to be expected for a newcomer to parse. Because I thought the whole point of containers in production was for them to keep running when you log out.

Of course, when you think about it, nobody expects a command to just survive logging out, but coming from docker, you still have that expectation. And I wonder, am I supposed to be running this on a tmux like the old days? No, you need to do a bunch of systemd/linger/stuff. So being that we are already in systemd land, you keep searching and end up in quadlets, which are a new-ish thing with (last I checked) bad docs, replacing whatever was used before (which has good docs). Docs, being said, that give k8s ptsd. Quadlet, podlet and pods.

It seems that when podman deviates from docker, it does in the least ergonomic way possible. Or maybe I have been lobotomized by years and years of using docker, or maybe my patience threshold is very low nowadays. But this has been my experience. I felt very stupid when I deployed something and it stopped after 5 minutes. I was ready to use podman, because it worked locally. And then it failed in production. Thanks no.

Re: I got hacked: My Hetzner server started mining Monero

#373

Earlier quoted context omitted.

Another is running containers in read-only mode, assuming they support this configuration... will minimize a lot of potential attack surface.

Never looked into this. I would expect the majority of images would fail in this configuration. Or am I unduly pessimistic?

It's hit or miss... you sometimes have to make /tmp writable or another data directory... some images just don't operate right because of initialization steps that happen on first run. It's hit or miss and depends... but a lot of your own apps can definitely be made to work with limited, or no write surface.

Re: I got hacked: My Hetzner server started mining Monero

#374

No firewall! Wow that's brave. Hetzner will let you configure one that runs outside of the box so you might want to add that too, as part of your defense in depth - that will cover you if you make a mistake with ufw. Personally I keep SSH firewalled only to my home address in this way; if I'm out and about and need access, I can just log into Hetzner's website and change it temporarily.

The only time I have ever had a machine compromised in 30 years of running Linux is when I ran something exposed to the internet on a well known port. I know port scanners are a thing but the act of using non-default ports seems unreasonably effective at preventing most security problems.

If you do any npm install, pip install ..., docker pull ... / docker run ... , etc in linux. It is very easy to get compromise.

I did docker pull a few times base on some webpost (looks reasonable) and detect app/scripts from inside the docker connect to some .ru sites immediately or a few days later....

Re: I got hacked: My Hetzner server started mining Monero

#375
post #372

Earlier quoted context omitted.

That sucks, I never had any problem running a Dockerfile in podman. I don't know what I do differently, but I would as a principle filter out any container that messes with stuff like docker in docker. Podman doesn't need these kind of shenegians. Also the Docker Compose tool is a well-know exception to the compatibility story. (There is some unofficial podman compose tool, but that is not feature complete and quadle…

A side-effect of running rootless and daemonless is that containers stop on user log out, and I can't believe how all this is to be expected for a newcomer to parse. Because I thought the whole point of containers in production was for them to keep running when you log out. Of course, when you think about it, nobody expects a command to just survive logging out, but coming from docker, you still have that expectation…

  loginctl enable-linger

Re: I got hacked: My Hetzner server started mining Monero

#376
post #372

Earlier quoted context omitted.

A side-effect of running rootless and daemonless is that containers stop on user log out, and I can't believe how all this is to be expected for a newcomer to parse. Because I thought the whole point of containers in production was for them to keep running when you log out. Of course, when you think about it, nobody expects a command to just survive logging out, but coming from docker, you still have that expectation…

loginctl enable-linger

Yes, but I want it to only apply to podman, not any running task.

    systemctl --user enable podman.socket loginctl enable-linger 

?

Re: I got hacked: My Hetzner server started mining Monero

#377
post #49

Earlier quoted context omitted.

Unintentional vulnerabilities in Docker and the kernel aside, it can only do that if it has access to the Docker API (usually through a bind mount of the Unix socket). Having access to the Docker API is equivalent to having root on the host.

Well $hit. I have been using Docker for installing NPM modules in interactive projects I was testing out. I believed Docker blocked access to the underlying host (my computer). Thanks for mentioning it - but now... how does one deal with this?

Podman makes this easier to do safely by default. I'd suggest checking that out.

Re: I got hacked: My Hetzner server started mining Monero

#378
post #376

Earlier quoted context omitted.

loginctl enable-linger

Yes, but I want it to only apply to podman, not any running task. systemctl --user enable podman.socket loginctl enable-linger ?

You should compare it imho to ssh. If you break your connection, your session is gone. So if you only want certain parts of your session survive, which ones should? Because maybe your container depends on avahi on the host, or cups, or...?

Just a random thought, but if you can create a user on the host that just has the most minimal set of systemd services enabled your container needs, you could apply it to that user.

But still, on a server that wouldn't make much sense imho, as the default user is usually the service user having a minimal set of services enabled. On a desktop, your default user is logged in anyways. So I think this isn't a real problem tbh.

Re: I got hacked: My Hetzner server started mining Monero

#379

Earlier quoted context omitted.

Virtual machines are treated as a security boundary despite the fact that with enough R&D they are not. Hosting minecraft servers in virtual machines is fine, but not a great idea if they’re cohosted on a machine that has billions of dollars in crypto or military secrets. Docker is pretty much the same but supposedly more flimsy. Both have non-obvious configuration weaknesses that can lead to escapes.

> Virtual machines are treated as a security boundary despite the fact that with enough R&D they are not. Hosting minecraft servers in virtual machines is fine, but not a great idea if they’re cohosted on a machine that has billions of dollars in crypto or military secrets. While I generally agree with the technical argument, I fail to see the threat model here. Is it that some external threat would have prior knowle…

Yes. Docker too.

Re: I got hacked: My Hetzner server started mining Monero

#380
post #315
post #202

So this is part of the "React2Shell" CVE-2025-55182 issue? I find it interesting that this seems to get so little publicity. Almost like the issue is normal or expected. And it looks like the affected versions go back a little over a year. So if you've deployed anything with Next.js over the last 12 months your web app is now probably part of a million node bot net. And everyone's advice is just "use docker" or "inst…

Frontend churn has chilled out so much over the last few years. The default webapp stack today has been the same for 5 years now, next.js (9yo) react (12yo) tailwind (8yo) postgres (36yo). I'm not endorsing this stack, it just seems to be the norm now. Compare that to what we had in the late 00's and early 10's we went through prototype -> mootools -> jquery -> backbone -> angularjs -> ember -> react, all in about 6…

I remember that. To be honest it was exhausting. Fun, but exhausting. It’s nice now to have found a stack that is “just fine” for most things.
Post reply on HN