Live data from Hacker News

A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

blog.newsblur.com

131–140 of 275 posts

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#131
post #4

The mistake here is a MongoDB that didn't require authentication, not that docker's clunky iptables setup exposed it to the internet. Relying solely on a host-based firewall for access control is, for reasons which must now be obvious, admin incompetence. They are responsible for securing the containers. They didn't. Your services should be using authentication even if they are only bound to localhost.

The docker issue and original HN thread have countless instances of folks who have hit and been hacked by the exact same issue. There is clearly a major problem with the docker documentation, usage, etc. that is causing people to continually be taken by surprise with its iptables behavior. The blame game doesn't help solve the real problem that people are unknowingly putting services directly on the public internet w…

If anything here has a bug it's the mongo container image that launches without requiring authentication, not docker.

You're confusing the proximate cause for the root cause.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#132
post #9

It's not all that great blaming the victim. They clearly made the right moves with at least some of their configuration decisions and leaned on the underlying platform not being bonkers (but alas, it was: https://github.com/moby/moby/issues/4737 and https://github.com/moby/moby/issues/22054 ). Should they have hardened in all the other ways for defense in depth, e.g requiring authentication from localhost? Sure. Shou…

> They clearly made the right moves

The machine with DB had a public interface. No matter firewalls, this is just bad. DB machine should be in private subnets, preferably with no inet access at all, even via NAT.

Proof that it had public interface. TFA: "Docker helpfully inserted an allow rule into iptables, opening up MongoDB to the world"

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#133
Docker should at least add a warning message that they are editing IP table rules.

I remember experimenting with docker running on a VM, and being confused for a long time why my ufw rules didn't seem to be working, until I found out this was a known issue.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#134
post #9

It's not all that great blaming the victim. They clearly made the right moves with at least some of their configuration decisions and leaned on the underlying platform not being bonkers (but alas, it was: https://github.com/moby/moby/issues/4737 and https://github.com/moby/moby/issues/22054 ). Should they have hardened in all the other ways for defense in depth, e.g requiring authentication from localhost? Sure. Shou…

> Published ports > This creates a firewall rule which maps a container port to a port on the Docker host to the outside world. Source : https://docs.docker.com/config/containers/container-networki... Requiring authentication from localhost does not seem relevant to me, given that the creds would be stored somewhere, either in memory either in a file anyway, but exposing a port is not "binding on localhost". However…

> Indeed, that's how I found out Docker was using the DOCKER-USER iptables chain that you can customize

Only on IPv4...

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#135

Earlier quoted context omitted.

> Published ports > This creates a firewall rule which maps a container port to a port on the Docker host to the outside world. Source : https://docs.docker.com/config/containers/container-networki... Requiring authentication from localhost does not seem relevant to me, given that the creds would be stored somewhere, either in memory either in a file anyway, but exposing a port is not "binding on localhost". However…

> Another thing, instead of using exposing ports like that, the easiest is to use Docker-Compose, so that your containers of a stack have their own private shared network, then you won't have to publish ports to make your services communicate. That only works with local communication unless you use docker swarm. So byebye high availability. If you're going to make suggestions, at least think about them from a product…

Most productions are fine with 99.9% uptime, and a single server is fine for that, so HA is not necessary for most production out there.

For example, I'm running a governmental, nationwide service handling 300k req/day, with 1k admins working daily on the site, on a single server without anyone complaining, and without less than 99.9% uptime even though I reboot to upgrade the kernel twice a month, I did have to fine tune the system but that was actually pretty easy.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#137
post #109
post #60

Earlier quoted context omitted.

This is perhaps the best arguments I’ve seen for a separate firewall device even if it’s in the cloud (and just software) - something on your box running as root may bypass your rules just to help you.

Alternatively, running all your services as VMs also helps. Having root in a VM doesn't typically give you any rights on the hypervisor (at least not on eg Xen).

[deleted]

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#138
I think the real lesson here should be to use managed services whenever possible. There is just no excuse to manage your own infra on the VPS level for a typical web app in 2021. There are a myriad of PaaS providers where legacy VPS setups can be migrated to without too much pain.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#139
post #104
post #85

Earlier quoted context omitted.

> Staying away from it is still not the best strategy There's a whole generation of sysadmins that use docker so that they can stay away from foundational knowledge. We interview experienced devops who do not know/understand how to build basic packages from source (e.g. they don't understand the ./configure, make, make install chain) and who only have basic knowledge of the underlying operating system.

To be honest, I know how to type `./configure`, `make` and `make install`. And I know more about Gnu Make than I would admit in a job interview (for fear of someone expecting me to work with Make). But so far, life has been too short for me to waste my time on Gnu Autoconf. And I don't feel guilty about this, or like someone who doesn't know fundamentals. Autoconf is by all accounts a horrible system. Gnu Make ain't…

All software is fundamentally helping deal with abstractions as this.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#140
post #75
post #8

Earlier quoted context omitted.

I don't know why this is being downvoted. Multiple overlapping layers of security would have given newsblur a backup in case of accidental "footguns". Unauthenticated mongodb instances are a pretty common problem - it's why a "script kiddie" was so successful.

> I don't know why this is being downvoted. The core message ("using auth on MongoDB would have prevented this, it's always a good idea to add password auth just in case") is perfectly reasonable; we can all learn from this, and it's perfectly fine to point out such things. But the way it was phrased was absolutely not okay. People make mistakes all the time and they are not "incompetent". This is the classic "I am v…

Open MongoDB servers getting hacked is literally a meme at this point, there is no excuse for someone to configure a _production_ instance with zero authentication.
Post reply on HN