Live data from Hacker News

A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

blog.newsblur.com

141–150 of 275 posts

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#141
I feel sorry for the victim, but it sounds like this whole setup is more or less careless. If your Docker configuration is the only thing that's stopping your database from being exposed - you should reconsider your approach to network and database security. At the very least there should be firewall protection at host level. Further is a standard procedure to put unsecure endpoints in a private VPC that's not accessible from the internet (and a bastion for administration). Relying solely on docker for something as important is bad even if you configure it correctly - it's still a single layer of software protecting you and it may have vulnerabilities of its own.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#142

I feel sorry for the victim, but it sounds like this whole setup is more or less careless. If your Docker configuration is the only thing that's stopping your database from being exposed - you should reconsider your approach to network and database security. At the very least there should be firewall protection at host level. Further is a standard procedure to put unsecure endpoints in a private VPC that's not access…

> At the very least there should be firewall protection at host level

If you read the Footgun at https://github.com/moby/moby/issues/4737, this is exactly what happens: someone sets up a conservative firewall, then Docker drills holes in it and opens itself up to the world, regardless of your firewall.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#143
post #79

Another issue that is not discussed, when I type “sudo ufw status verbose” I expect to see open ports in my system. It’s an issue that ufw doesn’t show some open ports (all relevant iptable rules).

ufw assumes you use only ufw to configure the firewall. If something else messes with iptables, all bets are off.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#144

I don't know how docker is to blame here; to me it's a bit unfair. It's more about the setting of the ufw. Docker relies on their own NAT table, and setting ufw accordingly requires a bit of setup. That said, having a good setting from hosting provider is quite easier ans safer.

Docker's default goes against best practices. Just look at all those opened issues on Github. Clearly people get caught by surprise and may suffer a security compromise as a result.

Docker is 100% to blame here and they need to change the default! Just put it in the release notes and get it over with.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#145
post #83

Earlier quoted context omitted.

That might be true, but regardless of docker (or other, similar solutions), shouldn't MongoDB have had auth protection?

MySQL/MariaDB have a completely open root account too... although default firewall rules should prevent public access too, unless Docker likes to punch that hole open too. Yes, root account password and access permissions should be changed upon a fresh install, but the real issue here is Docker's "helpfulness" by opening ports without explicit permission. That's absurd, and has no reasonable excuse.

I've ran a few webhosting companies back in the heyday of LAMP/LEMP.

This is a feature and a dangerous one.

Our provisioning had to carefully manage the firewall during installation, because MySQL would be open to the world *while provisioning*. Often this is mere seconds or less, but on a reasonable popular webhosting, this is enough time for automated tools to take over the server. If, during that provisioning some other tool would start punching holes in the firewall you carefully manage from provisining, I would be enraged. I dislike Docker for a lot of reasons[0], and this kind of behaviour is one of them.

So, yes: what MySQL or MongoDB do is dangerous and poor security practice. But what Docker does is worse: it breaks the countermeasures you take and rely on. IF all your layers of protection are removed by poor security practices of a tool that says "yea, but you need more than just this one layer" you still have no protection.

[0] yet I do use it a lot on development-machines to run the services needed on localhost and not have to manage all those with apt/snap/brew or whatever they come with.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#146

Earlier quoted context omitted.

> Another thing, instead of using exposing ports like that, the easiest is to use Docker-Compose, so that your containers of a stack have their own private shared network, then you won't have to publish ports to make your services communicate. That only works with local communication unless you use docker swarm. So byebye high availability. If you're going to make suggestions, at least think about them from a product…

Most productions are fine with 99.9% uptime, and a single server is fine for that, so HA is not necessary for most production out there. For example, I'm running a governmental, nationwide service handling 300k req/day, with 1k admins working daily on the site, on a single server without anyone complaining, and without less than 99.9% uptime even though I reboot to upgrade the kernel twice a month, I did have to fine…

That’s 10qps assuming working hours indeed you don’t need more than one server for that.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#147

Earlier quoted context omitted.

Totally agreed with this. MongoDB’s default security posture is ridiculous, and amazingly it used to be even worse.

It’s exactly the same for redis, which means if used for jobs, you can insert arbitrary code.

All my async job code lives in a codebase and redis merely contains the payload and some ids.

What system, framework or stack are you using that stores the code of the job in the database? I'm curious, because I can imagine it solves some issues, e.g. deploying new code while allowing old code to finish running and scheduled jobs.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#148
Along the same lines, I remember coming across a lot of bad information when learning about Docker & iptables. For example, this article about using fail2ban & docker will leave your system unprotected: https://chlee.co/how-to-secure-and-protect-nginx-on-linux-wi...

You have to tell fail2ban to use the DOCKER-USER chain when adding rules, otherwise none of the rules it adds do anything for docker.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#149
post #9

It's not all that great blaming the victim. They clearly made the right moves with at least some of their configuration decisions and leaned on the underlying platform not being bonkers (but alas, it was: https://github.com/moby/moby/issues/4737 and https://github.com/moby/moby/issues/22054 ). Should they have hardened in all the other ways for defense in depth, e.g requiring authentication from localhost? Sure. Shou…

> Yes, and the fact that docker seems to have persisted with the current state is the topic of discussion.

It's clearly written in the docs:

    To expose a container’s internal port, an operator can start the container with the -P or -p flag. The exposed port is accessible on the host and the ports are available to any client that can reach the host.
(from https://docs.docker.com/engine/reference/run/#expose-incomin...)

It should be common knowledge by now to either create a virtual interface / network on the host to publish ports on for usage by services that aren't running in Docker or (if you are in an environment where all services are in Docker) use --link between containers.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#150
post #104
post #85

Earlier quoted context omitted.

> Staying away from it is still not the best strategy There's a whole generation of sysadmins that use docker so that they can stay away from foundational knowledge. We interview experienced devops who do not know/understand how to build basic packages from source (e.g. they don't understand the ./configure, make, make install chain) and who only have basic knowledge of the underlying operating system.

To be honest, I know how to type `./configure`, `make` and `make install`. And I know more about Gnu Make than I would admit in a job interview (for fear of someone expecting me to work with Make). But so far, life has been too short for me to waste my time on Gnu Autoconf. And I don't feel guilty about this, or like someone who doesn't know fundamentals. Autoconf is by all accounts a horrible system. Gnu Make ain't…

I wholeheartedly agree, that I spend to much time of my live with GNU autoconf and make. But docker is on completely other level. After spending years and years of working with and around docker and speaking with others that do the same, I came to the conclusion that I will recommend docker only for one use-case in the future: Making easy examples to startup a system in one line.
Post reply on HN