Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

blog.newsblur.com

Post reply on HN
Print view View original

201–210 of 275 posts

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#201

Post by eru »

dmix wrote: post #139
eru wrote: post #104

Earlier quoted context omitted.

To be honest, I know how to type `./configure`, `make` and `make install`. And I know more about Gnu Make than I would admit in a job interview (for fear of someone expecting me to work with Make). But so far, life has been too short for me to waste my time on Gnu Autoconf. And I don't feel guilty about this, or like someone who doesn't know fundamentals. Autoconf is by all accounts a horrible system. Gnu Make ain't…

All software is fundamentally helping deal with abstractions as this.

Build systems in general can be ok, or even elegant and fun.

Autoconf is just really terrible.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#202

Post by ripperdoc »

oynqr wrote: post #115
naranha wrote: post #106

I think docker/podman with firewalld behaves the same way. I was astonished when I found it out. I think you can however bind to localhost exclusively by using the ip in the argument: docker run -p 127.0.0.1:5555:5555 ....

I am running a docker-compose setup with various services, only ever exposing ports so my reverse proxy on the same machine, which is not in a container, can hit them. The config explicitly specifies "127.0.0.1:external:internal". To my dismay, I can actually reach those external ports from the internet, even though I have some simple iptables rules to drop anything but 22, 80 and 443. I never really liked the whole…

Are you sure? I also have "127.0.0.1:external:internal" on some services and I cannot reach them with nc, but I can access those that are just configured as "externa:internal"

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#203

Post by yrro »

oynqr wrote: post #115
naranha wrote: post #106

I think docker/podman with firewalld behaves the same way. I was astonished when I found it out. I think you can however bind to localhost exclusively by using the ip in the argument: docker run -p 127.0.0.1:5555:5555 ....

I am running a docker-compose setup with various services, only ever exposing ports so my reverse proxy on the same machine, which is not in a container, can hit them. The config explicitly specifies "127.0.0.1:external:internal". To my dismay, I can actually reach those external ports from the internet, even though I have some simple iptables rules to drop anything but 22, 80 and 443. I never really liked the whole…

Sounds like your config is doing the right thing. It would be interesting to see what 'iptables -S -t nat' and 'iptables -S -t filter' say, as well as 'docker port CONTAINER_ID'.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#204

Post by arp242 »

mschuster91 wrote: post #149
eganist wrote: post #9

It's not all that great blaming the victim. They clearly made the right moves with at least some of their configuration decisions and leaned on the underlying platform not being bonkers (but alas, it was: https://github.com/moby/moby/issues/4737 and https://github.com/moby/moby/issues/22054 ). Should they have hardened in all the other ways for defense in depth, e.g requiring authentication from localhost? Sure. Shou…

> Yes, and the fact that docker seems to have persisted with the current state is the topic of discussion. It's clearly written in the docs: To expose a container’s internal port, an operator can start the container with the -P or -p flag. The exposed port is accessible on the host and the ports are available to any client that can reach the host. (from https://docs.docker.com/engine/reference/run/#expose-incomin...…

I don't think that's clear at all. If I set "bind_ip = *" in some application then it's also "available to any client that can reach the host", but the firewall is in front of that. I certainly wouldn't expect an application to frob with my firewall.

And as I understand it, this is very much an unintentional side-effect of ufw and Docker interacting – it's not Docker's the intention at all to override any iptable rules, just an unfortunate side-effect.

"It should be common knowledge by now" is very hand-wavy. I never used Docker much, and I could have been bitten by this. Where do I get this "common knowledge" from? Not everyone is a full-time sysadmin; some people just want to run a small service, read the basic documentation on various things etc., and set things up. Not everyone is super-invested in Docker (or any other tool for that matter) to have all the "common knowledge" that all the experts have. This is why defaults matter.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#205

Post by arp242 »

z3t4 wrote: post #166
nneonneo wrote: post #93

Network services really should have auth by default even if they only bind to localhost . There’s so many ways that a localhost service can become accessible to attackers. Any process with network access, no matter what user it’s running as, can access a localhost service - UNIX sockets on the other hand can be restricted by the usual user/group permissions. A localhost service can be exposed by e.g. an SSRF bug from…

Secure by default is best for those people who do not know every little config detail. Secure by default helps those that you try to make life easier for, eg. you open a port because you assume your user don't know how to... having security by default is the best option for those people. And by "those people" I mean everyone, including myself.

Making something secure "less secure" (according to needs) is usually easy. Making something insecure more secure can be complicated, do wrong, easy to forget, or just something people don't know about.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#206

Post by stuff4ben »

imtringued wrote: post #200
nuker wrote: post #132

Earlier quoted context omitted.

> They clearly made the right moves The machine with DB had a public interface. No matter firewalls, this is just bad. DB machine should be in private subnets, preferably with no inet access at all, even via NAT. Proof that it had public interface. TFA: "Docker helpfully inserted an allow rule into iptables, opening up MongoDB to the world"

>The machine with DB had a public interface. No matter firewalls, this is just bad. DB machine should be in private subnets, preferably with no inet access at all, even via NAT. This is the type of thinking that lead to the attack in the first place. No, you should not pretend that your database is secure because it is hidden deep behind dozens of layers of firewalls. You should assume that your database is always ac…

What he said about the DB being in a subnet wasn't incorrect, nor what you just said about setting a password and using certs. It's defense in layers and not "network security theater". You don't do just one of them.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#207

Post by papito »

simiones wrote: post #174

I see everyone discussing how much Docker is at fault, how much Mongo DB is at fault, how much NewsBlur should have had better settings, and I do agree to some extent. However, the much worse problem seems to be the fact that NewsBlur didn't test their network connectivity with something as basic as a port scan. This wasn't some complex attack based on some complex code injection jumping through legitimate ports or a…

As the after-action report concludes, which is painfully obvious as you read this - should have been in a VPC.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#208

Post by canadianfella »

eganist wrote: post #9

It's not all that great blaming the victim. They clearly made the right moves with at least some of their configuration decisions and leaned on the underlying platform not being bonkers (but alas, it was: https://github.com/moby/moby/issues/4737 and https://github.com/moby/moby/issues/22054 ). Should they have hardened in all the other ways for defense in depth, e.g requiring authentication from localhost? Sure. Shou…

Should you answer your own questions? No.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#209

Post by arp242 »

berkes wrote: post #147
jbverschoor wrote: post #54

Earlier quoted context omitted.

It’s exactly the same for redis, which means if used for jobs, you can insert arbitrary code.

All my async job code lives in a codebase and redis merely contains the payload and some ids. What system, framework or stack are you using that stores the code of the job in the database? I'm curious, because I can imagine it solves some issues, e.g. deploying new code while allowing old code to finish running and scheduled jobs.

I think what they mean is Redis' Lua scripting stuff with EVAL and such.

Also, if you have stuff like this in Redis, which will get picked up by a job runner:

  {"runjob": "foojob", "params": ["one", "two"]}
Depending on how your job runner interprets this, it can be a big problem if people can write to this. At the very least it'll be a DoS in most cases, at worst (SQL) database, shell access, or leaking of passwords.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#210

Post by mercora »

nuker wrote: post #159
diamondo25 wrote: post #154

Earlier quoted context omitted.

Even Google Cloud SQL just is a managed instance with a public IP (that you don't seem to be able to make private). Firewalls always help, and thats what Cloud SQL does: whitelist your ip when using the CLI.

What I do in AWS is having a VPC with 3 subnet types, public, private and restricted. Public ones have default route to inet, private to NAT, restricted - none. DBs go in restricted, load balancers in public and web/app servers in private subnets. Plus whitelist firewalling using Security Groups.

when i evaluated using a NAT setup using AWS NAT gateways it appeared to be quite expensive. we settled for strict VPC rules using only SecurityGroups in order to restrict communication. the instances did have public ip but nothing but intended services could connect to it anyways... i.e an instance running a HTTP API could only be connected to if initiated from the load balancer and itself could only connect to relevant services such as a RDS cluster...
Post reply on HN