Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

blog.newsblur.com

Post reply on HN
Print view View original

241–250 of 275 posts

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#241

Post by mschuster91 »

0xbadcafebee wrote: post #184
mschuster91 wrote: post #152

Earlier quoted context omitted.

> MySQL/MariaDB have a completely open root account too... That's flat out wrong. You can't start MySQL/MariaDB docker images without either explicitly specifying a root password, have it generate a random one on the first start of the container, or explicitly allowing an empty password. Regarding native installs of MySQL/MariaDB, the situation is a bit more murky, but at least the Ubuntu/Debian packages will ask you…

You can totally start a new mongodb container and connect other containers to it without ever using auth. I'm doing it right now with docker-compose for local development.

I and the post I replied to was talking about MySQL. MongoDB is an insecure pile of cow poo that has been so often implicated in hacks and data thefts that I don't get why a) anyone is STILL using it and b) the containers STILL don't have a security-first default.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#242

Post by mschuster91 »

arp242 wrote: post #204
mschuster91 wrote: post #149

Earlier quoted context omitted.

> Yes, and the fact that docker seems to have persisted with the current state is the topic of discussion. It's clearly written in the docs: To expose a container’s internal port, an operator can start the container with the -P or -p flag. The exposed port is accessible on the host and the ports are available to any client that can reach the host. (from https://docs.docker.com/engine/reference/run/#expose-incomin...…

I don't think that's clear at all. If I set "bind_ip = *" in some application then it's also "available to any client that can reach the host", but the firewall is in front of that. I certainly wouldn't expect an application to frob with my firewall. And as I understand it, this is very much an unintentional side-effect of ufw and Docker interacting – it's not Docker's the intention at all to override any iptable rul…

> "It should be common knowledge by now" is very hand-wavy.

It is written explicitly in the manual that any port you choose to publish is reachable by all machines that can reach the host.

Just how more explicit does this warning have to be?!

And anyways: a simple "netstat -lnp" would have shown the docker-proxy process and world-wide reachability.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#243

Post by wpietri »

mschuster91 wrote: post #242
arp242 wrote: post #204

Earlier quoted context omitted.

I don't think that's clear at all. If I set "bind_ip = *" in some application then it's also "available to any client that can reach the host", but the firewall is in front of that. I certainly wouldn't expect an application to frob with my firewall. And as I understand it, this is very much an unintentional side-effect of ufw and Docker interacting – it's not Docker's the intention at all to override any iptable rul…

> "It should be common knowledge by now" is very hand-wavy. It is written explicitly in the manual that any port you choose to publish is reachable by all machines that can reach the host. Just how more explicit does this warning have to be?! And anyways: a simple "netstat -lnp" would have shown the docker-proxy process and world-wide reachability.

What would you estimate as the total word count of the Docker documentation? The page you linked to alone is about 10k words, and there are a lot of pages. At a guess, we're looking at something the length of notoriously long novels like War and Peace. What percentage of the documentation had you read and internalized before you first put Docker into production?

It's very easy to come along after something blows up, find something a person could have done differently, and blame that person, treating them as stupid or negligent. It's easy, satisfying, and often status-enhancing. It's also going to make the world less safe, because it prevents us from solving the actual problems.

Those serious about reducing failures should read Dekker's "Field Guide to Understanding 'Human Error'": https://www.amazon.com/Field-Guide-Understanding-Human-Error...

It comes out of the world of airplane accident investigations. But so much of it is applicable to software. The first chapter alone, the one contrasting old and new views, can be enough for a lot of people. It's available for free via "look inside this book" or the Kindle sample.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#244

Post by mschuster91 »

wpietri wrote: post #243
mschuster91 wrote: post #242

Earlier quoted context omitted.

> "It should be common knowledge by now" is very hand-wavy. It is written explicitly in the manual that any port you choose to publish is reachable by all machines that can reach the host. Just how more explicit does this warning have to be?! And anyways: a simple "netstat -lnp" would have shown the docker-proxy process and world-wide reachability.

What would you estimate as the total word count of the Docker documentation? The page you linked to alone is about 10k words, and there are a lot of pages. At a guess, we're looking at something the length of notoriously long novels like War and Peace. What percentage of the documentation had you read and internalized before you first put Docker into production? It's very easy to come along after something blows up,…

> What percentage of the documentation had you read and internalized before you first put Docker into production?

Admittedly, not as much (and since I've got a couple of years experience, back then the docs were bananaware), which is why I ran into the same issue - but one thing I always do is set up staging environments and check if at least the basic expectations (=can't reach an internal service from the Internet, but can reach public services) are working. Aka: a portscan.

A basic check would have prevented so many carelessly-configured services. Generally, it doesn't hurt anyone to run a nmap scan as part of the regular monitoring - if the line count doesn't match the previous one, raise an alarm.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#245

Post by 2ion »

eganist wrote: post #9

It's not all that great blaming the victim. They clearly made the right moves with at least some of their configuration decisions and leaned on the underlying platform not being bonkers (but alas, it was: https://github.com/moby/moby/issues/4737 and https://github.com/moby/moby/issues/22054 ). Should they have hardened in all the other ways for defense in depth, e.g requiring authentication from localhost? Sure. Shou…

> It's not all that great blaming the victim

* Run a SaaS that costs money

* Do not perform the migration in prod replica setup with test data and monitor for, observe and fix any oddities

* Cause a 100% data breach by not following a sound change management procedure

Just because this individual is being blamed and not a Ltd. or Corp. doesn't mean that this is how a quality SaaS should be run. I know that more it's-just-a-side-gig SaaS wing it more than they do not but this is exactly what happens when you haven't got good ops. Rather than random error, this is a consequence from a systematic error.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#246

Post by thatwasunusual »

Alupis wrote: post #83
thatwasunusual wrote: post #41

Earlier quoted context omitted.

That might be true, but regardless of docker (or other, similar solutions), shouldn't MongoDB have had auth protection?

MySQL/MariaDB have a completely open root account too... although default firewall rules should prevent public access too, unless Docker likes to punch that hole open too. Yes, root account password and access permissions should be changed upon a fresh install, but the real issue here is Docker's "helpfulness" by opening ports without explicit permission. That's absurd, and has no reasonable excuse.

> MySQL/MariaDB have a completely open root account too...

My point was: why didn't the author of the article enable the use of authorisation? To rely on only a firewall is just stupid.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#247

Post by marcosdumay »

tremon wrote: post #218
superasn wrote: post #113

Earlier quoted context omitted.

Yes it links to this comment: https://news.ycombinator.com/item?id=17393292#:~:text=Genera... . The funny thing is there are definitions for it in wikionary and urbandictionary yet it is picking the HN one.

The other funny thing is that that definition is also from a thread involving MongoDB.

Well, the footgun on this thread is on Docker...

There exists an entire class of software that is best avoided. Looks like their usage is highly correlated. As a parallel, I was just saying in a work meeting that the first step towards a good IT environment is not adding some crazy non-causal component like our F5 middlewhare, while waiting for a command to end on our Oracle database. That other class is highly correlated too.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#248

Post by wpietri »

mschuster91 wrote: post #244
wpietri wrote: post #243

Earlier quoted context omitted.

What would you estimate as the total word count of the Docker documentation? The page you linked to alone is about 10k words, and there are a lot of pages. At a guess, we're looking at something the length of notoriously long novels like War and Peace. What percentage of the documentation had you read and internalized before you first put Docker into production? It's very easy to come along after something blows up,…

> What percentage of the documentation had you read and internalized before you first put Docker into production? Admittedly, not as much (and since I've got a couple of years experience, back then the docs were bananaware), which is why I ran into the same issue - but one thing I always do is set up staging environments and check if at least the basic expectations (=can't reach an internal service from the Internet,…

Oh, is that also in the docs?

My point is that this is the same behavior: looking for a reason why the person is wrong. That is a behavior that makes things worse because it takes the focus away from systemic improvements that make us safer. Please don't do it.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#249

Post by simiones »

tbarbugli wrote: post #238
simiones wrote: post #174

I see everyone discussing how much Docker is at fault, how much Mongo DB is at fault, how much NewsBlur should have had better settings, and I do agree to some extent. However, the much worse problem seems to be the fact that NewsBlur didn't test their network connectivity with something as basic as a port scan. This wasn't some complex attack based on some complex code injection jumping through legitimate ports or a…

yes, the author made a junior dev mistake and he is trying to blame [tool name here]. Making mistakes like this sucks but passing the blame makes it even worse.

Making mistakes is not the problem - everyone makes mistakes. But you need testing to catch your mistakes - this is a process failure.

Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database

#250

Post by Alupis »

mschuster91 wrote: post #241
0xbadcafebee wrote: post #184

Earlier quoted context omitted.

You can totally start a new mongodb container and connect other containers to it without ever using auth. I'm doing it right now with docker-compose for local development.

I and the post I replied to was talking about MySQL. MongoDB is an insecure pile of cow poo that has been so often implicated in hacks and data thefts that I don't get why a) anyone is STILL using it and b) the containers STILL don't have a security-first default.

I think this still misses the point.

It's Docker's responsibility to not open firewall ports unless explicitly asked to do so.

Opening random ports without A) Telling the developer first, B) Requesting permission to do so, C) Explaining why Docker wants to do so and D) Providing the ability to "opt-out" of this insecure-by-default configuration - is very bad.

It doesn't matter how secure or insecure the software inside the container is... it shouldn't be allowed to communicate with the public unless the developer specifically requests it.

Post reply on HN