It's amazing to see everyone pointing out the poor infrastructure security being downvoted. I guess engineering doesn't matter anymore, just blame your tools.
A Docker footgun led to a vandal deleting NewsBlur's MongoDB database
151–160 of 275 posts
Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database
#152Earlier quoted context omitted.
That might be true, but regardless of docker (or other, similar solutions), shouldn't MongoDB have had auth protection?
MySQL/MariaDB have a completely open root account too... although default firewall rules should prevent public access too, unless Docker likes to punch that hole open too. Yes, root account password and access permissions should be changed upon a fresh install, but the real issue here is Docker's "helpfulness" by opening ports without explicit permission. That's absurd, and has no reasonable excuse.
> MySQL/MariaDB have a completely open root account too...
That's flat out wrong. You can't start MySQL/MariaDB docker images without either explicitly specifying a root password, have it generate a random one on the first start of the container, or explicitly allowing an empty password.
Regarding native installs of MySQL/MariaDB, the situation is a bit more murky, but at least the Ubuntu/Debian packages will ask you for a password during setup (and the bind-ip is 127.0.0.1 by default which means you'll have to mess around with conf.d files if you want to setup an externally-reachable server).
Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database
#153Earlier quoted context omitted.
Most productions are fine with 99.9% uptime, and a single server is fine for that, so HA is not necessary for most production out there. For example, I'm running a governmental, nationwide service handling 300k req/day, with 1k admins working daily on the site, on a single server without anyone complaining, and without less than 99.9% uptime even though I reboot to upgrade the kernel twice a month, I did have to fine…
That’s 10qps assuming working hours indeed you don’t need more than one server for that.
How many user facing projects actually got so far they needed more than one server? 1% of them?
Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database
#154It's not all that great blaming the victim. They clearly made the right moves with at least some of their configuration decisions and leaned on the underlying platform not being bonkers (but alas, it was: https://github.com/moby/moby/issues/4737 and https://github.com/moby/moby/issues/22054 ). Should they have hardened in all the other ways for defense in depth, e.g requiring authentication from localhost? Sure. Shou…
> They clearly made the right moves The machine with DB had a public interface. No matter firewalls, this is just bad. DB machine should be in private subnets, preferably with no inet access at all, even via NAT. Proof that it had public interface. TFA: "Docker helpfully inserted an allow rule into iptables, opening up MongoDB to the world"
Even Google Cloud SQL just is a managed instance with a public IP (that you don't seem to be able to make private).
Firewalls always help, and thats what Cloud SQL does: whitelist your ip when using the CLI.
Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database
#155Earlier quoted context omitted.
> Another thing, instead of using exposing ports like that, the easiest is to use Docker-Compose, so that your containers of a stack have their own private shared network, then you won't have to publish ports to make your services communicate. That only works with local communication unless you use docker swarm. So byebye high availability. If you're going to make suggestions, at least think about them from a product…
Most productions are fine with 99.9% uptime, and a single server is fine for that, so HA is not necessary for most production out there. For example, I'm running a governmental, nationwide service handling 300k req/day, with 1k admins working daily on the site, on a single server without anyone complaining, and without less than 99.9% uptime even though I reboot to upgrade the kernel twice a month, I did have to fine…
Quick calc: 99.9% of 365 means 8:45 downtime a year. That's not a lot, especially if hardware or power fails.
In my personal experience, given a vendor SLA of 4:00 hours, Dell manages to send you replacement HW on time, but HP has up to now failed to deliver in window every time. YMMV of course. RAID is the only thing keeping us online in these cases.
There is also the normal server maintenance. Rebooting a kernel twice a month, if the reboot takes 2 minutes, already swallows 48 minutes.
Now to be honest 99.9% business hours only is mostly doable if you serve only 1 continent and manage to update/reboot outside business hours. I have a non-critical debian public web server running that updates and reboots every day at 3AM. Mostly because nginx didn't consistently apply lets encrypt certs. So far no one has noticed or at least complained.
Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database
#156Kinda surprised so few people have any idea how docker networking works...
The OP didn't (I think) say but... were they running their containers with the --expose or --publish options? Without those, there shouldn't have been anything listening. And with them, well, the clue's in the option name...
That said, I've never liked the way docker implements its networking. Not only does it monkey with iptables (not too bad once you expect it) but, worse, it silently enables IP forwarding on all interfaces! Any system where it is installed is now a router; this should make anyone who uses a VPN to connect to work rather nervous...
I really wish docker used a different solution for exposing container ports by default, such as having the docker daemon proxy traffic. Yes, it wouldn't be as fast. But this wouldn't matter for the vast majority of users, who would benefit from having behaviour that matches other listening ports, doesn't bypass the firewall, etc. Have the iptables implementation available as an option that is there when you really need the performance--and know what you're doing!
Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database
#157I feel there are two issues to fix: 1. Docker doing this iptables change out of the box, and 2. MongoDB not having a password set out of the box The life of a developer (and solo dev) means you often have limited time you need to navigate a project and try and do your best to understand, deploy and use it -- this is just one of many tasks on your TODO today to get you closer to operating your product. I really wish t…
One of the main issue with security is that there's no visible difference between a well secured system and one that's open to any script kiddie. And you might install something or make a mistake in a config file, and suddenly your system is completely unsecure, and again there be will nothing obvious about it.
I think what's missing is an easy to use tool, installed by default - you run it and see a clear overview of your system security - what ports are opened, how is SSH accessible, how secure are the running services, etc. with plain ticks/crosses to show what's good or not. The kind of tool that a developer can install on their server and then check that at least the basics are right.
Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database
#158I feel sorry for the victim, but it sounds like this whole setup is more or less careless. If your Docker configuration is the only thing that's stopping your database from being exposed - you should reconsider your approach to network and database security. At the very least there should be firewall protection at host level. Further is a standard procedure to put unsecure endpoints in a private VPC that's not access…
> At the very least there should be firewall protection at host level If you read the Footgun at https://github.com/moby/moby/issues/4737 , this is exactly what happens: someone sets up a conservative firewall, then Docker drills holes in it and opens itself up to the world, regardless of your firewall.
Okay, good point, and that's exactly why you isolate your networks. You don't want to be one configuration option from being wide open.
Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database
#159Earlier quoted context omitted.
> They clearly made the right moves The machine with DB had a public interface. No matter firewalls, this is just bad. DB machine should be in private subnets, preferably with no inet access at all, even via NAT. Proof that it had public interface. TFA: "Docker helpfully inserted an allow rule into iptables, opening up MongoDB to the world"
Even Google Cloud SQL just is a managed instance with a public IP (that you don't seem to be able to make private). Firewalls always help, and thats what Cloud SQL does: whitelist your ip when using the CLI.
What I do in AWS is having a VPC with 3 subnet types, public, private and restricted. Public ones have default route to inet, private to NAT, restricted - none. DBs go in restricted, load balancers in public and web/app servers in private subnets. Plus whitelist firewalling using Security Groups.
Re: A Docker footgun led to a vandal deleting NewsBlur's MongoDB database
#160The biggest issue with docker is the false sense of security it seems to give a lot of engineers into thinking they know infra when they really don't. I stay away from this because I don't understand it fundamentally, and now this proves it's better to not think these new technologies are your friend unless you actually know what you're doing (which apparently most don't).
I'd argue an equally big problem here is this setting is that engineers have been taught to think in a perimeter security mindset.
No: good security is layered aka "defense-in-depth".
docker breaches the localhost VS external port security model (and others)