Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

newsblur.com

Post reply on HN
Print view View original

81–90 of 336 posts

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#81

Post by lmm »

vngzs wrote: post #75
conesus wrote: post #20

NewsBlur's founder here. I'll attempt to explain what's happening. This situation is more of a script kiddie than a hacker. I'm in the process of moving everything on NewsBlur over to Docker containers in prep for the big redesign launching next week. It's been a great year of maintenance and I've enjoyed the fruits of Ansible + Docker for NewsBlur's 5 database servers (PostgreSQL, MongoDB, Redis, Elasticsearch, and…

I think there are some good lessons here: 1. Even if you have one way to protect your database (e.g., firewall rules), you should have another. In this case, use a database password or (better) client TLS certificate to authenticate traffic. We're all human and we mess up. You should be designing systems that are graceful in response to your inevitable mistakes. 2. If you can afford another server/a hosting provider…

I agree with only the third of those. The other lessons I'd take would be:

1. Be cautious about trendy technologies that promise to make life easy - often they cut corners to do so, and often security is one of those corners

2. Use real authentication rather than network firewalling. Make your datastore TLS-only and require a valid client certificate to connect; that way it doesn't matter if it's exposed to the internet (indeed I'd argue you probably should expose every server to the internet - much like Chaos Monkey, it's counterintuitive but it forces you to build your systems with the right kind of resilience from day one)

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#82

Post by haimez »

bitexploder wrote: post #77
serial_dev wrote: post #70

Earlier quoted context omitted.

Why do you call them "script kiddie" and not a hacker? IMO it's still a hacker even if the attack is not very sophisticated or even if you made a big security mistake.

I think it’s fine. Script kiddie is a strict subset of “hacker” in the negative sense of the word “hacker”. It’s a way to convey to the reader the level of sophistication used in the attack by describing the hacker in this way.

Right, and there should be a sense of shame associated with being pwned by, for example, not setting a password on your public internet accessible (redis || postgres || mongo) instances. You didn’t get hacked, you let a child have their way with your application. Hence: script kiddie

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#83

Post by steffan »

haimez wrote: post #73
steffan wrote: post #62

Earlier quoted context omitted.

Defaults have long been changed to prevent listening on 0.0.0.0 by default; this has been the case since May of 2017, over 4 years ago. In order for this to occur, a user has to explicitly turn on listening on all interfaces. In conjunction with this, prudence would dictate that you enable authentication as well. In this case, it seems that reliance was placed on Docker to maintain iptables settings to disallow conne…

Thanks for the info, I haven’t payed attention to mongo since the last time I was personally burned by data corruption in the pre-2017 and web-scale (/dev/null db) era. Sounds there’s a mix of blame to go around- but it also sounds like exactly the MongoDB era I remember and hate. They know very well what their dockerhub image install looks like, and if they didn’t like it- they could request a change. Edit: they/(yo…

(Earlier in the thread I mentioned my affiliation - I am not commenting in any official capacity here)

Sorry about your prior experience. I think early versions assumed a systems knowledge that was at odds with the idea that anyone could just start using a database without any prior database knowledge.

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#84

Post by rbut »

conesus wrote: post #20

NewsBlur's founder here. I'll attempt to explain what's happening. This situation is more of a script kiddie than a hacker. I'm in the process of moving everything on NewsBlur over to Docker containers in prep for the big redesign launching next week. It's been a great year of maintenance and I've enjoyed the fruits of Ansible + Docker for NewsBlur's 5 database servers (PostgreSQL, MongoDB, Redis, Elasticsearch, and…

Use the new rootless mode and you won’t have issues with it inserting it’s rules above UFW.

You can then expose ports to a specific IP and use UFW to allow it.

Much cleaner than any UFW-docker hacks out there, and more secure.

https://docs.docker.com/engine/security/rootless/

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#85

Post by lmm »

zylent wrote: post #65
whalesalad wrote: post #56

This is one reason why most of my services are inaccessible from the public internet by design (on private subnets). Live and learn.

Zero reason to have database servers with an internet connection. These days you can ship in an entire environment with containers into borderline air gapped environments.

I'd argue that that's a backwards approach and actually what lead to this hack - building this system around the database servers not being publicly exposed, thinking the database servers weren't publicly exposed, and then when you accidentally publicly expose them (and you will sooner or later, a network is too big a boundary to protect all of it) it's a disaster.

It's better to build every server for public exposure from day 1 and treat all connections as potentially hostile, even if they're coming from the internal network.

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#86

Post by beermonster »

ralphington wrote: post #55
conesus wrote: post #20

NewsBlur's founder here. I'll attempt to explain what's happening. This situation is more of a script kiddie than a hacker. I'm in the process of moving everything on NewsBlur over to Docker containers in prep for the big redesign launching next week. It's been a great year of maintenance and I've enjoyed the fruits of Ansible + Docker for NewsBlur's 5 database servers (PostgreSQL, MongoDB, Redis, Elasticsearch, and…

What kind of database auth did you have? Wouldn't they have had to access config files or related in order to obtain your passwords, usernames, etc?

I think by default mongodb has no enabled access control, so there is no default user or password.

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#87

Post by qbasic_forever »

js4ever wrote: post #74
dawnerd wrote: post #22

Earlier quoted context omitted.

This actually got me a while ago but with redis and some script kiddy turning my dev server into a bitcoin miner. Anyone else running docker and using iptables really needs to read this https://docs.docker.com/network/iptables/

Redis can be exploited to run executables ???

It can execute Lua, I'm sure there's plenty of fun for hackers to have with this: https://redis.io/commands/eval

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#88

Post by mannerheim »

haimez wrote: post #61
xref wrote: post #50

Earlier quoted context omitted.

From their Twitter feed: mongodb is just RSS feed data, personal data is in postgres and wasn’t accessible to the script kiddy

And would you take that statement at face value from a company that just left their docker based mongo instance Internet public? It’s safe to assume that your info has already been leaked, but situations like this are why that assumption is safe.

I suppose we'll find out if/when the data will be leaked as the hacker claims?

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#89

Post by dawnerd »

js4ever wrote: post #74
dawnerd wrote: post #22

Earlier quoted context omitted.

This actually got me a while ago but with redis and some script kiddy turning my dev server into a bitcoin miner. Anyone else running docker and using iptables really needs to read this https://docs.docker.com/network/iptables/

Redis can be exploited to run executables ???

Yeah - kinda crazy theres no auth by default AND eval is allowed. Pretty trivial for someone to have it download a script and run it pretty much with free reign.

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#90

Post by jacquesm »

rank0 wrote: post #79
conesus wrote: post #20

NewsBlur's founder here. I'll attempt to explain what's happening. This situation is more of a script kiddie than a hacker. I'm in the process of moving everything on NewsBlur over to Docker containers in prep for the big redesign launching next week. It's been a great year of maintenance and I've enjoyed the fruits of Ansible + Docker for NewsBlur's 5 database servers (PostgreSQL, MongoDB, Redis, Elasticsearch, and…

Good luck! I appreciate your transparent account of the situation but what does it say about your company if your database got popped by a “script kiddie”?

It says more about Docker than anything else. This is an insane default setting, it's something that should have been fixed when it was first brought to their attention.

Computer security is hard enough without loaded footguns like these lying around.

Post reply on HN