Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

newsblur.com

Post reply on HN
Print view View original

251–260 of 336 posts

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#251

Post by [deleted] »

tobiasbischoff wrote: post #164
conesus wrote: post #34

Earlier quoted context omitted.

In case anybody's interested, here's what the "hack" looks like: nbset:PRIMARY> show dbs READ__ME_TO_RECOVER_YOUR_DATA 0.000GB admin 0.000GB local 16.471GB newsblur 0.718GB nbset:PRIMARY> use READ__ME_TO_RECOVER_YOUR_DATA switched to db READ__ME_TO_RECOVER_YOUR_DATA nbset:PRIMARY> show collections README system.profile nbset:PRIMARY> db.README.find() { "_id" : ObjectId("60d3e112ac48d82047aab95d"), "content" : "All yo…

0.03 BTC?! Someone is doing this for a lousy thousand dollar? Unbelievable.

[deleted]

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#252

Post by dddddddddddd »

In production never ever use ufw or firewalld. Learn a few rules in iptables and that all or create a simple script. The problem with ufw or firewalld(not only with this 2) they add so many rules in iptables, that a human can do very easy mistakes or the rules are not on your control.

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#253

Post by cunthorpe »

jacquesm wrote: post #119

The big takeaway here is that you simply should not use any technology on an internet facing server that you are not 100% committed to. There is no such thing as a 'casual' SaaS, you either dedicate the time and the effort required to intimately familiarize yourself with the crappy little details of all the tools you are using as well as their caveats or you will end up getting hacked. That said, software suppliers h…

> The likes of AWS already make it a bit harder to expose an insecure server

Has this changed recently? S3 was a huge part of data leaks a few years ago, and that's basically a managed server.

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#254

Post by habosa »

conesus wrote: post #20

NewsBlur's founder here. I'll attempt to explain what's happening. This situation is more of a script kiddie than a hacker. I'm in the process of moving everything on NewsBlur over to Docker containers in prep for the big redesign launching next week. It's been a great year of maintenance and I've enjoyed the fruits of Ansible + Docker for NewsBlur's 5 database servers (PostgreSQL, MongoDB, Redis, Elasticsearch, and…

Just want to say that I love Newsblur have I have been using it for free for many years. Reading this today made me stop and think about how hard you must work on it, so today I will donate/subscribe!

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#255

Post by Aeolun »

gnyman wrote: post #129
qbasic_forever wrote: post #33

Earlier quoted context omitted.

Oooph, good luck. And when you have time please make Docker aware that this well known foot-gun has finally done serious harm. They have known and ignored for years that iptables/ufw on Linux is totally broken and wide open when using Docker: https://github.com/moby/moby/issues/4737

Glad someone else highlighted this old ticket. I bet this, in combination with the extremely irresponsible solution to ship mongodb without auth as default has caused countless of data leaks and destruction events. We just haven't heard about most of them. Elastic provides the same foot-gun. Last year someone deleted almost 4000 open mongodb and elastic databases in what was called the Meow attack [1]. In my opinion…

I’d argue that anyone that uses Mongo given those defaults is equally culpable

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#256

Post by slver »

wolverine876 wrote: post #4

These are crimes. Where is law enforcement? I am not sure why the public still gives them a pass and treats computer crimes differently. With the resources of the federal government, it shouldn't be hard to find and take down the criminals. Think of how easily the criminals exploit their victims - it is just as hard for the criminals to play defense as it is for everyone else.

> These are crimes. Where is law enforcement? I am not sure why the public still gives them a pass and treats computer crimes differently.

No one gives them a pass. Law enforcement, however, requires a method of enforcement. You have anonymous emails request Bitcoin to anonymous wallet, what do you "enforce" your laws on?

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#257

Post by amphorasource »

corobo wrote: post #250
amphorasource wrote: post #233

Earlier quoted context omitted.

This is horrific. So the hacker is claiming to have a copy of our data. 0.03 BTC is less than $1000. Regardless of you being able to restore from backups, I assume you're paying the ransom to hopefully avoid the leak, right?

Paying a ransom marks you as will-pay. The price will keep rising till they find your limit. The data is already leaked, let your users know what was leaked and recover from there. See also: 80% of orgs that paid the ransom were hit again https://news.ycombinator.com/item?id=27552611

You misunderstand.

I paid Samuel and entrusted him with my data. Not too much, but enough for it to matter. When faced with a massive leak like this, he downplays everything, calls the hacker a "script kiddie" and calls this "good practice for what will be the first of many sleepless nights", looking at it only from a "service disruption" perspective.

So far we've gotten no indication of what's been leaked, if it contains deleted feeds, or what he's doing to prevent the data from being leaked by the hacker, if anything. He's been solely focused on restoring the service and ignoring the leak. Compared to not having access to an RSS reader for any random period of time, the leak is orders of magnitude more serious to me and I'd wager to most of Newsblur customers.

I honestly don't care if paying a ransom or interacting with the hacker makes him more likely to be targeted in the future, his duty towards his customers was to keep their private data private and not only he failed at that, but he doesn't even seem to register that as his main priority. As far as I'm aware, if he allows the data to leak publicly, then there's no "recovering from there", he's not getting any more of my money.

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#258

Post by mafro »

jbrot wrote: post #155
qbasic_forever wrote: post #150

Earlier quoted context omitted.

Yeah, it seems like there's a weird inbetween phase when projects go from "awesome tool used and loved by some core people" to "this is the new normal, it's everywhere" where these issues get lost. I could see back in 2014 moby not really feeling like the quirks of ufw & iptables were its problem. But now in 2021 with how many millions of times docker run is used per day on machines all across the internet... it's ju…

I know the CICD code I wrote and manage at work launches approximately a million docker containers a day, so I suspect the total number of docker containers used per day is well into the billions.

Do tell about where you would need a million docker images for a CICD pipeline.. It's either many apps, or some very complicated pipelines

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#259

Post by corobo »

amphorasource wrote: post #257
corobo wrote: post #250

Earlier quoted context omitted.

Paying a ransom marks you as will-pay. The price will keep rising till they find your limit. The data is already leaked, let your users know what was leaked and recover from there. See also: 80% of orgs that paid the ransom were hit again https://news.ycombinator.com/item?id=27552611

You misunderstand. I paid Samuel and entrusted him with my data. Not too much, but enough for it to matter. When faced with a massive leak like this, he downplays everything, calls the hacker a "script kiddie" and calls this "good practice for what will be the first of many sleepless nights", looking at it only from a "service disruption" perspective. So far we've gotten no indication of what's been leaked, if it con…

I'm on the same side of the argument as you and indeed I believe I feel as strongly about it as you. Especially in regards to brushing it off, calling them script kiddies[1], generally being "well aw shucks aren't I great for not deleting my copy of the data, I'm so great"[2] about the whole thing grinds my gears too.

I'm saying whoever is ransoming the data already has the data, the data is out of Newsblur's control, therefore the data is already leaked.

The data leak is past tense. It has already happened, not will happen. No amount of money will undo that. If that means they've lost you as a customer, that's how it is.

What we now need to know is what data was leaked?

[1]: which to be fair Newsblur, they are, but if a script kiddie hacks you using something so basic as a missing firewall rule.. Arguably not knowing Docker's quirks but using it anyway is the same damn thing as what script kiddies do. Sys kiddie if you will.

[2]: Why is that cause for celebration? Do you not have backups?

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#260

Post by patrakov »

qbasic_forever wrote: post #33
conesus wrote: post #20

NewsBlur's founder here. I'll attempt to explain what's happening. This situation is more of a script kiddie than a hacker. I'm in the process of moving everything on NewsBlur over to Docker containers in prep for the big redesign launching next week. It's been a great year of maintenance and I've enjoyed the fruits of Ansible + Docker for NewsBlur's 5 database servers (PostgreSQL, MongoDB, Redis, Elasticsearch, and…

Oooph, good luck. And when you have time please make Docker aware that this well known foot-gun has finally done serious harm. They have known and ignored for years that iptables/ufw on Linux is totally broken and wide open when using Docker: https://github.com/moby/moby/issues/4737

It is 100% unfair to blame Docker for this foot-gun, especially because they cannot really do anything to fix it, because that's how the firewall works in the Linux kernel. Look: Podman has exactly the same issue when not running rootless.

The root cause is that IP packets going to the containers are not going through the INPUT chain of the "filter" table (they go through FORWARD), while various firewall projects like ufw or firewalld only provide convenient management of the INPUT chain, and, worse, don't event provide a good way to express the notion of "packets going to external port 27017 and then forwarded" (i.e. the equivalent of the --ctorigdst option provided by raw iptables).

Here are some options for container engines:

a) Use only a userspace proxy (which is what Docker does when configured with {"iptables": false}). This way, there is no packet forwarding, so packets go through the INPUT chain, just as expected by high-level firewall packages like ufw or firewalld. The major downside (which makes this method useless e.g. for containerized mail servers) is that the information about the source IP is lost, and there is no good way to fix this. I guess TPROXY can help here, but nobody uses it.

b) Use slirp4netns (which is what Podman does when running rootless). It has a really nice mode (available via "podman run --network=slirp4netns:port_handler=slirp4netns") where on the host side, there is only a userspace process listening (so that packets go through the INPUT chain), but inside the container, packets going out of tap0 have the correct source IP. The downside (actually a Podman limitation) is that you can't set up multiple containers communicating over internal IPs.

I would say that I am not really in favor of options (a) and (b) because of the overhead created by the proxy or by slirp4netns. If port forwarding can be done in the kernel (and it can, the only missing piece is --ctorigdst in high-level firewalls), it should be done in the kernel.

c) Document the situation better (e.g. I don't see the --ctorigdst option mentioned at all on https://docs.docker.com/network/iptables/), shift the blame to firewall authors so that they start creating a duplicate of each INPUT rule also in the FORWARD chain with --ctorigdst added as necessary.

d) Provide usable primitives (similar to Network Policies in Kubernetes) to control the firewall for containers, so that ufw or firewalld is not needed.

Post reply on HN