Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

newsblur.com

Post reply on HN
Print view View original

311–320 of 336 posts

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#311

Post by lima »

unionpivo wrote: post #208
lmm wrote: post #201

Earlier quoted context omitted.

In reality time and attention are limited resources. Does network-level filtering have a good RoI relative to putting the same effort into improving security at the service level? IMO no (at least not until you've reached a very high level of service security where you're hitting severely diminishing returns) - network-level security is necessarily at least one of a) crude b) complex enough that it becomes an attack…

You are looking it as a network level filtering. I am looking at, make private network, and then you have to explicitly add an gateway (usually a load balancer) that can access it. In short don't design system where you need to filter, design the system, where you need to take explicite action, to make something public. This is easily, done on something like AWS.

An independent firewall accomplishes the same even if every server has a public IP.

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#312

Post by vngzs »

lmm wrote: post #136
vngzs wrote: post #95

Earlier quoted context omitted.

Yeah, there's the whole "zero trust" possibility which I didn't mention. If you do authentication/authorization really well, you can stop doing (2). For things like databases, I think it's better to treat them as if they _could_ be exposed to the open internet, without actually doing so. It's generally not the case that anyone on the internet needs to query your DBs. As described, you're _only_ relying on client TLS…

My view is that for most cases the cost/benefit of multiple layers doesn't stack up. Given a fixed amount of available time and effort, you'll generally get better results by focusing that effort on making one really good layer - e.g. putting active monitoring in place so that you detect when your single layer breaks (whether that's an attacker from outside the network being able to connect to inside the network or a…

I just want to say that I think this is absolutely a fair approach for many systems, but it sounds so radical against the backdrop of perimeter security that you're catching undue flak. You need to take a lot of care to build and design the "one layer to rule them all" in a way that's ridiculously sound, but this can actually produce a strong design if you pull it off. I didn't give this advice because I looked at where NewsBlur is today and figured they had some learning curves to get over before they could make the right tradeoffs to do it safely. If they got that much wrong with their DB, they might not know how to design a sane zero-trust network, or even how to make rational decisions in that world.

You see this "authN/authZ above all else" line of thinking in Google's security design [0], with their ubiquitous login wall. For their employees, that login wall has extra hardening - you can't do regular password resets, and you need to possess a physical security key (which acts basically as a scaled-down single-purpose HSM) and pass a suite of posture checks on the device (proportional to the sensitivity of the protected resource) to pass through it.

Then they put this login wall in front of everything, even internal services, and that tends to be OK because the system "fails closed," and the only way to access their protected resources is via physical safe rooms deep in the Google offices in which elevated privileges may be obtained.

Putting all your systems on the Internet forces you to get the incentive structure right, and that can be useful in companies where "private networks" can serve as justification to weaken security - if your login page is Internet-exposed, then you simply have no choice but to make it strong enough to withstand the chaos of the Internet. There's significant merit in that.

[0]: https://sre.google/books/building-secure-reliable-systems/

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#313

Post by js4ever »

bigiain wrote: post #174
js4ever wrote: post #74

Earlier quoted context omitted.

Redis can be exploited to run executables ???

From antirez the guy who wrote redis's blog: 'The Redis security model is: “it’s totally insecure to let untrusted clients access the system, please protect it from the outside world yourself”.' -- http://antirez.com/news/96 That blog post also helpfully shows how to write you own key into .ssh/authorized_keys to you can log in as the redis user over ssh. From there use your favourite lunar priv escalation bug to p0w…

Holy shit! thanks for the details

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#314

Post by floatingatoll »

amphorasource wrote: post #229
0-1 wrote: post #76

From a quick skim through https://github.com/samuelclay/newsblur for models extending mongo.Document, it looks like the following private customer data has been breached: - all story content from all private feeds - any uploaded OPML files, including URLs for any private RSS feeds - User’s twitter/facebook account info and access tokens, if the user had linked those services with their newsblur account - all data tha…

Samuel's nonchalant reply to this is highly disturbing to me. I'm a Newsblur customer and as far as I can tell, my feed data is in the hands of some hacker and he doesn't care at all. I am much less concerned about the service being restored, which seems to be all that he's worried about, and more about knowing who has my data. On top of that, I used to use his "forward newsletters to Newsblur" feature for a long tim…

“Nonchalant”? Sounded like someone in the midst of a wildfire crisis, not someone who isn’t concerned about the aftermath.

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#316

Post by danielpops »

vngzs wrote: post #75
conesus wrote: post #20

NewsBlur's founder here. I'll attempt to explain what's happening. This situation is more of a script kiddie than a hacker. I'm in the process of moving everything on NewsBlur over to Docker containers in prep for the big redesign launching next week. It's been a great year of maintenance and I've enjoyed the fruits of Ansible + Docker for NewsBlur's 5 database servers (PostgreSQL, MongoDB, Redis, Elasticsearch, and…

I think there are some good lessons here: 1. Even if you have one way to protect your database (e.g., firewall rules), you should have another. In this case, use a database password or (better) client TLS certificate to authenticate traffic. We're all human and we mess up. You should be designing systems that are graceful in response to your inevitable mistakes. 2. If you can afford another server/a hosting provider…

Another takeaway, IMO, use infrastructure-as-code to define the components in your infrastructure, so something like this critical firewall configuration won't be missed

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#317

Post by exikyut »

achillean wrote: post #143
zootboy wrote: post #54

Earlier quoted context omitted.

They do a monthly scan, with additional spot checks available on-demand: https://help.shodan.io/the-basics/on-demand-scanning

We actually scan on average once a week. I used that language to be ultra conservative but I'll need to change it. For the past 8+ years we've been doing weekly scans.

In case you see this, the most interesting question I think I could possibly ask is, what does the current real-world impact of IPv6 appear to be practically speaking?

Abstractly and intuitively, IPv6's massiveness would seem to put an end to the interesting closed loop of address space vs backhaul capacity that has developed around v4. I can't help but wonder though - with for example some providers leasing out ginormous blocks of address space according to fairly predictable patterns (and customers just using the first v6 address that pops out - if at all), this makes me wonder if it'll be possible to steer v6 scans using a mix of statistics, machine learning, and Perl if statements :).

The other thing I'm idly curious about is how you actually scan on a regular basis. Broadly speaking about long-term viability, I guess the TL;DR probably boils down to coordination and careful nurturing of reputation similar to what the large-scale email providers maintain. But from a technical perspective, I do wonder if/how much things like peering, and BGP, and noise-cancelling routing (if you will), etc, come into the picture - and how big the links are :D

I would be very happy to coincidentally discover writeups touching on these questions anytime. Thanks for reading :)

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#318

Post by qbasic_forever »

patrakov wrote: post #260
qbasic_forever wrote: post #33

Earlier quoted context omitted.

Oooph, good luck. And when you have time please make Docker aware that this well known foot-gun has finally done serious harm. They have known and ignored for years that iptables/ufw on Linux is totally broken and wide open when using Docker: https://github.com/moby/moby/issues/4737

It is 100% unfair to blame Docker for this foot-gun, especially because they cannot really do anything to fix it, because that's how the firewall works in the Linux kernel. Look: Podman has exactly the same issue when not running rootless. The root cause is that IP packets going to the containers are not going through the INPUT chain of the "filter" table (they go through FORWARD), while various firewall projects lik…

My point is that this is a well-known issue with absolutely catastrophic results when you get it wrong. It would cost the docker app almost nothing to do a quick check on docker run to see if you're on a machine with iptables & ufw enabled, and then violently complain or fail in a very obvious ways like:

*** WARNING YOUR FIREWALL ISN'T WORKING!! RUN AGAIN WITH --my_firewall_is_broken_and_I_accept_the_risks OPTION TO CONTINUE!!! SEE THIS FOR MORE INFO: ***

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#319

Post by Avamander »

indymike wrote: post #276
pmontra wrote: post #46

Earlier quoted context omitted.

> you face a heavy fine or arrest Heavy fine yes but not arrest AFAIK. Anyway this is a script programed to scary the target. Do you even store personal data inside that database?

There's something about this threat that really is awful. The legal extortion angle. We'll turn you over to the regulator if you don't give us money. Aside the fact they can take the money and package you to the regulator anyway, with complete impunity, it seems like the regulation needs to be revised in some way to take this very serious threat out of the hands of people who will abuse it.

This is just an another reason why user data should be dealt with very carefully, not a reason to nerf the legislation designed to dissuade people being careless.

Re: Hacker deleted all of NewsBlur’s Mongo data and is now holding the data hostage

#320

Post by zylent »

lmm wrote: post #85
zylent wrote: post #65

Earlier quoted context omitted.

Zero reason to have database servers with an internet connection. These days you can ship in an entire environment with containers into borderline air gapped environments.

I'd argue that that's a backwards approach and actually what lead to this hack - building this system around the database servers not being publicly exposed, thinking the database servers weren't publicly exposed, and then when you accidentally publicly expose them (and you will sooner or later, a network is too big a boundary to protect all of it) it's a disaster. It's better to build every server for public exposur…

Defense in depth / zero trust is definitely the way to go, however it's trivial to prevent a system from having internet access - for this hack to occur, the system had to be deployed with a public IP address directly assigned. NAT based internet access (IGW in AWS) or a private VPC with no IGW and no public on the instance is borderline standard in production cloud deployments these days.

Re: "You will sooner or later..." it's super easy to test for stuff like this with sentinel - I use this and scan dev / stage in my CI pipelines with rapid7 which will SCREAM about stuff like no DB password.

Post reply on HN