Live data from Hacker News

The ransomware surge

bbc.com

91–100 of 216 posts

Re: The ransomware surge

#91
post #80

Backups. I cannot emphasize enough the importance of backups. Take backups, verify your ability to restore from them, and keep them segregated from the rest of your infrastructure. It doesn't matter how inelegant and hacky your backup solution is, so long as you can restore from it. Any backup you can restore from is better than no backup. You might get a call from one of your application engineers shortly before bed…

A crucial point you missed: Ransomware gangs often destroy your backup infrastructure. So it's important to create pull-only backups or backups that cannot be deleted / overwritten.

I thought this was sufficiently implied with "keep them segregated from the rest of your infrastructure," but yes, you are correct. It is important to have a set of backups that can't be destroyed by the attackers. You might get lucky that your hand-rolled solution is so hacky that a ransomware gang overlooks it. But better to not rely on that luck.

In the past, this was achieved by having a set of tapes offsite. Today, one might configure Veeam to lie when issued a delete command, and instead send the data off to an Amazon glacier instance that requires different credentials to read, write, and delete.

Re: The ransomware surge

#92
post #62

Earlier quoted context omitted.

I would like to see rate-limiting built into OS's. Eg. an application is only allowed to touch 100 files per second or 1000 files per hour. When it reaches those limits, it gets paused and a popup asks the user if this application really should be doing X. Then at least ransomware can't run through stuff too quickly.

Indeed - I think Windows Defender dabbled in offering this as a feature. I at least recall seeing programs prevented from creating files in the Desktop or Documents folders. A rate limit, with group-policy controllable "automatic response" would perhaps help - you need the GPO integration though so that an IT admin can say "never allow file system rate limit to be exceeded". If you enforce a rate limit locally, and o…

My business involves working a lot with such situations, and frankly speaking, none of the above would help in the least bit.

Cost cutting is probably the biggest threat to most businesses. The mythos of the hyper-converged infrastructure, with the datastores and repositories for backups being hosted on the same physical device, are some sort of infection that just cannot be wrenched from people's heads.

IT Professionals (not managers, not hapless non-techies, actual persons with a cornucopia of certs and accolades on their linkedin) are in denial as to how to design a proper infrastructure to respond to ransomware. At this stage and for the foreseeable future, Ransomware is an inevitability; not "if" you get attacked, "when". But the countless number of conversations I've had where basically a group of people from the IT department theorycrafted a perfect defense only to get attacked because one of them clicked on a random excel document from a spoofed email is too high.

When clients ask me "what do we need to do to protect against ransomware" and I explain what airgapping means (tape, removable drive arrays), we're either ignored, or they say they accept and the clients just don't have the discipline to follow the required practices.

Modern IT prefers cargo-cult security, and IT professionals love their checklists from some organization, regardless of the fact that most of the checkboxes are useless to protect against ransomware. But the professional can eschew responsibility because "hey, I checked all the boxes."

Until technical professionals as a whole start to take security seriously and exhibit the discipline that is required for such security right now, Ransomware is going to continue to be prevalent. No amount of rate limiting from vendors will help, because users will simply just not use such versions, will disable such limits, will work around such limits, or any of dozens of workarounds to avoid it because such limits would be inconvenient (neverminding such limiting tooling probably will just be exploited)

We need discipline first, not tooling to try to correct for lack of discipline.

Re: The ransomware surge

#93
post #80

Backups. I cannot emphasize enough the importance of backups. Take backups, verify your ability to restore from them, and keep them segregated from the rest of your infrastructure. It doesn't matter how inelegant and hacky your backup solution is, so long as you can restore from it. Any backup you can restore from is better than no backup. You might get a call from one of your application engineers shortly before bed…

I think we should make every sysadmin watch Mr.Robot, at the least the first season. That may drive home the point: secure backups are important. ;)

Re: The ransomware surge

#94

Earlier quoted context omitted.

I would like to see rate-limiting built into OS's. Eg. an application is only allowed to touch 100 files per second or 1000 files per hour. When it reaches those limits, it gets paused and a popup asks the user if this application really should be doing X. Then at least ransomware can't run through stuff too quickly.

Behavioral heuristics are best learned in-situ; you need to know how the software is used with which data to correctly profile normal behavior. Some users and workloads hate sandboxes, though, and a 'Run as Adminstrator'-esque familiar-escape thus demanded by users will no doubt destroy its utility. Ultimately, someone must correctly articulate what the system is supposed to do, and this requires knowledge.

Okay, but these particular heuristics aren’t rocket science. Is a process rewriting 25% of my hard disk, and/or 10% of one of my backup drives? Time to send an alert to the user, and an IT admin if this isn’t a personal devices. There are very few legitimate use cases for that.

Re: The ransomware surge

#95

Earlier quoted context omitted.

>Why is it that even slightly old systems are so buggy that they are trivially hackable for a moderately well funded group? Because there's not enough money in making things bug-free from the start. It is possible (see seL4 and They Write the Right Stuff), but the incentives aren't there. Some kind of liability or minimum standard (similar to building code) would help, but I'm not sure just how it would be best imple…

That money would have to come from somewhere, though, and that's the pockets of consumers. Do they, in general, care enough? Is the security of software worth enough to them to spend the extra money? You don't just get what you pay for; you get what you're willing to pay for. And does the consumer have the expertise to evaluate the costliness of the threat or the security of the software? For that matter, I doubt the…

I for one would pay a lot just to not install updates all the time, if nothing else.

Re: The ransomware surge

#96
post #80

Backups. I cannot emphasize enough the importance of backups. Take backups, verify your ability to restore from them, and keep them segregated from the rest of your infrastructure. It doesn't matter how inelegant and hacky your backup solution is, so long as you can restore from it. Any backup you can restore from is better than no backup. You might get a call from one of your application engineers shortly before bed…

Why do you sweep?

https://ericalexander.org/post/devops-and-ransomware/

Re: The ransomware surge

#97
post #80

Backups. I cannot emphasize enough the importance of backups. Take backups, verify your ability to restore from them, and keep them segregated from the rest of your infrastructure. It doesn't matter how inelegant and hacky your backup solution is, so long as you can restore from it. Any backup you can restore from is better than no backup. You might get a call from one of your application engineers shortly before bed…

You know, I'm curious how large cloud providers handle this.

Obviously EC2 for AWS, but what about managed services?

A bad ransomware attack on a large cloud provider could cripple a significant portion of the internet.

Re: The ransomware surge

#98
post #80

Backups. I cannot emphasize enough the importance of backups. Take backups, verify your ability to restore from them, and keep them segregated from the rest of your infrastructure. It doesn't matter how inelegant and hacky your backup solution is, so long as you can restore from it. Any backup you can restore from is better than no backup. You might get a call from one of your application engineers shortly before bed…

I sometimes do infrastructure consulting. One of the first questions I ask is if they have at least one fully independent, full/incremental off-site backup that can't be corrupted from the main infrastructure, and if they have ever checked if they actually work and are restorable. I'm continuously surprised how often the answer turns out to be no after dinner digging, even in larger companies with otherwise well-run…

> No, the automatic 7 day RDS snapshots or turning on S3 versioning is not a sufficient backup. Neither is mirroring to a S3 Glacier bucket in the same org, or rsyncing to a a backup server in the same datacenter.

Why?

Re: The ransomware surge

#99

Earlier quoted context omitted.

A crucial point you missed: Ransomware gangs often destroy your backup infrastructure. So it's important to create pull-only backups or backups that cannot be deleted / overwritten.

> backups that cannot be deleted / overwritten That gets complex if your database contains PII. If a user asks for their account to be deleted...

Would it be sufficient that the delete accounts script be managed and merged at restoration time?

Re: The ransomware surge

#100
post #98

Earlier quoted context omitted.

I sometimes do infrastructure consulting. One of the first questions I ask is if they have at least one fully independent, full/incremental off-site backup that can't be corrupted from the main infrastructure, and if they have ever checked if they actually work and are restorable. I'm continuously surprised how often the answer turns out to be no after dinner digging, even in larger companies with otherwise well-run…

> No, the automatic 7 day RDS snapshots or turning on S3 versioning is not a sufficient backup. Neither is mirroring to a S3 Glacier bucket in the same org, or rsyncing to a a backup server in the same datacenter. Why?

Because they share the failure domain. In all of those cases, there's a single point of failure.
Post reply on HN