Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Update on IT Security Incident at UCSF

ucsf.edu

Post reply on HN
Print view View original

121–130 of 150 posts

Re: Update on IT Security Incident at UCSF

#121

Post by carapace »

ss2003 wrote: post #114
schoolornot wrote: post #28

CISO: https://cio.ucop.edu/spotlight-patrick-phelan-once-a-ucla-br... I can't think of any reason not to use a cloud hosted service for backup today. OneDrive, Dropbox, and Google Drive all sign BAAs and give you versioning amongst a million other security features. AWS even has offerings that let you take periodic snapshots of on-premise volumes. Point in time recoveries for the entire account would be nice add too…

"I'd fire everyone." Too late. They did that 3 years ago. Fired the IT staff and outsourced to India. A great decision that saved them tons of money. /s https://sanfrancisco.cbslocal.com/2017/02/28/ucsf-tech-worke... Fire the leadership from the top down. Every one approved outsourcing the IT staff.

From that link:

> This move will save UCSF $30 million over the next 5 years.

So they're ahead?

I'm being slightly snarky here, but also earnest. If they save 30 but lose 1.14 isn't it worth it?

Re: Update on IT Security Incident at UCSF

#122

Post by akiselev »

brippalcharrid wrote: post #57
mcny wrote: post #10

Earlier quoted context omitted.

If the data is worth paying a million dollar ransom to unlock, it is worth setting up proper backups. I for one am grateful to people who commit these crimes in which they "lock" data in place rather than sell it to the highest bidder. Proper data hygiene isn't brain surgery. There is zero excuse for this event. I don't blame the criminals. I blame the university system. Shame!

... "lock" data in place rather than sell it to the highest bidder. Why not both? And once the rightful owner of the data has paid a fat ransom, surely that's got to provide some kind of proof of its market value. The University did say that The attackers obtained some data as proof of their action so unless they're logging their outbound traffic, who's to say they didn't exfiltrate all of it? It's the kind of thing…

The data is worth that much to the university because they're critical to grant continuity - it'll be hard or impossible for their researchers to keep the money flowing without it. It's pretty much useless in everyone else's hands because those grants also depend on individual reputation and research history.

Re: Update on IT Security Incident at UCSF

#124

Post by tenuousemphasis »

robbrown451 wrote: post #20

Maybe there should be a law that if you pay a ransom, you are required to pay the same amount as a fine. Because paying these ransoms is funding the criminals.... how about you have to also fund law enforcement to combat those criminals? (also, this should reduce the amount that actually goes to the bad guys, since the amount of ransom would have greater downward pressure, i.e. if they'd probably not be able to colle…

Or since we already have to pay for law enforcement anyway, they could just do their jobs and catch the bad guys. But I think we all know at this point that "catching bad guys" is just a pretense, right?

Re: Update on IT Security Incident at UCSF

#125

Post by AnthonyMouse »

throw0101a wrote: post #107
AnthonyMouse wrote: post #95

Earlier quoted context omitted.

But then you lose the main advantages of using them to begin with. It becomes harder to use, it may not be able to do efficient differential backups or snapshots anymore or require you to do some complicated thing to make it work because their interface isn't meant to be used that way etc. And if it's actually important for the data to remain private, you then have to get the cryptography right, not think that your b…

> But then you lose the main advantages of using them to begin with. Poppycock. There's plenty of software that can do encrypted incrementals / differentials: Commvault, NetBackup, Veeam, tarsnap, Duplicity, ZFS snapshot send-recv,

You're asserting that these things are as easy to use as keeping all your junk on a shared Google Drive or whatever it is many companies are currently doing?

They all require you to have a level of technical competence equivalent to what would be needed to implement the backups yourself.

Re: Update on IT Security Incident at UCSF

#126

Post by AnthonyMouse »

vinay_ys wrote: post #118
nindalf wrote: post #81

Earlier quoted context omitted.

“No, we can’t use leading cloud providers to store this data. What if someone unauthorised looks at it?” This attitude leaves the victim paying millions in ransom. Look, Microsoft and Amazon probably have a better handle on security than your IT dept which is two people who also have to fix any issues like the WiFi not working or software not updating.

There is no reason to believe clouds don't have security issues or software bugs that cause inadvertent data loss. Just because the cloud provider is a big company doesn't mean the security is better. At the end of the day, they are a team of software engineers with all the usual people and org problems and can have usual human mistakes. But cloud solutions are better in practice due to totally non-technical reasons.…

> Public cloud solutions can help overcome these above shortcomings of in-house security+IT teams in a org politics friendly manner.

The problem there being that the choice of "public cloud" is subject to all the same forces, so then the vendor with the best corporate marketing team gets the business even if their security is crap.

Re: Update on IT Security Incident at UCSF

#127

Post by to11mtm »

carapace wrote: post #121
ss2003 wrote: post #114

Earlier quoted context omitted.

"I'd fire everyone." Too late. They did that 3 years ago. Fired the IT staff and outsourced to India. A great decision that saved them tons of money. /s https://sanfrancisco.cbslocal.com/2017/02/28/ucsf-tech-worke... Fire the leadership from the top down. Every one approved outsourcing the IT staff.

From that link: > This move will save UCSF $30 million over the next 5 years. So they're ahead? I'm being slightly snarky here, but also earnest. If they save 30 but lose 1.14 isn't it worth it?

Depends what the total cost was for this...

The 1.14M is just the ransom, doesn't account for how much money was wasted on other aspects of recovery and total downtime...

Re: Update on IT Security Incident at UCSF

#128

Post by throw0101a »

AnthonyMouse wrote: post #125
throw0101a wrote: post #107

Earlier quoted context omitted.

> But then you lose the main advantages of using them to begin with. Poppycock. There's plenty of software that can do encrypted incrementals / differentials: Commvault, NetBackup, Veeam, tarsnap, Duplicity, ZFS snapshot send-recv,

You're asserting that these things are as easy to use as keeping all your junk on a shared Google Drive or whatever it is many companies are currently doing? They all require you to have a level of technical competence equivalent to what would be needed to implement the backups yourself.

If you're talking about something that is worth paying US$ 1M in ransom, then paying someone a few (tens of) thousands in cost centre chargebacks upfront, and probably an ongoing annual fee, is cheap insurance if you want to go with the 'enterprise software'.

And if you want 'consumer software' that offers cloud / offsite encrypted back, then Backblaze offers it in a very clicky-clicky fashion that most folks can handle with a bit of hand-holding:

* https://help.backblaze.com/hc/en-us/articles/217664688-Can-y...

As does Arq, with a documented file format:

* https://www.arqbackup.com/arq_data_format.txt

Every cloud-capable backup system (for consumers and enterprises) does encryption nowadays, so talking about 'data access' in the cloud is a straw man.

Re: Update on IT Security Incident at UCSF

#129

Post by justinclift »

to11mtm wrote: post #127
carapace wrote: post #121

Earlier quoted context omitted.

From that link: > This move will save UCSF $30 million over the next 5 years. So they're ahead? I'm being slightly snarky here, but also earnest. If they save 30 but lose 1.14 isn't it worth it?

Depends what the total cost was for this... The 1.14M is just the ransom, doesn't account for how much money was wasted on other aspects of recovery and total downtime...

Substantial reputational damage will have to figure in there somewhere as well. ;)

Re: Update on IT Security Incident at UCSF

#130

Post by justinclift »

JeremyNT wrote: post #109
schoolornot wrote: post #28

CISO: https://cio.ucop.edu/spotlight-patrick-phelan-once-a-ucla-br... I can't think of any reason not to use a cloud hosted service for backup today. OneDrive, Dropbox, and Google Drive all sign BAAs and give you versioning amongst a million other security features. AWS even has offerings that let you take periodic snapshots of on-premise volumes. Point in time recoveries for the entire account would be nice add too…

These sound like servers used by researchers. I've worked with higher education research computing and you might be surprised at what you would find. Researchers may be generating or churning through countless TB of intermediary data, scratch files, etc. Often, the people who actually run the it infrastructure for researchers are... grad students. Sometimes they have grants for hardware and tight budgets, and paying…

> But maybe the new grad student didn't get the memo and developed his model in vim on the compute node.

Was trying to think through scenario's like that as well, but they don't really make sense.

If various staff members created a few weeks (or even months) worth of work on storage that isn't backed up, then the response from mgmt would generally be "Bad luck, you'll need to redo it".

But mgmt decided that work was worth paying US$1.1M+ for, and copping the reputation damage.

So, it's a weird mix of potential scenarios to actually get that happening. ;)

Maybe someone internal was actually responsible for that malware and is going for some kind of weird payday?

Post reply on HN