Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mudge is a cyber activist, not a business executive

cybersect.substack.com

Post reply on HN
Print view View original

21–30 of 40 posts

Re: Mudge is a cyber activist, not a business executive

#21

Post by TedDoesntTalk »

akira2501 wrote: post #18
bink wrote: post #9

Some very good points made in this article but also so many cringey takes. The author is an engineer but compares the user to server ratio at Twitter to Netflix. Those services are so very different and have such vastly different hardware, software, and network requirements I don't even know where to begin. They dismiss Mudge's comments on lack of disk encryption by saying that anything less than end-to-end encryptio…

> There are all kinds of security controls to mitigate that threat that it sounds like Twitter isn't using. I mean.. they created a "God mode" for their service. It pretty much tells you where their entire approach to security of user data sits.

God mode, if compared to something like sudo or root access, is not a problem. It’s only a problem of the wrong people have access to it.

Re: Mudge is a cyber activist, not a business executive

#22

Post by quantified »

debacle wrote: post #6

The argument falls flat by paragraph 7 of this long rambling piece. > But there’s no objective evidence of this, only the subjective opinion of Mudge that Twitter wasn’t doing enough for cybersecurity. ... > specifically that they lied to investors and failed to live up to a 2014 FTC agreement to secure “private” data. So is it subjective, or objective? Is the 2014 FTC agreement subjective? And then he just starts pi…

Is lying to a board member really against the law? Lying to a regulator or investor, sure. Lying to your boss or to your employee, no, happens all the time.

Might be, I don't know. So I ask.

Re: Mudge is a cyber activist, not a business executive

#23

Post by ralph84 »

chx wrote: post #8

> In other words, in all likelihood, Twitter is ahead of the norm, ahead of the average, just not up to the same standard set by the leaders in tech. Really? It came out Twitter doesn't have a dev/staging environment. Source: https://twitter.com/lauren_feiner/status/1569695337190944775 in the same thread, "Twitter can't assure regulators that it's able to delete all data at a user's request because it's unsure where…

> It came out Twitter doesn't have a dev/staging environment.

And? Dev/staging is fine for enterprises that don't have to operate at Internet scale, think having a build script means they have CI/CD, and are doing waterfall development even if they call it agile. For truly Internet-scale services the cost/benefit of trying to build a staging environment that would have any usefulness doesn't pencil out. Canary deploys with well thought-out rollback plans works fine.

Re: Mudge is a cyber activist, not a business executive

#24

Post by williamsmj »

chx wrote: post #8

> In other words, in all likelihood, Twitter is ahead of the norm, ahead of the average, just not up to the same standard set by the leaders in tech. Really? It came out Twitter doesn't have a dev/staging environment. Source: https://twitter.com/lauren_feiner/status/1569695337190944775 in the same thread, "Twitter can't assure regulators that it's able to delete all data at a user's request because it's unsure where…

Re: test environments, of course Twitter has test environments, but if you're asking does Twitter have "staging.twitter.com" that matches the load, configuration and topolgy of prod, the answer is probably no because, like many companies at this scale they choose to handle moving stuff from dev to production with feature flags and incremental rollout techniques instead of the dev/staging/prod promotion process that is conventional wisdom at smaller scales. This choice increases velocity (especially if you have the platform team to support incremental rollouts and rollbacks) and building (or simulating) test environments that match prod would be insanely expensive, if it were even possible.

Like monorepos, this approach to ops might seem strange if you've only worked at small companies, and even then it's something reasonable people can disagree about. But I'm not convinced it's a useful discussion to have with an 88 year old senator. And it's not a sign that a company doesn't take engineering or security seriously, or that it has no idea what it's doing.

Re: resiliency, one of Twitter's datacenters is offline right now and twitter.com is up (https://www.cnn.com/2022/09/12/tech/twitter-data-center-cali.... So clearly the "small number" mudge is referring to is greater than one. Beyond that, yes, I'm absolutely sure that the loss of a "small number" of datacenters could take twitter offline and even result in permanant data loss (the data is in datacenters too!), because that statement is true of literally all tech companies. Twitter's expected uptime is a bunch of 9s. It's not 100.0%. The fact that it's not 100% is not evidence it "simply doesn't have a handle on its own infra any more".

And FWIW, resiliency appears to be pretty high for Twitter relative to the rest of the industry. They were down for 40 minutes early this summer, and haven't had an outage > 1 hour for 6 years (https://www.theguardian.com/technology/2022/jul/14/twitter-e...). Facebook (which I think most of us think of as pretty good at engineering, relative to the clown show most people seem to think Twitter is) was offline for 5.5 hours last year (https://en.wikipedia.org/wiki/2021_Facebook_outage).

Re: Mudge is a cyber activist, not a business executive

#25

Post by MattPalmer1086 »

I do not agree with most of the points made in this article. I didn't watch the entire testimony, but what I heard I found worrying.

For example, half the company had prod access to user accounts. And there was no way to find out who accessed what. This is not normal, it's a huge red flag.

The article claims that it should be perfectly expected that an executive should order someone to lie to the board about the risk the company is exposed to. Apparently this is Mudge's fault for not pushing back hard enough.

And then there's the rant against the "cyber security crusader", for whom nothing but perfect security is enough. Sounds like a perfectly annoying character, along with the architecture astronauts I guess. I've never met any of these stereotypes in the real world, although people do disagree on what is enough.

So it's either trolling us, or just not very good. Or both.

Re: Mudge is a cyber activist, not a business executive

#26

Post by MattPalmer1086 »

bink wrote: post #9

Some very good points made in this article but also so many cringey takes. The author is an engineer but compares the user to server ratio at Twitter to Netflix. Those services are so very different and have such vastly different hardware, software, and network requirements I don't even know where to begin. They dismiss Mudge's comments on lack of disk encryption by saying that anything less than end-to-end encryptio…

Yes, the argument for not having disk encryption is completely bogus.

Disk encryption protects against disposal or theft of physical storage. Companies like Twitter will be replacing disks all the time.

The argument that nothing less than an end-to-end encrypted system is worth doing is also flawed. No, don't lock the windows and keep leaving the key under the mat, since we don't live in a castle.

He attacks cyber activists for blocking business because security is not perfect. It seems he would rather have no security if it isn't perfect.

Re: Mudge is a cyber activist, not a business executive

#27

Post by 411111111111111 »

MattPalmer1086 wrote: post #25

I do not agree with most of the points made in this article. I didn't watch the entire testimony, but what I heard I found worrying. For example, half the company had prod access to user accounts. And there was no way to find out who accessed what. This is not normal, it's a huge red flag. The article claims that it should be perfectly expected that an executive should order someone to lie to the board about the risk…

Or you're just not as experienced as you seem to think you are.

> half the company had prod access to user accounts

That's pretty normal depending on the size of the company.

The chances of that being true is pretty high if it's a smallish upstart with > I've never met any of these stereotypes in the real world, although people do disagree on what is enough.

I've encountered several - they were all in the same company. It got so bad that multiple people quit and others were going to, until the people in question where "promoted" to a position where they weren't in the decision loop anymore.

Re: Mudge is a cyber activist, not a business executive

#28

Post by MattPalmer1086 »

411111111111111 wrote: post #27
MattPalmer1086 wrote: post #25

I do not agree with most of the points made in this article. I didn't watch the entire testimony, but what I heard I found worrying. For example, half the company had prod access to user accounts. And there was no way to find out who accessed what. This is not normal, it's a huge red flag. The article claims that it should be perfectly expected that an executive should order someone to lie to the board about the risk…

Or you're just not as experienced as you seem to think you are. > half the company had prod access to user accounts That's pretty normal depending on the size of the company. The chances of that being true is pretty high if it's a smallish upstart with > I've never met any of these stereotypes in the real world, although people do disagree on what is enough. I've encountered several - they were all in the same compan…

I've worked in multiple sectors for nearly 2 decades: government, energy, transport, retail, finance and software. So I've got a pretty good read on what is a normal level of access.

Very small companies and start ups I'd agree often don't have this kind of separation. But if they grow into one of the worlds biggest brands, I would not expect it to be run like a 50 man startup.

Re: Mudge is a cyber activist, not a business executive

#29

Post by LinuxBender »

MattPalmer1086 wrote: post #26
bink wrote: post #9

Some very good points made in this article but also so many cringey takes. The author is an engineer but compares the user to server ratio at Twitter to Netflix. Those services are so very different and have such vastly different hardware, software, and network requirements I don't even know where to begin. They dismiss Mudge's comments on lack of disk encryption by saying that anything less than end-to-end encryptio…

Yes, the argument for not having disk encryption is completely bogus. Disk encryption protects against disposal or theft of physical storage. Companies like Twitter will be replacing disks all the time. The argument that nothing less than an end-to-end encrypted system is worth doing is also flawed. No, don't lock the windows and keep leaving the key under the mat, since we don't live in a castle. He attacks cyber ac…

Adding to this, vendors like HP and Dell have a "keep your disk" option/policy so that you can have disk shredding vendors physically destroy the faulty disks. It costs more up front but is a high value option for some financial companies. I was in a company that had this policy with over 50k servers around the world and it gave the customers more confidence in our security and simplified part of our security audit. The shredding vendors recycle the material.

I should add that our customer data was encrypted at-rest but we shredded the disks to remove any doubt that on-demand debug logging potentially leaked sensitive data. New managers and directors thought this was a waste of money but as the sales people could attest to, this process paid for itself many times over.

Re: Mudge is a cyber activist, not a business executive

#30

Post by 411111111111111 »

MattPalmer1086 wrote: post #28
411111111111111 wrote: post #27

Earlier quoted context omitted.

Or you're just not as experienced as you seem to think you are. > half the company had prod access to user accounts That's pretty normal depending on the size of the company. The chances of that being true is pretty high if it's a smallish upstart with > I've never met any of these stereotypes in the real world, although people do disagree on what is enough. I've encountered several - they were all in the same compan…

I've worked in multiple sectors for nearly 2 decades: government, energy, transport, retail, finance and software. So I've got a pretty good read on what is a normal level of access. Very small companies and start ups I'd agree often don't have this kind of separation. But if they grow into one of the worlds biggest brands, I would not expect it to be run like a 50 man startup.

Eh, this kind of change usually needs something to go wrong before something is done. Lots of people get outraged when you take privileges from them, even if they probably never should've gotten them to begin with.

It's also easy to loose track of your own bias. There is a strong selection bias in employment which makes it look as if all workplaces are similar, simply because a company is more likely to hire you if you've worked in a similarly run one before.

Post reply on HN