Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta quickly detects silent data corruptions at scale

engineering.fb.com

Post reply on HN
Print view View original

91–100 of 104 posts

Re: Meta quickly detects silent data corruptions at scale

#91

Post by ignoramous »

someotherperson wrote: post #47
chaorace wrote: post #26

Earlier quoted context omitted.

I've noticed this is quite popular among the kings of cargo-cult security: banking websites. I can only hope the proliferation of VPN-gating is more contained compared to the recent (banking-led) upswing in Android root-checks. This type of security theater can be easily bypassed by any determined attacker and thus only serves to deter honest users.

> This type of security theater can be easily bypassed by any determined attacker and thus only serves to deter honest users. To play devil's advocate, the large amount of attackers aren't really determined. They're just fishing for easy targets. If you check the logs on a VPS you'll see an endless stream of people trying to exploit things like Wordpress 24/7 on your brand new VPS that has nothing but a html landing…

> To play devil's advocate, the large amount of attackers aren't really determined. They're just fishing for easy targets.

The rise of the copy-paste attacker: https://en.wikipedia.org/wiki/Script_kiddie

Re: Meta quickly detects silent data corruptions at scale

#92

Post by jamesfinlayson »

gitfan86 wrote: post #90
fleddr wrote: post #88

Earlier quoted context omitted.

Exactly. And that doesn't even take into account higher res photos and 4K video. I remember the staggering statistic where just Instagram sees 100 million photos added per day, every day. And that was years ago.

Instagram does not guarantee photo quality. They can resize photos anytime they want. Eventually they turn 10 year old photos that have not been viewed in 9 years into thumbnails or just delete them.

I looked back at some photos from Christmas in Facebook Messenger and they looked noticeably degraded in quality.

Re: Meta quickly detects silent data corruptions at scale

#93

Post by 20395858439 »

PragmaticPulp wrote: post #48
Hnrobert42 wrote: post #12

Interestingly, this site fails ungracefully (HTTP error code 500) when I try to visit from NordVPN, even after cycling through a few IP addresses. I’m noticing more and more sites block all VPN track. I get why, but it’s not good.

I've been on the other side of this (threat analysis, not Facebook). Known VPN-associated IP addresses were far more likely to be associated with abuse than average. Not just a little bit, but approaching 2 orders of magnitude worse in our case. It's not even close. It's too bad for the people who need to use public VPN services for whatever reason, but until we have perfect bot/abuse detection, banning VPN, Tor, and…

No post body was provided.

Re: Meta quickly detects silent data corruptions at scale

#95

Post by tomrod »

pavlov wrote: post #43
Sindisil wrote: post #36

Earlier quoted context omitted.

How many candidates wouldn't know that Meta == Facebook, at least within the tech spheres?

The point of the rebrand is not to hide the association, but to make people think Meta > Facebook. I suspect it does help because a lot of hiring goes on for Quest, Instagram, Portal, WhatsApp, Workplace — you can talk to candidates about those specific products rather than make them think of grandpa posting right-wing memes on the blue app.

Yeah, this techie is not convinced that this whole Meta nonsense is meaningful. It's extremely disingenuous if recruiters actually expect people to not realize they are exactly who they are, a high paying corporation that has major negative externalities associated due to bad behavior.

Re: Meta quickly detects silent data corruptions at scale

#96

Post by [deleted] »

jreese wrote: post #79
fleddr wrote: post #78

Earlier quoted context omitted.

I once read that Facebook was opening 2 or 3 massive new data-centers in the US for the purpose of hosting stale content. You may have posted a photo 7 years ago, and statistics show that basically nobody ever revisits it. However, in case you do, it needs to be there. So these enormous buildings do basically nothing, but still need to be there. It makes me wonder how it can go on like this. Users only keep adding co…

There's no meaningful benefit to dedicating any amount of DC equipment just to stale content. Those are spindles (and networks) that could be taking meaningful hot reads and writes, and colocating stale and hot data is generally a better use of capacity than concentrating hot data in fewer locations.

[deleted]

Re: Meta quickly detects silent data corruptions at scale

#98

Post by Dylan16807 »

danuker wrote: post #51
londons_explore wrote: post #49

Earlier quoted context omitted.

But it's hard to 'abuse' reading a blog post...

There's denial of service, which wastes server resources, reducing the accessibility to humans interested in the content.

I'd think that your average VPN block could just be rate limited if that's a big worry.

Re: Meta quickly detects silent data corruptions at scale

#100

Post by ksec »

fleddr wrote: post #88
darkwater wrote: post #86

Earlier quoted context omitted.

What you say totally makes sense but still, even if new media takes more storage space, the accumulated stale date in the long run will win. Or maybe I'm underestimating how much space newer material needs?

Exactly. And that doesn't even take into account higher res photos and 4K video. I remember the staggering statistic where just Instagram sees 100 million photos added per day, every day. And that was years ago.

That is just Instagram, with photos. Imagine Youtube.

And again I have been saying this since ~2015/16, we dont have any meaningful roadmap for cost reduction on storage, whether that is Hot as in NAND, Bulk as in HDD, or Cold as in Optical Disc. I dont see 2TB SSD dropping below $100 in next 5 years, or 10TB HDD below $120.

Remember when Google promise infinite Gmail storage?

Post reply on HN