Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon FSx for Lustre

aws.amazon.com

Post reply on HN
Print view View original

11–20 of 22 posts

Re: Amazon FSx for Lustre

#11

Post by pinewurst »

davidmr wrote: post #9
pinewurst wrote: post #8

That would have to be the worst job in the world - keeping Lustre going as an Amazon service, with management that utterly lacks understanding and sympathy.

They’d have to pay me a lot of money to do it, that’s for sure. I’d love to see the disaster recovery plans. Every major Lustre site I’m aware of has had a data loss “incident” at some point in their history. It’s possible AWS has it all figured out with background backups and block device replication and whatnot, but I’m skeptical.

I have doubts based on my experience with Lustre and a certain understanding of AWS operations. I'm guessing they're going for the Iranian minefield clearing technique - get a mob of kids, hand them plastic keys to heaven (or RSUs), and march them through the field.

Re: Amazon FSx for Lustre

#15

Post by bovermyer »

dschuetz wrote: post #14
bovermyer wrote: post #13

Earlier quoted context omitted.

Posting the same comment on every single one of these doesn't help anything. Basically, you're just spamming.

That's exactly my point. Apparently Amazon is being spammend all over HN. What the heck?!

Welcome to AWS Re:invent week, friend.

Re: Amazon FSx for Lustre

#16

Post by notacoward »

pinewurst wrote: post #8

That would have to be the worst job in the world - keeping Lustre going as an Amazon service, with management that utterly lacks understanding and sympathy.

Same thought here. I spent two-plus years debugging Lustre issues for a very small set of customers. It was an absolute beast. Build process was a compatibility-killing license-violating nightmare. Fell over at the slightest provocation, with little info to help figure out why. Provided no metrics to speak of, and the thicket of inter-related settings (especially timeouts) made effective tuning almost impossible. I'd guess that Amazon spent many engineer-years removing or rewriting significant pieces, and even more establishing the safe configuration envelope for what remained. Even then, it's probably a nightmare for the SREs (or whatever Amazon calls them) who have to keep it running.

Re: Amazon FSx for Lustre

#19

Post by pinewurst »

mbreese wrote: post #10
davidmr wrote: post #9

Earlier quoted context omitted.

They’d have to pay me a lot of money to do it, that’s for sure. I’d love to see the disaster recovery plans. Every major Lustre site I’m aware of has had a data loss “incident” at some point in their history. It’s possible AWS has it all figured out with background backups and block device replication and whatnot, but I’m skeptical.

Given that they call the non-S3 linked version 'ephemeral', I'm not sure there is a plan. I think S3 is the plan.

'Ephemeral' was/is the original Lustre design model. It was intended for high performance swap/scratch at Livermore with a short data lifespan - your higher priority bomb sim forces mine to roll out to disk and back in later, and that's it. Lustre, even today, isn't long term stable. The longer you leave data on it, the greater the probability of corruption - even silently.

Re: Amazon FSx for Lustre

#20

Post by mbreese »

pinewurst wrote: post #19
mbreese wrote: post #10

Earlier quoted context omitted.

Given that they call the non-S3 linked version 'ephemeral', I'm not sure there is a plan. I think S3 is the plan.

'Ephemeral' was/is the original Lustre design model. It was intended for high performance swap/scratch at Livermore with a short data lifespan - your higher priority bomb sim forces mine to roll out to disk and back in later, and that's it. Lustre, even today, isn't long term stable. The longer you leave data on it, the greater the probability of corruption - even silently.

I've seen Lustre backed with ZFS listed a few places. Is the idea here to help mitigate the possibility of corruption?
Post reply on HN