Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Visualizing Meltdown on AWS

blog.appoptics.com

Post reply on HN
Print view View original

51–60 of 72 posts

Re: Visualizing Meltdown on AWS

#51

Post by gnosek »

user5994461 wrote: post #47
gnosek wrote: post #24

Earlier quoted context omitted.

https://www.kimsufi.com/us/en/ Granted, that's the bottom of the barrel (single disk, no IPKVM etc.), but $100 keeps you running for over a year. Better servers are easily available as well, usually a couple of times cheaper than AWS. Is this a US thing? Based on HN only, I'd never know there's anything between the public cloud and racks of own hardware that you have to wire up and maintain. I have a bunch of quad co…

HN has a lot of professionals. They can't run a business on a refurbished server without ECC and without RAID and without dual power supplies. Saying that they should run on kumsufi is like explaining to a wholesale company that they should use motorbikes instead of trucks, because motorbikes are cheaper.

Of course. I'm not telling anybody to run on kimsufi, like I'm not running kimsufi myself, but on brand-new Supermicro machines that check all your boxes.

Kimsufi is still dedicated hardware (however low-end) that costs 1 figure instead of 5/6. For normal server-grade hardware it's just 5x cheaper than AWS, not 20x.

Re: Visualizing Meltdown on AWS

#52

Post by [deleted] »

late2part wrote: post #50
user5994461 wrote: post #47

Earlier quoted context omitted.

HN has a lot of professionals. They can't run a business on a refurbished server without ECC and without RAID and without dual power supplies. Saying that they should run on kumsufi is like explaining to a wholesale company that they should use motorbikes instead of trucks, because motorbikes are cheaper.

AWS Advocates Eventual Consistency, and I believe offers less than 3 nines guaranteed uptime on many products. We've been taught to build distributed system with unreliable componens and temporaral JIT eventual consistency. Of course we can run production cloud-scale operations on unreliable systems. And single power supplies without ECC or RAID is pretty low on my list of things that cause outages. Most big hadoop/c…

[deleted]

Re: Visualizing Meltdown on AWS

#53

Post by tomsmeding »

mrep wrote: post #22
schak wrote: post #20

Earlier quoted context omitted.

Hi, How are you trying to measure the performance impact? Are you checking the cloudwatch data or running any specific test? I am interested to assess the baseline statistics (unpatched so far) and after patching. Any suggestions?

my teams usage is pretty even so the cloudwatch graph is pretty obvious: https://m.imgur.com/a/khGxU

That is the clearest graph ever to show the degradation in performance. How do you get your usage so flat?

Re: Visualizing Meltdown on AWS

#54

Post by user5994461 »

gnosek wrote: post #51
user5994461 wrote: post #47

Earlier quoted context omitted.

HN has a lot of professionals. They can't run a business on a refurbished server without ECC and without RAID and without dual power supplies. Saying that they should run on kumsufi is like explaining to a wholesale company that they should use motorbikes instead of trucks, because motorbikes are cheaper.

Of course. I'm not telling anybody to run on kimsufi, like I'm not running kimsufi myself, but on brand-new Supermicro machines that check all your boxes. Kimsufi is still dedicated hardware (however low-end) that costs 1 figure instead of 5/6. For normal server-grade hardware it's just 5x cheaper than AWS, not 20x.

Kimsufi and soyoustart are brands of OVH. You don't need to explain again and again that they sell cheaper servers than AWS. Same for super micro. The HN audience know the common suppliers.

Again, you're arguing for motorbikes instead of trucks because they are cheaper. Noone care about that, trucks were not selected based on costs in the first place.

Re: Visualizing Meltdown on AWS

#55

Post by user5994461 »

late2part wrote: post #50
user5994461 wrote: post #47

Earlier quoted context omitted.

HN has a lot of professionals. They can't run a business on a refurbished server without ECC and without RAID and without dual power supplies. Saying that they should run on kumsufi is like explaining to a wholesale company that they should use motorbikes instead of trucks, because motorbikes are cheaper.

AWS Advocates Eventual Consistency, and I believe offers less than 3 nines guaranteed uptime on many products. We've been taught to build distributed system with unreliable componens and temporaral JIT eventual consistency. Of course we can run production cloud-scale operations on unreliable systems. And single power supplies without ECC or RAID is pretty low on my list of things that cause outages. Most big hadoop/c…

hadoop/cassandra are replicated to multiple nodes by the software. That's a terrible argument for not using any raid in regular setups.

Re: Visualizing Meltdown on AWS

#56

Post by cthalupa »

mike_heffner wrote: post #40

Would love to know if anyone else had data on: * Impact on M5/C5 instances over similar time period, any difference with the Nitro hypervisor? * Were Dedicated instances ( https://aws.amazon.com/ec2/purchasing-options/dedicated-inst... ) patched as well? * Other examples of software that adapted batching performance automatically with increase in call latency.

Not able to answer your questions, but a comment on the article -

>During this same time period, we saw additional CPU increases on our PV instances that had been previously upgraded. This seems to imply some level of HVM patching was occurring on these PV instances around the same time that all pure-HVM instances were patched

This is likely due to Vixen: https://lists.xenproject.org/archives/html/xen-devel/2018-01...

>.... Instead of trying to make a KPTI-like approach work for Xen PV, it seems reasonable to run a copy of Xen within an HVM (or PVH) domU ..... >.... all PV instances in EC2 are using this ....

So the initial bump after the reboot would have been the shim hypervisor which mitigates Vixen. The secondary bump, and bump the native HVM instances saw, would have been the Spectre related stuff.

Based on https://aws.amazon.com/security/security-bulletins/AWS-2018-... - guessing Intel microcode updates

Re: Visualizing Meltdown on AWS

#57

Post by late2part »

user5994461 wrote: post #55
late2part wrote: post #50

Earlier quoted context omitted.

AWS Advocates Eventual Consistency, and I believe offers less than 3 nines guaranteed uptime on many products. We've been taught to build distributed system with unreliable componens and temporaral JIT eventual consistency. Of course we can run production cloud-scale operations on unreliable systems. And single power supplies without ECC or RAID is pretty low on my list of things that cause outages. Most big hadoop/c…

hadoop/cassandra are replicated to multiple nodes by the software. That's a terrible argument for not using any raid in regular setups.

I suspect your and my definition of regular setups is different. My regular setups are stateless and automatically installed and configured.

Re: Visualizing Meltdown on AWS

#58

Post by dgsb »

perfmode wrote: post #33

Over a 15 year time scale, there is no way AWS will remain competitive with GCP.

Why is that ? They both belongs to the top cloud providers today, both seems to invest highly on their R&D. I don't see anything obvious on what could happen in the future.

Re: Visualizing Meltdown on AWS

#59

Post by user5994461 »

late2part wrote: post #57
user5994461 wrote: post #55

Earlier quoted context omitted.

hadoop/cassandra are replicated to multiple nodes by the software. That's a terrible argument for not using any raid in regular setups.

I suspect your and my definition of regular setups is different. My regular setups are stateless and automatically installed and configured.

Lucky you, never having to deal with any database or storage.

Re: Visualizing Meltdown on AWS

#60

Post by scurvy »

013a wrote: post #45
scurvy wrote: post #36

Earlier quoted context omitted.

PCI compliance had little to do with where you're hosted, but how you're hosted and what you do with data. AWS is not a magical PCI compliance button.

Theoretically, it isn't. In actuality, and in our experience, there are a lot of compliance standards where just saying "we're on AWS" gets you 90% of the way toward acceptance. Its mostly buzzword compliance.

That's just not true. Most of PCI compliance is documenting practices and operations. Have you even completed all various levels of compliance? Or have you just done one level?
Post reply on HN