Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Company was hosting cardiac patient monitoring on EC2

forums.aws.amazon.com

Post reply on HN
Print view View original

61–70 of 85 posts

Re: Company was hosting cardiac patient monitoring on EC2

#61

Post by agotterer »

sliverstorm wrote: post #19
zaidf wrote: post #15

This is where you wish you had a data center address and a cabinet number where you could pop in, grab your servers and move the heck out to another DC. I wonder if this will get more companies to maintain a non-cloud, vanilla setup on some dedicated or collocated boxes. May be we should have an annual day to bring down our primary servers and see how the backups do. The idea shouldn't be to confirm if everything wor…

You know, that gets me thinking- wouldn't it be pretty damn slick if Amazon offered a way to plug your own hardware into the greater EC2 cluster remotely, so that if Big EC2 goes down, you still have copies of your machines running locally? Maybe you could even timeshare your machines out for some other instances for a discount on your service, similar to how you can send solar power back up the power grid for an ele…

Not exactly what your talking about... But I believe RackSpace has an offer for mixed cloud and physical machines...

Re: Company was hosting cardiac patient monitoring on EC2

#62

Post by aquark »

sliverstorm wrote: post #19
zaidf wrote: post #15

This is where you wish you had a data center address and a cabinet number where you could pop in, grab your servers and move the heck out to another DC. I wonder if this will get more companies to maintain a non-cloud, vanilla setup on some dedicated or collocated boxes. May be we should have an annual day to bring down our primary servers and see how the backups do. The idea shouldn't be to confirm if everything wor…

You know, that gets me thinking- wouldn't it be pretty damn slick if Amazon offered a way to plug your own hardware into the greater EC2 cluster remotely, so that if Big EC2 goes down, you still have copies of your machines running locally? Maybe you could even timeshare your machines out for some other instances for a discount on your service, similar to how you can send solar power back up the power grid for an ele…

You can ... sort of. Amazon offer dedicated instances: http://aws.typepad.com/aws/2011/03/amazon-ec2-dedicated-inst...

Just rather than supplying the hardware you have to rent it from Amazon.

Would be interesting to know how these instances were affected by the outage. Since the problem was with the EBS infrastructure not the actual compute hosts I'm guessing this wouldn't have helped any

Re: Company was hosting cardiac patient monitoring on EC2

#63

Post by ceejayoz »

sliverstorm wrote: post #39
ceejayoz wrote: post #33

Earlier quoted context omitted.

That sounds horrid. It'd mean my instances might be running on your ancient crap computer in a closet hooked up to dialup.

You could always add min spec requirements for machines to join, increase redundancy of instances ec2-wide, add quality tiers that control what machines your instances go on, etc. Amazon could even simply benchmark your server and network connection, and compensate you as a function of that, which would drastically reduce the incentive to hook up old crappy hardware to the cloud. It's really the idea of being able to…

> It's really the idea of being able to run copies of your OWN instances locally that intrigues me though.

No reason you can't already do this.

Re: Company was hosting cardiac patient monitoring on EC2

#64

Post by airlabam »

sliverstorm wrote: post #19
zaidf wrote: post #15

This is where you wish you had a data center address and a cabinet number where you could pop in, grab your servers and move the heck out to another DC. I wonder if this will get more companies to maintain a non-cloud, vanilla setup on some dedicated or collocated boxes. May be we should have an annual day to bring down our primary servers and see how the backups do. The idea shouldn't be to confirm if everything wor…

You know, that gets me thinking- wouldn't it be pretty damn slick if Amazon offered a way to plug your own hardware into the greater EC2 cluster remotely, so that if Big EC2 goes down, you still have copies of your machines running locally? Maybe you could even timeshare your machines out for some other instances for a discount on your service, similar to how you can send solar power back up the power grid for an ele…

Couldn't you already do that by just setting up a Eucalyptus cluster? From Wikipedia entry:

"Eucalyptus is a software platform for the implementation of private cloud computing on computer clusters. There is an enterprise edition and an open-source edition. Currently, it exports a user-facing interface that is compatible with the Amazon EC2 and S3 services but the platform is modularized so that it can support a set of different interfaces simultaneously. "

Re: Company was hosting cardiac patient monitoring on EC2

#65

Post by madaxe »

sliverstorm wrote: post #5

Hmm... That would rather seem to be the monitoring company's own fault. Admittedly I haven't used ec2 for about 3 years, but I seem to recall there being an explicit disclaimer about how it wasn't supposed to be used for this kind of uptime-critical application. I want to say it even specifically mentioned medical stuff.

Nah, you can use EC2 for uptime-critical apps, you just have to a) not be a dipshit and b) stick your stuff in as many AZs as you can, and ELB it.

Re: Company was hosting cardiac patient monitoring on EC2

#66

Post by jrockway »

zaidf wrote: post #16
sliverstorm wrote: post #5

Hmm... That would rather seem to be the monitoring company's own fault. Admittedly I haven't used ec2 for about 3 years, but I seem to recall there being an explicit disclaimer about how it wasn't supposed to be used for this kind of uptime-critical application. I want to say it even specifically mentioned medical stuff.

Agree. I'd avoid cloud for half as critical apps where I cannot CLEARLY picture where my servers are in a physical sense and how I can physically get a hold of them if shit hits the fan.

How do you physically get a hold of your servers if the gas main under them blows up or a tornado sucks them up and drops them a hundred miles away?

Servers are not physical things that you can just go get when there are problems. They are imaginary devices that can suddenly vaporize, leaving no trace. Plan your deployment accordingly, and you won't need to be writing emails saying that people are dying because your data center blew up. Your systems will silently switch over to some other data center and you'll file a claim with your insurance company to have the servers replaced on Monday morning.

In other words, "the cloud" had nothing to do with this problem. It could happen if the server was physically attached to you or in a abandoned missle silo with an army of armed guards.

Re: Company was hosting cardiac patient monitoring on EC2

#67

Post by stick »

This is a combination of posters not understanding what the technology is used for and the tech guy exaggerating the urgency of the situation.

Remote cardiac monitoring isn't for people who are imminently going to die of catastrophic heart attacks or who risk dropping into a fatal arrhythmia (wonky heart rhythm). In fact, there isn't even solid evidence that it saves lives. What it is used for is investigating patients who have vague symptoms which might be related to transient changes in their heart beat.

These are people who come to their doctor at the age of 54 and say "I felt my heart beating really quickly and felt kind of faint" or "I felt kind of dizzy and then passed out. It's happened twice in the last month." The more traditional way to investigate these patients are with what's called a Holter monitor, as alluded to by a previous poster. These are little belt packs you carry around while wired-up and that record your ECG for 24-48 hours at a time. The main weakness? You said it: the device only has a 24-48 hour windows to capture the weird, often rare, rhythm.

There are different ways remote cardiac monitoring systems report their results and I'm not sure which this particular company was using, but it doesn't really matter. Some of them only report weird stretches, others only report events when the patient says they're feeling symptom X, others are reporting continuously.

Take-home message: this is not life-or-death data. When doctors (at least those who are allowed to keep practicing) think a patient needs critical cardiac monitoring, they admit them to hospital.

This was a tech guy, looking to jump the queue by trying to raise a red flag because his servers were being used for--OMFG!--cardiac monitoring. A lot of my doctor buddies use similar strategies when they're caught speeding by the police. "I just got called in to the hospital!"

…to see a patient with a really nasty rash.

Re: Company was hosting cardiac patient monitoring on EC2

#68

Post by jrockway »

chrisjsmith wrote: post #24
MichaelApproved wrote: post #14

Earlier quoted context omitted.

Ec2 is fine here. All systems have downtime. Its not having a backup system in place that's the problem.

Cardiac monitoring systems DO NOT have downtime believe it or not.

He's saying that any computer system hosting the data would have downtime. Your own custom data center can blow up just as easily as Amazon's.

Now, if zero downtime is the requirement, then you just need to have a lot of data centers and well-tested failover procedures. It's all very simple, and EC2 can easily be one of those redundant data centers.

Re: Company was hosting cardiac patient monitoring on EC2

#70

Post by weaksauce »

dr_ wrote: post #55
esrauch wrote: post #51

Earlier quoted context omitted.

I could be way off base here since I only know what I saw on the site, but it doesn't look like this is life-or-death monitoring. It looks like its long term monitoring for later review by a doctor. It doesn't seem at all likely that they have someone monitoring the incoming data in realtime for heart attacks, that would be far better suited by hooking up the system to dial 911. I think some IT person was freaking ou…

Seems like it was real time monitoring. Otherwise why not just use the traditional Holter monitor, which is later reviewed by a doctor?

Apparently it only has a 24-48 hour limit which lowers the likelihood of finding the super rare event that they are looking for. Still they should have a local cache of a few days that they can sync when amazon gets back up at the very least. Typically though you would want to have a separate datacenter to handle the emergency situations like this though and a smaller local cache for local Internet outages.
Post reply on HN