Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Could Better Testing Practices Have Prevented the Healthcare.gov Defects?

blog.smartbear.com

Post reply on HN
Print view View original

21–30 of 54 posts

Re: Could Better Testing Practices Have Prevented the Healthcare.gov Defects?

#21

Post by so_says »

JPKab wrote: post #17
jrochkind1 wrote: post #9

Sure, they could have been prevented. By spending more money. Isn't that always the answer? Creating a website that can handle, well, nearly the entire country, out of the gate, is not trivial. (Facebook etc got to scale up to that level, they didn't need to do it out of the gate). But it also can be done. How? Well, you hire real experts in doing this sort of thing. And you give them enough calendar time, and enough…

Your opinion shouldn't be high. There are talented IT people working at government contractors, but they aren't going to get put where they are needed, because the incentives for contractors are not performing the work at a top-notch level. Gov't contracting is, in economic terms, a rent-seeking business. The best and brightest in a given contracting firm are dedicated to pursuing new work. I work for a contractor, a…

I'm mad as hell about this because I want the ACA to succeed. I don't think its ideal, but its vastly better than the existing system of hospitals as clinics for the uninsured hordes.

I couldn't agree more. You only get one chance to make a first impression and they blew it. (sigh)

Re: Could Better Testing Practices Have Prevented the Healthcare.gov Defects?

#22

Post by mlinksva »

bmelton wrote: post #15

If anyone has seen my posts in the past, you'll know that I'm not a fan of the ACA, but disregarding that, let me be the first to say that this probably isn't the fault (or, at least not solely the fault) of those building the web frontends. In Maryland, our exchange website is poorly designed and written, just on the frontend (I obviously can't see the backend), but at the same time, all of the frontend issues could…

> I obviously can't see the backend

You can't see what's running at the moment of course, but why shouldn't you expect to be able to see the source? Why are new government IT projects closed source?

Re: Could Better Testing Practices Have Prevented the Healthcare.gov Defects?

#23

Post by coldcode »

chintan wrote: post #11

http://blog.netizencorp.com/2013/10/06/how-a-scrappy-startup... "A scrappy little startup working out of a DC garage that completely influenced the course of how the web-facing portions of the Affordable Care Act were to be implemented"

STATIC site content. Not the part being hammered by people signing up. People keep confusing the two.

Re: Could Better Testing Practices Have Prevented the Healthcare.gov Defects?

#24

Post by bmelton »

mlinksva wrote: post #22
bmelton wrote: post #15

If anyone has seen my posts in the past, you'll know that I'm not a fan of the ACA, but disregarding that, let me be the first to say that this probably isn't the fault (or, at least not solely the fault) of those building the web frontends. In Maryland, our exchange website is poorly designed and written, just on the frontend (I obviously can't see the backend), but at the same time, all of the frontend issues could…

> I obviously can't see the backend You can't see what's running at the moment of course, but why shouldn't you expect to be able to see the source? Why are new government IT projects closed source?

Interesting you mention that. I saw this the other day, and was debating whether or not it was (at least the frontend of) the actual running site:

https://github.com/CMSgov/healthcare.gov

Re: Could Better Testing Practices Have Prevented the Healthcare.gov Defects?

#25

Post by camus »

ErikAugust wrote: post #19
programminggeek wrote: post #13

I've dealt with having to scale a site to millions of pageviews and honestly, if you've never had to deal with a traffic spike, you probably aren't going to build for the scale issues you will have. For example, there is a lot of caching you just have to do. Tons of it. Cache as much as you can. Memcache and Varnish are your friends, use them as much as you can. Unfortunately, if Healthcare.gov was in a very write-he…

"Slow DB queries are the enemy and you don't realize how bad they are until you are at scale. Sure, it only takes a few seconds on your local machine, but multiply that times thousands of concurrent users and your DB gets swamped. If you are using an ORM, it is MUCH harder to track down where in your code that 3 way join that scans every record is happening. Ideally you'd be using straight SQL and maybe use comments…

Problem is your ORM might generate 300 fast queries to get 300 records where 1 would be faster than 300. using an ORM make things tricky since you basically surrender most of Db control .

Scaling databases is hard,developpers like ORMs , but ORMs make scaling databases even harder.

Re: Could Better Testing Practices Have Prevented the Healthcare.gov Defects?

#26

Post by integraton »

bmelton wrote: post #24
mlinksva wrote: post #22

Earlier quoted context omitted.

> I obviously can't see the backend You can't see what's running at the moment of course, but why shouldn't you expect to be able to see the source? Why are new government IT projects closed source?

Interesting you mention that. I saw this the other day, and was debating whether or not it was (at least the frontend of) the actual running site: https://github.com/CMSgov/healthcare.gov

This is only the content site, not the marketplace. See: "This project does not include any source code for the Federal Health Insurance Marketplace (the online systems located under www.healthcare.gov/marketplace."

Re: Could Better Testing Practices Have Prevented the Healthcare.gov Defects?

#27

Post by Pxtl »

camus wrote: post #25
ErikAugust wrote: post #19

Earlier quoted context omitted.

"Slow DB queries are the enemy and you don't realize how bad they are until you are at scale. Sure, it only takes a few seconds on your local machine, but multiply that times thousands of concurrent users and your DB gets swamped. If you are using an ORM, it is MUCH harder to track down where in your code that 3 way join that scans every record is happening. Ideally you'd be using straight SQL and maybe use comments…

Problem is your ORM might generate 300 fast queries to get 300 records where 1 would be faster than 300. using an ORM make things tricky since you basically surrender most of Db control . Scaling databases is hard,developpers like ORMs , but ORMs make scaling databases even harder.

The big thing with an ORM is that you must keep those lazy loads in mind when you work with it. Lazy loads are the big place where you get "300 queries instead of 1".

ORMs aren't bad on principal, imho... but they give you more than enough rope to hang yourself while promising a magical efficient user-friendly rope.

Re: Could Better Testing Practices Have Prevented the Healthcare.gov Defects?

#28

Post by RougeFemme »

jrochkind1 wrote: post #9

Sure, they could have been prevented. By spending more money. Isn't that always the answer? Creating a website that can handle, well, nearly the entire country, out of the gate, is not trivial. (Facebook etc got to scale up to that level, they didn't need to do it out of the gate). But it also can be done. How? Well, you hire real experts in doing this sort of thing. And you give them enough calendar time, and enough…

Well, you can give them your own business requirements, which should be high-level and state "I need a system that performs these functions for this many people". What you should not do is dictate design and implementation details. That's when you rely on their expertise. . .unfortunately, as you stated, that's rarely how it works.

Also, it would have really helped to have someone - likely not a developer - to bridge the gap and ask the customer "do you really need people to set up user access before viewing plans or you just want that? why?" and then turn to the developer and say "ok, what does that mean technically? what the pros and cons?"

Re: Could Better Testing Practices Have Prevented the Healthcare.gov Defects?

#29

Post by jasonpeacock »

Regardless of any technological or architectural decisions, simple load testing before launch would have prevented this.

1. Load test, discover issues. 2. Fix issues. 3. Repeat steps 1-2 until no more issues.

That is the flabbergasting part. Nobody actually tried applying real load to the system before release? It fell over so easily and quickly the only assumption is either no loadtesting was performed, the results were ignored, or massively incorrect load numbers were used (all of which are signs of incompetent mangement).

Re: Could Better Testing Practices Have Prevented the Healthcare.gov Defects?

#30

Post by RougeFemme »

JPKab wrote: post #17
jrochkind1 wrote: post #9

Sure, they could have been prevented. By spending more money. Isn't that always the answer? Creating a website that can handle, well, nearly the entire country, out of the gate, is not trivial. (Facebook etc got to scale up to that level, they didn't need to do it out of the gate). But it also can be done. How? Well, you hire real experts in doing this sort of thing. And you give them enough calendar time, and enough…

Your opinion shouldn't be high. There are talented IT people working at government contractors, but they aren't going to get put where they are needed, because the incentives for contractors are not performing the work at a top-notch level. Gov't contracting is, in economic terms, a rent-seeking business. The best and brightest in a given contracting firm are dedicated to pursuing new work. I work for a contractor, a…

I don't think PMP certifications are any worse than any other certifications, though often, as with other certifications, they often aren't worth the paper they are printed on. And I agree that the project managers often have little-to-no technical expertise in the project they are managing; but that's not supposed to be their job. Their job is supposed to be to manage the job that the experts are working on and let the experts do their thing. Although they (PMs) should have at least enough knowledge to understand discussions at a high level, facilitate when necessary and recognize when one of the experts is blowing smoke out his/her ass.
Post reply on HN