Obama's Trauma Team: Inside the Nightmare Launch of HealthCare.Gov • Hacker News Board

Re: Obama's Trauma Team: Inside the Nightmare Launch of HealthCare.Gov

#241

Post by nzp » Fri, Feb 28, 2014, 5:53 PM UTC

ekianjo wrote: post #190

Fomite wrote: post #187

Earlier quoted context omitted.

Not only do they have significant value, but they're the only type of evidence you'll ever have for something like this - it's impossible, and almost certainly unethical - to run a randomized trial of keeping people from having health insurance. Also, while "Correlation is not causation" is, as you mentioned, a tired canard on the internet, people often seem to forget that all things that do have a causal relationshi…

Sorry, but no. 100 times no. Correlation is very often linked to a third factor or multiple factors which are not visible, nor measured in observational studies. Besides, let's not disregard the fact that correlation still has some good chance to be pure luck. Even correlation with 95% confidence statistical significance can be a random result in a non-nil number of times. So, no, you never prove anything nor imply a…

> So, no, you never prove anything nor imply anything at all with correlation. You're still guessing.

This is a totally unreasonable stance to take. You can't even imply anything at all with correlation? Really, nothing at all? It's no better than a random guess? Try actually doing some actual science with this attitude, and keep to it consistently, and let me know how far you make it. In fact, try the same thing with ordinary life, any kind of reality where your decisions have consequences in reality.

Re: Obama's Trauma Team: Inside the Nightmare Launch of HealthCare.Gov

#242

Post by Fomite » Fri, Feb 28, 2014, 7:19 PM UTC

mattfenwick wrote: post #238

Fomite wrote: post #192

Earlier quoted context omitted.

Find me where I said proof . There are all kinds of things that we cannot prove, because it is either impossible or wildly unethical to conduct a randomized study. For those things, you can make a determined effort to control for as many "third factors" - the technical term for them is confounders - and that gives you a level of evidence which is well above guessing. Since I can't reply to your comment, my responses…

> make a determined effort to control for as many "third factors" How can you tell if you missed one? > [...] gives you a level of evidence which is well above guessing > It is better than guessing Why? Without any support, this seems to be an appeal to probability. > a well conducted observational study > a really bad study The fallacy of moving the goalposts; also the no true Scotsman fallacy.

> How can you tell if you missed one?

Similar studies, using as many variables as you can find. Residual confounding is always and ever a problem, but the odds that something is both a strong residual confounder and has never been observed to have an association with the outcome or the exposure is pretty rare?

> Why? Without any support, this seems to be an appeal to probability.

It's really not - if for no other reason than it's forced you to think about your system more than a simple guess would. It's not an appeal to probability, its using data to update whatever prior you came in with. Guesswork is just using your prior.

> The fallacy of moving the goalposts; also the no true Scotsman fallacy.

Not really, no. Some observational studies are crap - this is just true. But that doesn't say anything about the potential quality of observational evidence, and many of the commonly raised objections to observational studies are actually objections to poorly run studies. The example used was a study that examined no potential confounding variables, looked at a correlation with no prior evidence suggesting any linkage between the two or biological plausibility, and then asserts that they've found a causal link.

That's a bad study. It's not 'No True Scotsman Fallacy' to say that the problems with a bad study don't generalize to all studies. If it is, then we're all screwed, because you can run a bad RCT too.

Re: Obama's Trauma Team: Inside the Nightmare Launch of HealthCare.Gov

#243

Post by lorenyu » Fri, Feb 28, 2014, 7:28 PM UTC

vinhboy wrote: post #199

kalvin wrote: post #160

Earlier quoted context omitted.

(I'm working with Brandon) To clarify, you don't need to be able to commit long term-- if you can take leave from what you're otherwise doing for a few months, that's enough. If you want to work on something with real impact and help change the culture of government technology, email Brandon above! We're actively looking for a few good people.

So, out of genuine curiosity, why are you doing it? To help the country? It's exciting? For the money? Or some other reason completely? What about your co-workers. What's everyone motivation to help fix this site?

I believe that healthcare is one of the most important issues the country faces, on the same level as education, or climate change. And this was an opportunity for me to help out with that.

This is also one of rare times where we have a non-zero chance of pushing the government towards solving technological problems in better ways than hundred million dollar contracts with broken specifications for projects that are doomed for failure.

Kalvin is a friend of mine, and when he called me up I saw it also as a chance to work with a good friend and a great group of people, so it is exciting as well. It has been a great experience working with Brandon and everyone else on the team.

Re: Obama's Trauma Team: Inside the Nightmare Launch of HealthCare.Gov

#244

Post by keyhole_downs » Fri, Feb 28, 2014, 7:50 PM UTC

lorenyu wrote: post #243

vinhboy wrote: post #199

Earlier quoted context omitted.

So, out of genuine curiosity, why are you doing it? To help the country? It's exciting? For the money? Or some other reason completely? What about your co-workers. What's everyone motivation to help fix this site?

I believe that healthcare is one of the most important issues the country faces, on the same level as education, or climate change. And this was an opportunity for me to help out with that. This is also one of rare times where we have a non-zero chance of pushing the government towards solving technological problems in better ways than hundred million dollar contracts with broken specifications for projects that are…

Where do you draw the line on enabling a broken system that awards contracts to ill prepared monolithic RFP generating machines, and start saying, "We need real reform for technology in government and not just half-hearted bloviating about transparency"?

Re: Obama's Trauma Team: Inside the Nightmare Launch of HealthCare.Gov

#245

Post by brandonb » Fri, Feb 28, 2014, 9:14 PM UTC

coldcode wrote: post #182

brandonb wrote: post #115

Earlier quoted context omitted.

So far, it's been on a rotation -- folks go on leave from their jobs for 1-3 months, move out to DC/Baltimore area, and work as hard as possible while they're here. :) It's such a high-intensity project that I think the we'll want to keep it to full-time for the time being. Otherwise, it's hard to even keep up, let alone make progress. (As a technical point, though, even though everybody is working on this for well o…

What technology knowledge are required?

No particular knowledge is required, but all of our new code uses technologies like Backbone, Node, Capistrano, and MySQL.

We also make API calls into the existing system, which is written in Java, so it helps to be able to grasp a complex Java codebase with way too many layers and enterprise systems. :)

But mostly, we're just looking for great software engineers who want to make a difference, not any particular set of acronyms.

Re: Obama's Trauma Team: Inside the Nightmare Launch of HealthCare.Gov

#246

Post by Fomite » Fri, Feb 28, 2014, 9:47 PM UTC

anatoly wrote: post #209

Fomite wrote: post #187

Earlier quoted context omitted.

Not only do they have significant value, but they're the only type of evidence you'll ever have for something like this - it's impossible, and almost certainly unethical - to run a randomized trial of keeping people from having health insurance. Also, while "Correlation is not causation" is, as you mentioned, a tired canard on the internet, people often seem to forget that all things that do have a causal relationshi…

You keep saying 'unethical' as if it's the end of the discussion. But what if our current standard of medical ethics prevent us from finding the better policies or the better treatments? Is this standard set in stone and never to be doubted? Probably taking away insurance from people who have it as an experiment is too extreme, and not implementable anyway. But is it really that unethical to take a subset of populati…

> You keep saying 'unethical' as if it's the end of the discussion. But what if our current standard of medical ethics prevent us from finding the better policies or the better treatments? Is this standard set in stone and never to be doubted?

No, it's not set in stone, but if you're talking about implementing studies now, you're not going to get major medical ethics reform first. You go with the system you have, and the system you have is probably going to push back pretty hard.

> No one from the control group is prevented from getting insurance on their own

This alone is a difference between the control and treatment groups that takes place post randomization, and knocks said experiment back into the realm of "correlational"

Re: Obama's Trauma Team: Inside the Nightmare Launch of HealthCare.Gov

#247

Post by keyhole_downs » Fri, Feb 28, 2014, 10:32 PM UTC

liyanchang wrote: post #231

twistedpair wrote: post #149

Earlier quoted context omitted.

Amen. Just watch any documentary about building an aircraft carrier. The systems integration phase is the longest part!

The difference here is that they had a very public deadline.

The difference here is that politicians treat technology the way voters treat elections.

Re: Obama's Trauma Team: Inside the Nightmare Launch of HealthCare.Gov

#248

Post by keyhole_downs » Fri, Feb 28, 2014, 10:34 PM UTC

brandonb wrote: post #125

keithwarren wrote: post #121

Earlier quoted context omitted.

What is the biggest issue right now architecturally?

Complexity. We have 10x more code, 10x more components, and 10x more layers than we need. If the initial architecture had been dirt-simple, I don't think the site would have had so much trouble scaling or staying up. But, of course, removing complexity without breaking things takes longer than adding it in the first place. The other big problem is operations — many steps are done manually which should be done with to…

Better call Scott!

Re: Obama's Trauma Team: Inside the Nightmare Launch of HealthCare.Gov

#249

Post by brandonb » Fri, Feb 28, 2014, 11:18 PM UTC

pikewood wrote: post #234

"...government regulations did not allow them, even though they offered, to be volunteers if they worked for any sustained period. So they were put on the payroll of contractor QSSI as hourly workers, making what Dickerson says was "a fraction" of his Google pay" This information troubles me, because QSSI was just hired to fix Maryland's broken Health Exchange site. It sounds like the company is cashing in on the don…

Unfortunately, each of the 14 states that made their own exchange used a separate team with a separate codebase.

To be clear, it's not like QSSI or anybody else is ripping us off. We're all getting paid the standard contracting rate. But many of us probably would've worked for free if they had let us.

Re: Obama's Trauma Team: Inside the Nightmare Launch of HealthCare.Gov

#250

Post by blendo » Fri, Feb 28, 2014, 11:55 PM UTC

brandonb wrote: post #245

coldcode wrote: post #182

Earlier quoted context omitted.

What technology knowledge are required?

No particular knowledge is required, but all of our new code uses technologies like Backbone, Node, Capistrano, and MySQL. We also make API calls into the existing system, which is written in Java, so it helps to be able to grasp a complex Java codebase with way too many layers and enterprise systems. :) But mostly, we're just looking for great software engineers who want to make a difference, not any particular set…

Is it the existing system the "bad" kind of Java (J2EE 1.4/EJB 1/xml hell/Abstract Factory madness) or the nicer and more modern RESTy/Dependency Injected/JEE6 (or Spring) variant?

And I recall that one of the trickier requirements is correctly sending and acking enrollment info (EDI?) to and from the many (dozens?) of insurance companies involved. Is that piece looking ok?