Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

technologyreview.com

Post reply on HN
Print view View original

361–370 of 688 posts

Re: CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

#361

Post by gunsch »

dopylitty wrote: post #249
offtop5 wrote: post #200

Earlier quoted context omitted.

For various reasons this wouldn't work, you have the various bands of government salary https://federalnewsnetwork.com/pay/2020/01/more-top-career-e... >But for a GS-15, step 10 in Washington, his or her salary should total roughly $185,509 in 2020. But due to the federal pay ceiling, that same GS-15, step 10 in Washington will make the limit of $170,800 this year. And that's the absolute Max, after you go to the tro…

As someone with a degree in software "engineering" and more than a decade of experience in the industry I'd gladly take a stable government job creating or maintaining important applications that help the government provide high quality and necessary services to people. I would take pride in working on such a system as a vaccine administration management platform. Instead those jobs are doled out to contracting firms…

There are government contractors who are doing this kind of work! There's a group of younger companies that grew up around the Healthcare.gov rescue and work in conjunction with USDS, trying to build cultures of better software development practices in government. Check out the companies in the Digital Services Coalition (https://digitalservicescoalition.org/) and you'll see a lot of companies doing the kind of work you talk about.

I work for Ad Hoc (one of these companies) on software projects at the VA, and I'm happy to chat with anyone who's interested to hear more about this kind of work. I left a cushy Google job to work on important and needed systems, and it does feel so much better!

Re: CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

#362

Post by rapnie »

ethbr0 wrote: post #292
eigenvector wrote: post #277

Earlier quoted context omitted.

Now think of the problem in reverse, from the government's side. Why are they contracting it out when all of the most difficult aspects involve integrating with government systems, policies and personnel - things that government already knows how to do? "Because we don't have the expertise to do software development." The comments in this thread, which I fully agree with by the way, just cement my belief that if gove…

This is pretty much what I've learned doing 5+ years of big enterprise consulting (though not government). If the hardest / longest part of any project requires domain knowledge and personal relationships, then why are you outsourcing that part? I don't care if it's not a core competency / something you can do. Make it one.

> then why are you outsourcing that part?

To what extent is this due to neo-liberal "small government" policies, where from the highest political levels - through lobbying - it is decided that outsourcing is a win-win that leads to financial savings and budgets that are cut, and later on cut some more, etcetera?

I've witnessed the crazy dance of enterprise consultants in large corporations (not government) and it is astounding how much money is wasted in inefficiency.

I was in a small tech shop and we had just won an RFP at a big corporation, with a very practical solution and for a limited domain. Our down-to-earth approach led to so much enthusiasm with corporate employees, that they sent all their consultants to us to investigate interfaces to ever more back-end systems. Scope ballooned, and we - being a small party - were dictated to support all this stuff. It was utter madness to such extend that we decided to more-or-less graciously wringle out of the contract. Just-in-time before we were committed to a monstrous project that was 100% certain to fail. And that was not the only time we experienced this phenomenon in big corp.

Re: CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

#363

Post by cheschire »

eaton wrote: post #276
eckza wrote: post #263

Earlier quoted context omitted.

> Development is literally the simplest part of the problem set. This is the single most important and insightful sentence that a lot of HN-ers are going to read today, and they don't even know it.

I'd never in a million years argue that Deloitte did an awesome job on a large government contract, but it's mind-boggling to watch the regular ritual of HN posters blithely insist that anything more over $20K and a pile of pizzas for government infrastructure is a scam.

Tech people are just as vulnerable to the Dunning-Kruger effect as the rest of the population.

Re: CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

#364

Post by rabidrat »

ethbr0 wrote: post #287
eckza wrote: post #263

Earlier quoted context omitted.

> Development is literally the simplest part of the problem set. This is the single most important and insightful sentence that a lot of HN-ers are going to read today, and they don't even know it.

> most missteps happen early, dooming projects before anyone sits down to code This one is more insightful (in the big non-tech enterprise / government spaces). Because the takeaway for most HN developers who flip between projects should be: (1) learn how to recognize one of these projects & (2) stay away / find a new job when you do. You'll save years of your work.

How might you recognize a project like this?

Re: CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

#365

Post by onion2k »

krrrh wrote: post #163
NearAP wrote: post #98

Earlier quoted context omitted.

>>Internet Explorer has less than a 2% market share in the US, so under normal circumstances (given the sacrifices you’d have to make everywhere else) not bothering to support is is often a sensible decision. When it’s an access to healthcare issue though you can’t really exclude 2% of users. That thinking is exactly the reason why this system turned out to be problematic. Even if it wasn't for healthcare purposes, b…

Detect IE, redirect to a static site with a 1-800 number, have a call centre staffed with people running Chromebooks schedule an appointment over the phone. Probably cheaper overall, and certainly a better user experience for the tech-challenged seniors.

Something to note about this strategy is that older versions of IE (10 and older) only support TLS 1.0 by default. Below 8 you're stuck with TLS 1.0 and can't even enable anything higher. So if "https://www.cdc.gov" is using modern web security IE users won't even get to request data from the server, so you can't detect their browser.

Re: CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

#366

Post by offtop5 »

dopylitty wrote: post #249
offtop5 wrote: post #200

Earlier quoted context omitted.

For various reasons this wouldn't work, you have the various bands of government salary https://federalnewsnetwork.com/pay/2020/01/more-top-career-e... >But for a GS-15, step 10 in Washington, his or her salary should total roughly $185,509 in 2020. But due to the federal pay ceiling, that same GS-15, step 10 in Washington will make the limit of $170,800 this year. And that's the absolute Max, after you go to the tro…

As someone with a degree in software "engineering" and more than a decade of experience in the industry I'd gladly take a stable government job creating or maintaining important applications that help the government provide high quality and necessary services to people. I would take pride in working on such a system as a vaccine administration management platform. Instead those jobs are doled out to contracting firms…

I mean I do this for the money

Re: CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

#367

Post by didibus »

NearAP wrote: post #53
didibus wrote: post #13

I just don't understand these software deals, the price is so high. At 44 million, you can hire 146 engineers each paid 300k for a full year. Trust me, you need much less to build something like this, and there's no excuse for it to suck so bad. I'm also curious, anyone know where the development actually happened? Did Deloitte further subcontracted out? Was it outsourced?

This is not a 'software engineering' thing. If you look at it from a software development POV, then the issue is more of not having 'product managers' whose job it is to - understand the requirements, write the specs, basically champion the POV of the end user (after actually identifying the correct end user). Examples a) The article talked about the software not working on Internet Explorer (IE). Older people still…

I agree with you on all those listed points, but I don't know if it's my experience as a software engineer, but all those things you point out in general are something I find developers are cogniciant of, but often the processes in place doesn't allow them to do what's best, instead they are forced to always seek approval and follow stringent specs and requirements even if they don't make sense.

If you let qualified engineers drive those instead, in my experience, the end result is always better.

Re: CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

#368

Post by gunsch »

brundolf wrote: post #166
lmilcin wrote: post #17

I have worked with Deloitte on two projects (on clients' side, I would never willingly work for Deloitte). Generally expect them to do absolute minimum necessary to meet the contract. The code was obfuscated to ensure support contract. The people employed to write it were not interested in the quality in the slightest but were interested a lot in specifics of the contract.

Serious question: why doesn't the government have its own internal software development department at this point? One that's familiar with all the regulatory and bureaoucratic hurdles and the legacy systems, but doesn't have the incentive to grift. For the amount of money they spend on these contracts surely they could afford to pay competitive salaries?

As others have mentioned already, 18F and USDS are in this space. They do some work, but their scope and manpower is limited, and they try to multiply their impact through contracting. Part of that is helping federal agencies write better contracts to be able to source the kind of contractors they want --- ones who think in terms of product development, user research ("build with, not for"), and are using modern software development practices. Bringing in contractors helps them multiply their impact, especially with how difficult hiring people into government is. (and a lot of people in tech simply didn't want to work directly under the Trump administration, which --- as liberal as the tech field tends to lean, seems to be a factor on hiring at 18F and USDS).

I work at one of the contractors in the space (Ad Hoc [1]) and get to work with people from USDS every day. Like others have commented here, the government salary ranges are a problem for them to hire enough. Our last COO had been an administrator at OPM, and would say things like "on the government side, I couldn't ever hire the kinds of software developers that we're able to here on the contractor side."

[1] https://adhoc.team/ -- we're hiring, full-time remote!

Re: CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

#369

Post by db48x »

NearAP wrote: post #53
didibus wrote: post #13

I just don't understand these software deals, the price is so high. At 44 million, you can hire 146 engineers each paid 300k for a full year. Trust me, you need much less to build something like this, and there's no excuse for it to suck so bad. I'm also curious, anyone know where the development actually happened? Did Deloitte further subcontracted out? Was it outsourced?

This is not a 'software engineering' thing. If you look at it from a software development POV, then the issue is more of not having 'product managers' whose job it is to - understand the requirements, write the specs, basically champion the POV of the end user (after actually identifying the correct end user). Examples a) The article talked about the software not working on Internet Explorer (IE). Older people still…

Exactly. Civil Engineering does this better, I think. In a civil engineering project such as a bridge, the requirements may be written down by the customer, but the engineering design of the bridge is entirely the responsibility of a single architect. You can pretty much bet that the city council who bought the bridge doesn't know a thing about physics or engineering, so if the bridge falls down it has to be the architect's fault (assuming that it was built according to the plan). This gives the architect the power to say "no" to the customer.

Re: CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

#370

Post by rorykoehler »

nataz wrote: post #284
eigenvector wrote: post #277

Earlier quoted context omitted.

Now think of the problem in reverse, from the government's side. Why are they contracting it out when all of the most difficult aspects involve integrating with government systems, policies and personnel - things that government already knows how to do? "Because we don't have the expertise to do software development." The comments in this thread, which I fully agree with by the way, just cement my belief that if gove…

Ha, I'm currently a federal government program manager, so I am thinking about it from the government's side right now. I've also been a consultant cog in a mega-corp and run my own small consulting business, so I've seen it from both sides. Tech folks underestimate non tech challenges (basically anything outside of their domain), and clients don't actually know what they want. It's an age old problem in lots of indu…

I’ve been working in enterprise/government sector for a few years now and my observation is that they need to focus on better systems design. Everything is too convoluted. Trying to make the software fit the poorly thought out and implemented legacy ERP design from yesteryear is where most of the money is wasted.
Post reply on HN