Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

technologyreview.com

Post reply on HN
Print view View original

461–470 of 688 posts

Re: CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

#461

Post by getlawgdon »

mdorazio wrote: post #52
didibus wrote: post #13

I just don't understand these software deals, the price is so high. At 44 million, you can hire 146 engineers each paid 300k for a full year. Trust me, you need much less to build something like this, and there's no excuse for it to suck so bad. I'm also curious, anyone know where the development actually happened? Did Deloitte further subcontracted out? Was it outsourced?

These contracts are generally bloated, but if you've never seen how a $10M+ contract for things like this works you're probably thinking about it wrong. The actual development piece is a pretty small portion of the overall work. The bulk of it is figuring out all the requirements, wrangling legacy systems and data into some kind of shape that can either be interfaced with or imported, documentation, support (includin…

Still not a $44mm problem.

Re: CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

#462

Post by rmason »

If I was to guess the CDC wrote poor specs. Probably never talked at length to a variety of different county health departments to understand their needs. Never had a persona written so they didn't understand the need for IE or that people might be using it on a mobile phone.

Deloitte just coded the spec, cashed their check and walked away. Deloitte probably outsourced development overseas.

I know some smaller government contractors in the D.C area who would have done it for a third as much and done some of the CDC's work for them when they saw that it wasn't done.

Re: CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

#463

Post by lmilcin »

ntsplnkv2 wrote: post #80
lmilcin wrote: post #17

I have worked with Deloitte on two projects (on clients' side, I would never willingly work for Deloitte). Generally expect them to do absolute minimum necessary to meet the contract. The code was obfuscated to ensure support contract. The people employed to write it were not interested in the quality in the slightest but were interested a lot in specifics of the contract.

They're getting paid for a deliverable. Why do you expect free work?

Imagine you are signing a contract with a car shop to add a spoiler to your car.

The car shop spends months of your time obsessing about the dimensions of the spoiler. Then the car is out of service for as much time and you start wondering what they have been doing.

Then the car arrives and there is a bunch of paint missing around the spoiler, the spoiler is completely unpainted an it falls off when you accelerate to 100mph.

So you ask, "WTF?" and they tell you that none of these things were specified in the contract.

That's more or less how it worked.

I have been contracted for a number of projects and I always assume some level of standard that my client would want me to meet.

For example, around 2003 I have been contracted by a publishing house to restore couple of their servers. Their employees made a mess they were not able to get out of, completely mangled Linux distributions, took out drives out of RAID5 and did not know what to do with them, etc. Drives had files with only copies of texts of books they had so it was extremely valuable to them.

So obviously I did what ANY sane person would do:

- I have discussed with responsible employees what exactly happened,

- We went out on a little trip to buy backup drives

- We made backups of every single drive involved in the mess (I keep saying we because I involved the guys in the recovery so they know how it works)

- We put together the drives and they worked

- We fixed OS-es which they mangled

- We tested that everything works

- We have discussed why they need some procedures and especially backup procedures

- I have showed them how to construct couple of these procedures so that they know how to do that.

I didn't have to do all that. I was already being paid handsomely for the recovery and would not suffer (other than lost pay) if I failed recovery. But I also recognize how important this data was for the company and so I took necessary precautions proportional to how important this.

Re: CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

#464

Post by notagoodidea »

zaroth wrote: post #115

> Deloitte is the only contractor that can meet the project requirements, because configuration of the VAMS application is occurring using Deloitte’s propriety GovConnect platform. Therefore, no other contractor has rights or access to leverage the system to carry out O&M activities, such as the administration of the system, change and defect management and end user support. As the only contractor with the GovConnect…

That is the easiest a company can leverage an invitation to tender ad infinitum. I believe that going for the cheapest is always the wrong way to conducting those projects. On the company side, bargain you first contract, install your proprietary program, lock everyone up in it and higher your fees when you have leverage on the next requirements.

Government, public offers, invitation to tender must not only play by the cheapest but also by the long term impact that changing contractor every 4, 5 years has on their own management and system. This is getting as absurd and stupid. Force contract to use standard/boring technology and leverage a large market of companies for support. Being the cheapest never was a fair decision maker.

Re: CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

#465

Post by that_guy_iain »

didibus wrote: post #13

I just don't understand these software deals, the price is so high. At 44 million, you can hire 146 engineers each paid 300k for a full year. Trust me, you need much less to build something like this, and there's no excuse for it to suck so bad. I'm also curious, anyone know where the development actually happened? Did Deloitte further subcontracted out? Was it outsourced?

My first IT job was with a hosting company that noticed how much the goverment contracts were bloated. They then spent ages trying to become an offical supplier so they could undercut companies charging so much more. Literally, one of the issues they had was that so many people were curious as to how the company could make a profit since the prices were so much lower than all the other contractors like Serco.

We also ended up being in contact with Serco about a tender they had to redo one of the goverment websites. We ended up not getting any work from that because Serco was under pressure to stop outsourcing all their contracts.

Goverment contracts just send lots of money to companies who already have lots of money.

Re: CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

#466

Post by juliendorra »

brtkdotse wrote: post #415
aasasd wrote: post #400

Earlier quoted context omitted.

Reads like a copy of Russian government contract practices. Put up a contest, list requirements in such a way that only one company would satisfy them without going bust or mad, award the contract to that company. ‘Make the thing using this and that technologies’ does appear regularly.

There was some hubbub in Sweden a while back about the armed forces putting up a _really_ specific job ad, like it was tailored to one person. Well, it turned out it was, because the person posting the add forgot to take out the phrase “Make sure Björn meets these requirements” in the middle of the ad.

It’s an open secret in France that many, maybe most (?) calls for new teaching/research positions in university labs are designed to be won by someone already attached to the lab (like a doctoral student for example, or someone with a temporary contract).

In French it’s called being "fléché", as in attributed for someone. It’s of course totally contrary to the rules. (It’s most of the time somehow well intentioned, from the point of view of labs doing it, like there’s this great person doing a lot of work and they deserve this new opening the lab fought to get budgeted.)

The sad part is that some poor naive young researchers will prepare and cross the country to interview (hopefully now there will be more video calls) not knowing that the call is already custom fitted to someone in the lab.

The fun part is that, as there has to be a majority of external jury in the commission appraising the candidates, the external jury sometimes don’t even know that the call is rigged and pick another, clearly better candidate Then, a few months later the candidate starts their new job and is really puzzled at the attitude of this one person… the one that was supposed to win the call

Re: CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

#467

Post by barrkel »

that_guy_iain wrote: post #465
didibus wrote: post #13

I just don't understand these software deals, the price is so high. At 44 million, you can hire 146 engineers each paid 300k for a full year. Trust me, you need much less to build something like this, and there's no excuse for it to suck so bad. I'm also curious, anyone know where the development actually happened? Did Deloitte further subcontracted out? Was it outsourced?

My first IT job was with a hosting company that noticed how much the goverment contracts were bloated. They then spent ages trying to become an offical supplier so they could undercut companies charging so much more. Literally, one of the issues they had was that so many people were curious as to how the company could make a profit since the prices were so much lower than all the other contractors like Serco. We also…

Serco

Re: CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

#468

Post by imtringued »

smsm42 wrote: post #430
8bitsrule wrote: post #418

Earlier quoted context omitted.

Geez, I thought the whole idea is that 'government is the problem. We need to let the private sector handle all these things.' Making the CDC look bad appears to have been part of that program last year. Shame the price has been so high.

Well, the thing is that when the government takes tax money and allocates it to private companies via obscure and arguably corrupt procedures, you get the worst of both worlds. Inefficiency and rigidity of the government, combined with opacity and greed of the private companies. When people say privatizing things could be a solution, they don't usually mean "take tax money and award it through a no-bid contract to a…

Isn't the problem that the compensation structure is wrong?

By paying lump sums you are basically encouraging your contractor to screw you over. You should compensate them based on metrics that align with your goals. For example, pay $1 (random number) for each vaccinated citizen.

Re: CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

#469

Post by GordonS »

blackrock wrote: post #307
mdorazio wrote: post #52

Earlier quoted context omitted.

These contracts are generally bloated, but if you've never seen how a $10M+ contract for things like this works you're probably thinking about it wrong. The actual development piece is a pretty small portion of the overall work. The bulk of it is figuring out all the requirements, wrangling legacy systems and data into some kind of shape that can either be interfaced with or imported, documentation, support (includin…

It sounds like they needed a senior business systems analyst to map out the functional requirements specifications document. Usually, though not always, this helps to give a road map for the organization to follow. Bugs will arise of course, but usually, it becomes localized, and can be addressed individually. But for large systems, with lots of moving parts and data integrations with other systems, then you normally…

I work for a megacorp outsourcer, and when we run big projects, we embed Business Analysts in the business, so they are working alongside the key stakeholders/decision makers for the duration of the project.

A recent project was valued at around $10M USD, but for that the customer got a fully-featured mobile app that their entire blue-collar workforce will use every single day, replacing gigantic paper work packs.

The backend was hybrid-cloud, interfacing with multiple on-prem systems, including a legacy ERP system which had started life in the early 80's.

So, it was a big and complex project - but it was also a successful one, because we know how to run projects like this. There were various factors that led to success. The BAs were probably most integral. We also held regular week-long workshops with the customer to work on the backlog, with the architect/tech lead (me) and dev team leads all present too. There was a group of pilot users, that always had access to the latest features. And of course, we used automated tests.

Any big corp, like Deloitte, should be able to deliver a relatively simple website, even if it has multiple integrations with legacy systems. And they should be able to advise and steer the customer if they are woolly on what they actually need - there is only so much blame you can put at the customer's door when systems like this are a disaster.

Re: CDC website built by Deloitte at a cost of $44M is abandoned due to bugs

#470

Post by kuschku »

closeparen wrote: post #420
zaroth wrote: post #115

> Deloitte is the only contractor that can meet the project requirements, because configuration of the VAMS application is occurring using Deloitte’s propriety GovConnect platform. Therefore, no other contractor has rights or access to leverage the system to carry out O&M activities, such as the administration of the system, change and defect management and end user support. As the only contractor with the GovConnect…

For your next laptop, car, etc. try doing what you think the government ought to do here. Write down the requirements, give them to a neutral third party who doesn’t doesn’t use computers/drive, and ask them to figure out the cheapest thing that meets the requirements. Commit yourself to buying only that. See how it goes for you.

But that's how I procure anything and everything. I go to geizhals.eu, select the category, select my requirements based on comparative testing from test.de, sort by cheapest total, and click buy.

Every single piece of electronics I own was bought that way.

How else would you buy stuff without being influenced by some subconscious bias caused by some stupid ad?

Post reply on HN