There are a bunch of problems that result from contracting work out though.
The contracting process itself is an incredibly onerous burden with a huge number of factors other than mere technical ability (are you a disabled veteran minority-owned business?). And the companies with the fortitude and resources to actually make it through the contracting process tend to be the exact companies you don't want doing the work - it's not your scrappy 30 person startup that gets the bid.
Furthermore, those companies are the only ones that can, frankly, put up with the government's shit. The problem isn't just that the government doesn't have the technical ability or the culture to get the job done, they don't even have the ability to oversee a technical contract, and it inevitably results in weird and obtuse requirements, stumbling blocks injected from the contract coordinators, etc.
You can sometimes have a setup where one contractor executes and one contractor oversees, but anyone who's worked on a project overseen by one of those contracting management companies (DeLoitte, Accenture, etc) knows that doesn't fix anything either, and the contract coordinator isn't going to stop their bullshit either, now you just have twice as many layers of management shitting things up.
There's no magic bullet here, but management and IT services within the USG drastically need a shakeup, and increasing pay scales to match the private sector has to be there too. On top of that you basically need a top to bottom review of the regulations and SOPs. It's simply too hard to actually get work done, at every level. There is too much security theater, too little real security, too many fiefs, too many bosses, too many approval processes, too many dumb requirements that were coded into law 20 years ago when they maybe made a little sense but have long-since ceased to be best practices.
Also, contracts are for a specific and limited period of time. Often development and maintenance are under different contracts - which will be bid out separately. So the first contractor doesn't really care about anything except getting it out the door, then the USG will be super surprised that they're pulling the plug once they've met the bare minimum of the requirements, and then there's no money to actually maintain it on an ongoing basis. If there is, that money will be bid out again and probably goes to a different contractor.
Keeping it internal at least means that somebody has ownership of the project. Contracting turns everything into a game of hot potato with everyone racing to throw the potato as soon as they've done the bare minimum to accomplish their contractual obligations and get paid.