Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pentagon fails audit for sixth year in a row

breakingdefense.com

Post reply on HN
Print view View original

91–100 of 279 posts

Re: Pentagon fails audit for sixth year in a row

#91

Post by dragonwriter »

nradov wrote: post #74
encoderer wrote: post #45

Earlier quoted context omitted.

I can imagine a future where new software is, mainly, written by humans while legacy software is maintained by ai. It can take months for a new developer to understand a legacy code base but an LLM with a big enough context window would be instantly productive.

The hard part of accounting systems is doing the requirements analysis (including interviewing humans who give inaccurate or incomplete answers) and writing detailed functional specifications which account for every possible edge case. No LLM can do that. Once you have the detailed specifications the coding is relatively easy. Some of that can be partially automated using AI CASE tools, but that only gives a marginal…

> The hard part of accounting systems is doing the requirements analysis (including interviewing humans who give inaccurate or incomplete answers) and writing detailed functional specifications which account for every possible edge case. No LLM can do that.

One of my first one afternoon (for prompt refinement and trying it out) toy GPT-3.5 “applications”—really just a prompt—was having it interview for reauirements and draft and progressively refine a requirements doc in a specified (by reference to an author in popular Agile literature, not by specific templates, so just relying on the model’s general training) format. Its pretty much what convinced me of the broad utility of the technology.

I absolutely think that with a bit of effort, GPL-4-128K, a custom RAG framework and/or the Assistants API, you could build something that handles a lot of this. I’m not really bullish near term on LLMs as full replacements, but I can see it as a big force multiplier.

Re: Pentagon fails audit for sixth year in a row

#93

Post by jfoutz »

I know I'm incredibly naive, and there's no political will to actually make the system work.

It seems like various sections and departments are passing audits. If you're within, say, 1% just allocate the funds.

If you didn't pass the audit, fill out this form and then get the money. The form can be classified.

I have no idea, but I'd imagine like payroll and taxes are pretty much perfect.

Once you get a handle on who doesn't know where their money is going, you can help them figure it out, throughout the year, and be on track next year.

It's huge, it's massive it's trillions of dollars, it's a completely incomprehensible system. But, like, some parts are comprehensible. Those parts are easy. The incomprehensible parts can be broken down and figured out. Is divide and conquer not a thing?

Re: Pentagon fails audit for sixth year in a row

#94

Post by ryandrake »

pstuart wrote: post #50
marricks wrote: post #37

Earlier quoted context omitted.

Eisenhower warned this would be a thing and here we are, it's a thing. The best we will ever get is the illusion of spending being under control. It's wild, but whenever a big change occurs there's always smart folks who knew what was going to happen and exactly called it out. Usually as a "ok guys be careful cause X could happen" and X inevitably does.

Equally wild is when "fiscal conservatives" complain about spending, it's only on social programs they disagree with. Fraud and/or waste should be a subject of concern regardless of the program that experiences it.

I wouldn't pin this on any particular party or ideology. Both major parties in the USA support dumping infinite unaccountable money into the military. It's one of the few things they agree on.

Re: Pentagon fails audit for sixth year in a row

#95

Post by NovemberWhiskey »

warner25 wrote: post #81
wolverine876 wrote: post #64

Earlier quoted context omitted.

I'm not an insider, but those sound like generally hard-to-avoid issues of large organizations, public and private, profit-oriented and service-oriented. You can't manage that many people, millions in DoD, without rules, and rules can't be defined to apply well to the very wide variety of situations in reality - just like an algorithm, they'll be inflexible. These kind of gripes are, understandably, in every large or…

Yeah, it just seems like we've spent an inordinate amount of time on things that would either (1) save what amounts to peanuts in the context of the Army and DoD budgets or (2) not save any money at all. I guess the primary purpose of the audit is to make all the numbers add up, and maybe that can inform cost-saving decisions down the road, like you said about warehouse storage space(?). I'd rather see a number of "e…

>I guess the primary purpose of the audit is to make all the numbers add up, and maybe that can inform cost-saving decisions down the road

That's right - the, not unreasonable basis, is that you can't make informed decisions about whether you're getting good value for money, or how to change allocations, unless you've transparency into what that money actually does.

It's approximately the same reason why financial statements and their audits are relevant in the private sector - just substitute "investors" for the "the public, with Congress as proxy".

Re: Pentagon fails audit for sixth year in a row

#96

Post by thelastgallon »

bogota wrote: post #43
MrMetlHed wrote: post #38

Worked with a guy ten years ago that would obsessively read these reports from the Pentagon. Here's a couple of his stories about the matter: https://www.reuters.com/investigates/pentagon/ and https://www.reuters.com/investigates/special-report/usa-mari... (very old and I apologize for my horrific javascript at the time.) It sounds like nothing has changed. Relevant to this crowd, maybe: "Most of the Cobol code the P…

Start paying 250k a year for cobol developers and the problem fixes itself really fast. That or start migrating the system. Or more likely they keep using it until it completely breaks and then run to the government for a big handout to fix it. Easy to be irresponsible with other peoples money.

Incentives only work on low-skilled labor that is quantifiable. Eg. more money for flipping more burgers. It is not a settled science paying people a lot more will automagically solve problems. People crave Autonomy, Mastery & Purpose: https://www.youtube.com/watch?v=rbR2V1UeB_A

What is definitely going to happen is you will attract the cowboys who have convinced themselves and can convince you to pay top dollar. They will come up with lets do kubernetes + service-mesh + multi-cloud hybrid + blockchain + crypto + AI/MI on this COBOL ... They will take top dollar, make a bigger pile of mess, sell it as a success story and move on make bigger messes.

Re: Pentagon fails audit for sixth year in a row

#97

Post by whatshisface »

thelastgallon wrote: post #96
bogota wrote: post #43

Earlier quoted context omitted.

Start paying 250k a year for cobol developers and the problem fixes itself really fast. That or start migrating the system. Or more likely they keep using it until it completely breaks and then run to the government for a big handout to fix it. Easy to be irresponsible with other peoples money.

Incentives only work on low-skilled labor that is quantifiable. Eg. more money for flipping more burgers. It is not a settled science paying people a lot more will automagically solve problems. People crave Autonomy, Mastery & Purpose: https://www.youtube.com/watch?v=rbR2V1UeB_A What is definitely going to happen is you will attract the cowboys who have convinced themselves and can convince you to pay top dollar. The…

250k is not top dollar, it is average for experienced engineers.

Re: Pentagon fails audit for sixth year in a row

#98

Post by SteveNuts »

uticus wrote: post #44
MrMetlHed wrote: post #38

Worked with a guy ten years ago that would obsessively read these reports from the Pentagon. Here's a couple of his stories about the matter: https://www.reuters.com/investigates/pentagon/ and https://www.reuters.com/investigates/special-report/usa-mari... (very old and I apologize for my horrific javascript at the time.) It sounds like nothing has changed. Relevant to this crowd, maybe: "Most of the Cobol code the P…

Of course, devil's advocate would point out COBOL in 60s was written with a lot more focus on lasting in general. They didn't largely have the same general attitude of 'we iterate quickly with our script kiddies' as is prevalent today. Not that all software from then was wonderful. Or that COBOL magically guarantees great code. But, on the flip side, you can't say the code base isn't battle-hardened (to various liter…

It'd be battle hardened to 1960s standards, payroll has changed a _lot_ since then. And if they can't accurately describe exactly how certain calculations are made (or have to do some of those manually outside of the program after it runs), that would be a huge problem in an audit.

This is why big companies use Oracle or SAP for their ERP, auditors are very familiar with it and the way it functions is well-known and understood.

Re: Pentagon fails audit for sixth year in a row

#99

Post by SteveNuts »

encoderer wrote: post #55
Yahivin wrote: post #53

Earlier quoted context omitted.

I can imagine the opposite.

Cool tell me more.

Not OP but I could see some shops pushing AI generated code to production, then when changes need to be made, they can't get the AI to modify the existing code in just the way they need, so a human has to intervene.

Re: Pentagon fails audit for sixth year in a row

#100

Post by TinyRick »

jfoutz wrote: post #93

I know I'm incredibly naive, and there's no political will to actually make the system work. It seems like various sections and departments are passing audits. If you're within, say, 1% just allocate the funds. If you didn't pass the audit, fill out this form and then get the money. The form can be classified. I have no idea, but I'd imagine like payroll and taxes are pretty much perfect. Once you get a handle on who…

There's no political will to actually make the system work because the system is working perfectly fine for those in charge.
Post reply on HN