Live data from Hacker News

Why Does DARPA Work?

benjaminreinhardt.com

21–30 of 98 posts

Re: Why Does DARPA Work?

#21
If it were possible, a closer reading of history might provide additional lessons. For instance, ARPA took a hit in the early '90s under Bush. Some silver linings, but that's a transition which might illuminate process tradeoffs.

There was (at least back then) an ongoing meta discussion about how to do better. So it might be useful to explore not just the organizational designs which were realized, but also the space of things considered. For instance, before Bush hit, there was discussion of tiny "fix that!" grants. Like there's one person who is an outlier in understanding how to do X, and their book just isn't getting finished. So rather than society waiting years on diffusion and reinvention (which is what ended up happening), it might be worth paying someone to sit outside their office, and stand on their desk, and be a forcing factor on making the book happen.

At least back then, with a failed attempt at a commerce ARPA clone, it was thought important to have a clear metric to prioritize projects. "What's better for DoD?", rather than the far less tractable "what's better for the commercial economy?".

Perhaps I missed it skimming, but a major issue has been the death valley between research and commercial impact. And attempts to address that making things even worse (ie, researchers encouraged to think commercialization, sacrifice impact by holding things close, and then commercialization generally fails, so there's no offsetting benefit). And there's the unfortunate pipeline from research to patent to unsuccessful startup to dominant company having yet more anticompetitive ammo. I wonder if it might be fruitful to broaden focus to the research pipeline rooted in ARPA? Because a successful clone would presumably again face this difficulty. And there might be some other design point that is less ARPA-like, but does impact better.

How well ARPA works comes and goes. It's not a stable equilibrium. So instead of asking how to create a successful clone, perhaps one might ask how to create something viable in the vicinity of success, and separately, how to increase time spent less distant from success?

The difference between an old-school autonomous PM, rolodex and checkbook in hand, showing up on your doorstep and saying "I've heard you interested in doing X - what would you need?", and say NSF exploratory grants of "groups with the following characteristics, may submit grants addressing the following issues, with a deadline of mumble, and the following logistics"... is really really big.

Re: Why Does DARPA Work?

#22
When you say that ‘DARPA’ “works”, what precisely do you mean by that, and how do you measure success? It seems to me that it primarily works by throwing large amounts of money at academics and then claiming credit for anything they invent, whether or not it contributes to whatever DARPA is trying to accomplish.

Re: Why Does DARPA Work?

#23
DARPA is effective, but this document paints a rosy picture that is far from the truth. I've been a part of DARPA projects both commerical, university and on the government side.

The real process is:

1. PM gets picked due to knowing someone or being a former employee.

2. The biggest test of a program isn't if it's doable or a good idea, but if its able to be transitioned to another government agency with deeper pockets.

3. Most contracts are lost before you begin writing, as people have insider information about what the PM wants. This is done through just talking with each other (remember that most of the PMs come from the same companies), and not through any other formal process.

4. DARPA has some really cool stuff, but fails to transition it well enough (leading to 2.)

DARPA is not without it's problems, but has a better track record then NSF (NIH has them beat). What is funny is that you quickly realize how much bunk there is in scientific research and how many papers are not replicated.

Re: Why Does DARPA Work?

#24
post #3

Does DARPA work(anymore)? The first example of how it works was it's "badass program managers", but all of the program managers I've met have been, to put it lightly, idiots. One of the worst was the PM in charge of the entire subject research division. It turned out, they had a graduate thesis that was entirely observing an instrument artifact (showing bad jugement). Not too long after I interacted with them, there…

Author here. It's an important question. I would argue that DARPA does not work as well as it did in the 1960's but that your experience with PMs isn't representative.

Even though it's less than other government orgs, DARPA still has more process than it used to. Additionally, the opportunity cost for people who would make excellent PMs has become steadily higher over time.

Re: Why Does DARPA Work?

#25
post #9

Earlier quoted context omitted.

...so your best anecdote about why DARPA doesn't work is that they fire incompetent PMs?

Facebooks building 8 was run by a former darpa chief and went up in smoke. Also anecdata but it was enough to make me question how good their process really is

Keyword is "former". She probably left because of this:

https://en.wikipedia.org/wiki/Regina_E._Dugan#Potential_conf...

I don't think a single person that no longer works there can represent the entire process of an agency.

If anything, Facebook's building 8, only reflects badly on her and Facebook.

Re: Why Does DARPA Work?

#26
post #7

Earlier quoted context omitted.

Yes, after being promoted to the head of the division with several people speculating that they were being groomed for even higher posts. And it took a scandal to get rid of them. Person should never had been a PM in the first place. You never hear people say "it's proof that Congress works because they ejected that pedophile (sending lewd messages to the pages) from their seat".

Hate to break it to you, but "an incompetent person rising through the ranks quickly before crashing and burning" is something that happens in pretty literally every large org. Any 60 year old organization with a large budget is going to have multiple instances of this happening.

It however matters quite a lot what it takes to removed them from such position and how often does that happen. It also depends whether there is a system that can identify such people people and minimize damage they cause. In this case apparently, it took a lot and there was no system.

Re: Why Does DARPA Work?

#28

IMO this analysis gets caught up looking for answers in process when the real problem that DARPA solves is a political-economic one. ATT didn't invent the internet not for lack of process, they were extremely innovative. They held back on innovations because they were making huge sums of money overcharging long distance mainframe-mainframe data link rates in the 50's-70's. DARPA succeeded as a trust buster. They have…

Author here. Good point about DARPA serving a political-economic role. It's a different lens on the coordination role.

Is the corporate-political landscape becoming more trust-based than the Military-Industrial Complex that Eisenhower warned about?

Re: Why Does DARPA Work?

#29

Earlier quoted context omitted.

Hate to break it to you, but "an incompetent person rising through the ranks quickly before crashing and burning" is something that happens in pretty literally every large org. Any 60 year old organization with a large budget is going to have multiple instances of this happening.

Perhaps I have a flair for the dramatic, but consider my statement that I haven't met a competent pm in the division I was in (n~5). The incentives are not that well-aligned. Think about it this way. If you're really smart, why do you become a DARPA PM, and not a PI? Yes there are a few good reasons, but the population it applies to have largely been selected out by the postdoc phase (where I would say you have had s…

What does PI mean in this context? I've worked in software projects for 20 years and not once have I stumbled upon the term.

Re: Why Does DARPA Work?

#30

Small anecdote, but good experience with DARPA and SBIR both. One thing I liked - they seemed pretty low overhead / low hassle and outcome / objective focused. That contrasts VERY strong with most govt contracting which is under allowable cost or cost reimbursement setups where the absolute most important criteria is to bill enough costs to draw contract in the right cost buckets (which can be super annoying if local…

SBIR? Totally corrupt.

Please don't ask. Can not, will not, must not tell.

I can tell you one thing. If not the best but the second best or third best proposal is accepted, IT IS a problem for the US in the long run. Side node: there is money on China!

Fell free to down vote me down. Again Please don't ask. Can not, will not, must not tell.

Post reply on HN