The first project; the one that finished nearly on time, ran for three or four years. I joined half way through as a crispy graduate. It came in within a couple of weeks of schedule. It seemed very well managed for the whole of time on it. The project manager left that company, and went to work for Apple. When I googled him just now, all the top hits where at Microsoft. Perhaps he knows how to do software engineering. Or perhaps he got lucky that time. I don't know which.
The project developed a new CPU architecture, several variant implementations and ported several operating systems to that architecture. Amongst the more impressive feats I saw during that project:
- a compiler from assembler of architecture A to assembler of architecture B. This was necessary because we had a lot of code [~10^6 lines] in architecture A assembler
- several other compilers that handled code written for A, with no thought to portability, and produced asm for B.
- a modular emulation environment. Initially it ran on a larger example of A that had a programmable microcode feature. Later it was ported to other examples of A. Some of my colleagues were in the room when the ported OS first booted on a machine of type A with the funky microcode. As I recall, it took most of a day to reach the banner.
- my own part was very modest, but I had fun. Amongst other things, I ported a disk checking utility [think fsck] from A to B. That was easy. To test it I had to create bad blocks for it to find. So I got a SCSI 1.44 floppy drive [!], and a box of new disks. Then I formatted and scratched each disk with a paperclip, and did a disk copy.
So, that was lots of bucks, hundreds of software engineers, and some number of hardware people. I don't know how many of the latter were involved. But remember that nearly everything was custom; CPUs obviously, boards, buses [though that was fixed eventually], bent metal, power supplies. The lot.
That was one fairly large company.
I've done other large projects that had many cooperating companies, usually with one or two large partners and a number of small outfits. Those involved similar sums, but a large part of that was in the equipment deployment, land purchase and so on. Not so much fun. I did once crash the wireless data network of New Zealand because a radio [or possibly an antenna] came loose on a mast and flapped around in the wind. Every time it swept through 120 degrees, all the effected clients channel hopped, provoking an exciting synchronisation bug in my multithreaded code.
Oh, and I got paid for all of it. Not as much as I would if I'd become a banker [though it's presumptuous to assume I _could_ become a banker]. I'd quite like the money, but then again I have more fun that the bankers I know. I've built software that was used [and is used] by fairly large numbers of people [if you're reading this, then you're one of them]. This is satisfying. Even it I can't call it 'engineering' with a clear conscience. If computer programming hadn't been invented, I don't know what would have become of me. There aren't that many other occupations for people with my particular blend of what I'll call talents. If you fall into that category too, then get your laughs while you can. You're a lucky person right now.
[and since it behoves me to actually answer the question, I've never done any work for a government or defence contractor. I've no particular objection, it just hasn't come along]