Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Exploring the software that flies SpaceX rockets and starships

stackoverflow.blog

Post reply on HN
Print view View original

101–110 of 118 posts

Re: Exploring the software that flies SpaceX rockets and starships

#101

Post by noneeeed »

plq wrote: post #2

The article is a bit light on technical details, but the following is noteworthy: > Flight software for rockets at SpaceX is structured around the concept of a control cycle. “You read all of your inputs: sensors that we read in through an ADC, packets from the network, data from an IMU, updates from a star tracker or guidance sensor, commands from the ground,” explains Gerding. “You do some processing of those to de…

Embedded high-integrity systems have traditioanlly used this kind of cycle.

It's simple to reason about and analyse. You can make assertions about worst case execution time and memory useage and you know for a fact what the order of execution is. The concurrancy on large systems like this often comes in the form of a lot of small computers talking to each other over some network/bus rather than having many systems all running on the same hardware.

Building concurrant systems with predictable real-time characteristics is hard. When you have a bunch of things that really need to happen every Nth of a second in order to fly straight, a simple approach like this is definitely preferrable. In situations like this predictability tends to be more important than raw performance (assuming you can reliably hit minimums).

That doesn't mean people don't use multi-threading in domains like this, but it's one of those things you want to keep as simple as you can and avoid where possible.

Re: Exploring the software that flies SpaceX rockets and starships

#102

Post by dralley »

hutzlibu wrote: post #45
ACS_Solver wrote: post #44

Earlier quoted context omitted.

The article's indeed light on details, but what it describes sounds very similar to autonomous driving / driver assistance software I have experience with. That's not really surprising, as the overall system for an autonomous car and a spaceship is very similar - keep processing inputs from sensors, calculate some values and manage state machines, use those results to adjust outputs, some of which may operate actuato…

"One frame could render in 20ms and the next in 25ms, which is completely fine." No it is not fine, if you want to have a real smooth gameplay. So also in game loops, you can adjust for that, by checking timediff. But sure, the difference is that, if you miss a frame in a game loop - no real rockets crash, so there is probably (and hopefully) not as much effort put into that, like they do on SpaceX.

This is just the distinction between soft real-time and hard real-time.

Re: Exploring the software that flies SpaceX rockets and starships

#103

Post by numpad0 »

icegreentea2 wrote: post #41
numpad0 wrote: post #29

Earlier quoted context omitted.

I had once thinking about this type of system, and later found that the SNMP(1988) as well as NIST/Army 4D/RCS project(1980s) had this train of thought before I was even born. Now I'm wondering why this type of distributed, synchronized, hard realtime decision making framework don't seem to exist as some Apache Foundation project or something. It just sounds right and massively useful but I can't find an implementati…

There are a few layers to your question. If you mean distributed to mean "over multiple computers", then ya... this is a really specialized field. Your ability to synchronize and maintain hard real-timeness between computers is completely dependent on IO and hardware features, and topology. This makes it much harder to make a generalized framework. We we ignore the distributed portion, then you're describing a real-t…

I mean realtime inter-node message passing in a tree topology network, with message frequency associated to edges, like shown in [1]. Each nodes might run on its own CPU clocks, but message exchanges occur realtime, progressively faster at each depth, and the system as a whole(what "Autonomous System" probably supposed to mean) runs as a slow but coherent realtime cluster.

Perhaps the idea that message passing has to happen realtime is something obvious or elementary to military/defense/aerospace from technical requirements or experiences fighting fog of war etc., but to me it was dumbfounding new and I guess it might also be for the general public?

1: https://en.wikipedia.org/wiki/File:4D-RCS_reference_model_ar...

Re: Exploring the software that flies SpaceX rockets and starships

#104

Post by [deleted] »

noneeeed wrote: post #101
plq wrote: post #2

The article is a bit light on technical details, but the following is noteworthy: > Flight software for rockets at SpaceX is structured around the concept of a control cycle. “You read all of your inputs: sensors that we read in through an ADC, packets from the network, data from an IMU, updates from a star tracker or guidance sensor, commands from the ground,” explains Gerding. “You do some processing of those to de…

Embedded high-integrity systems have traditioanlly used this kind of cycle. It's simple to reason about and analyse. You can make assertions about worst case execution time and memory useage and you know for a fact what the order of execution is. The concurrancy on large systems like this often comes in the form of a lot of small computers talking to each other over some network/bus rather than having many systems al…

[deleted]

Re: Exploring the software that flies SpaceX rockets and starships

#105

Post by HeyLaughingBoy »

bombcar wrote: post #66
jb1991 wrote: post #37

Earlier quoted context omitted.

> including garbage collection. Not only does flight control software typically never use garbage collection, it is also preferable to never allocate memory during the runtime -- all memory ever needed is allocated at program startup, and memory is not acquired or released until landing or some other major end event. Because memory issues are often the most significant causes of crashes, which you don't want to trans…

They've been known to leak memory ... https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98... >This sparked an interesting memory for me. I was once working with a customer who was producing on-board software for a missile. In my analysis of the code, I pointed out that they had a number of problems with storage leaks. Imagine my surprise when the customers chief software engineer said "Of course it leaks". H…

Reminds me of an old discussion in comp.arch.embedded (dating myself here!) about what you say when people at a party ask you what you do.

Hands down best answer was the engineer from a defense company: "I build flying robots that arrive at their destination in a really bad mood."

Re: Exploring the software that flies SpaceX rockets and starships

#106

Post by HeyLaughingBoy »

icegreentea2 wrote: post #10
devit wrote: post #5

Earlier quoted context omitted.

You could abstractly structure the computation as a graph (there are many concrete approaches to that) and only recompute the parts that change due to changed input or changed intermediate results. If you have multiple outputs you can also have a scheduling/prioritization system for the subtasks. And yes, use interrupts or multiple timers to detect only changed parts without having to compare current input to previou…

Right, but since you typically want timing guarantees (for example, "respond to change in measured acceleration within 20ms"), you'd often end up structuring and resourcing everything for the worst case (everything needs to be computed) anyways... which means that your graph optimization didn't end up saving any resources. Also, there sometimes ARE interrupts meshed in with these main loop based systems (for whatever…

That pretty well describes the board in front of me ATM. Most of the system runs on a 20ms (I think) tick to manage the sensor read/evaluate/control loops, but there's a 10kHz timer interrupt always going that handles commutation of a stepper motor if it needs to run. 99.9% of the time, the interrupt fires, sees there's nothing to do and just returns.

Re: Exploring the software that flies SpaceX rockets and starships

#107

Post by cronix »

plq wrote: post #2

The article is a bit light on technical details, but the following is noteworthy: > Flight software for rockets at SpaceX is structured around the concept of a control cycle. “You read all of your inputs: sensors that we read in through an ADC, packets from the network, data from an IMU, updates from a star tracker or guidance sensor, commands from the ground,” explains Gerding. “You do some processing of those to de…

> Wow, sounds an awful lot like a typical event loop from a game engine.

Or Arduino's loop() method, where all the code just runs in a loop "forever."

Re: Exploring the software that flies SpaceX rockets and starships

#108

Post by Teknoman117 »

plq wrote: post #2

The article is a bit light on technical details, but the following is noteworthy: > Flight software for rockets at SpaceX is structured around the concept of a control cycle. “You read all of your inputs: sensors that we read in through an ADC, packets from the network, data from an IMU, updates from a star tracker or guidance sensor, commands from the ground,” explains Gerding. “You do some processing of those to de…

The part that's way harder though is that at least in a game engine, you know the absolute world state and when you make a change to your world state, it happens exactly as you wanted.

In dynamic systems, such as a rocket, robot, etc., reality is fuzzy. Your loop is mostly about "what is my best guess at my present state" and "what is my desired next state". You make changes to try and get from A to B but you may not exactly achieve B. You may have not even been in state A. The error propagates back into your next guess of state, and this repeats forever.

Sensors like GPS give you an absolute position but they're imprecise. Inertial navigation is extremely accurate in telling you the change from a sample to the next, but as your position is the second integral of your acceleration, any error compounds quickly (admittedly the quality of the INS available to people willing to spend 10s of millions of dollars on a vehicle FAR exceeds what you'd be able to put in a car). Some rockets have even used ground based radar measurements uplinked to them to further improve position estimates. They probably still do this, I just don't have any hard data on current launchers.

Re: Exploring the software that flies SpaceX rockets and starships

#109

Post by mottosso »

crazypython wrote: post #77
TchoBeer wrote: post #74

Earlier quoted context omitted.

I think we've solved the GC vs Manual debate. Just add more RAM and explode your computer when it's done running

Really, this is just region-based memory management or actor-based memory management.

It's actually rocket-based memory management. Ba dum tss!

Re: Exploring the software that flies SpaceX rockets and starships

#110

Post by PaulDavisThe1st »

devit wrote: post #5
codeulike wrote: post #3

Earlier quoted context omitted.

I can't really think of any other way of doing it. Interrupts? That would be bonkers though.

You could abstractly structure the computation as a graph (there are many concrete approaches to that) and only recompute the parts that change due to changed input or changed intermediate results. If you have multiple outputs you can also have a scheduling/prioritization system for the subtasks. And yes, use interrupts or multiple timers to detect only changed parts without having to compare current input to previou…

Remind me not to drive a vehicle where you implemented the control systems.

This is a great example of a little knowledge being dangerous. There's a reason control systems are not implemented the same way as a browser DOM. There's a reason control systems are designed to avoid subtasks, conditional execution, interrupts (as much as possible) and complex data models. The reasons are pretty damn solid, IMO.

Conceptually, you can talk about the two domains as being somehow equivalent. That says nothing about the appropriate implementations.

Post reply on HN