Ask HN: What should a systems/low-level software engineer know?

Learn to get an intuition for how your code performs.

When is it appropriate to allocate memory from the heap? If you're in a rendering or audio processing routine in realtime context, avoid it at all cost.

Think about which parts of the code could profit from optimization. Does learning an assembler for a specific platform pay off or can the compiler do a sufficiently good job with -O3? Use profilers to identify performance bottlenecks.

Think about portability. Does the code have to compile with ancient C89 compatible compilers [Mine has to, and I would be excited to see a Rust to C transpiler]? Can you choose your compilers by yourself? Are they provided by your customers?

Not able to edit! But i meant NOT worth the switch.

Also the ability to understand and check the performance impact of these choices. For example at a old job on an embedded motor controller the previous engineer had learned 'floats are slow never use them on a microcontroller' and went and removed them from the code wherever he found them. However in the motor control update loop this meant using longs (or possibly even long longs, I don't remember) in a couple calculations. A quick check with a pin toggle showed this was significantly slower than just using floats.

I'm an ME who went into embedded systems, so I'm not sure what qualifies as low level for a web developer and if checking timing using an oscilloscope or saleae is feasible for the type of work OP wants to do. But even just looking at the assembly would have made things obvious in my example.

EDIT- This made me realize that a rudimentary understanding of the assembly language for your architecture will be very valuable. Doesn't have to be enough to actually write any code in it, but it's great to be able to take a look at what was generated when things are acting weird.

From the top of my head, these are some areas I often would like other system programmers to know better:

- Understanding how to write purely event-based programs using e.g. epoll since that will avoid lots of complexity and potential problems compared to throwing threads at everything.

- Floating point representation and gotchas ("What Every Computer Scientist Should Know About Floating-Point").

- Unit-testing as part of designing interfaces to get low coupling, sane ergonomics, and forcing some thought to error handling instead of just working in the happy-path case.

- Interrupt- and thread-safety (races, mutexes, deadlocks, orderings etc.)

- Storing state in maintainable ways (i.e. not just dumping C-structs to disk).

- Understanding basic computer architecture concepts like virtual memory, caches, and pipelines.

- Know what can be achieved with OpenMP and GPU programming using OpenCL/CUDA etc.

- Write great commit messages (What is the problem being solved, why this way, etc.)

- Basics of database and distributed systems theory like ACID properties and Lamport timestamps.

- Using regular expressions instead of lots of a complicated set of if-statements.

- Understanding networks regarding error sources and latencies.

If by low level you mean embedded systems programming, you will definetly need to be proficient in C. The other knowledge depends on the product that you will make. For example I work in the automotive industry, where you need to be interested in cars and how the different parts work. Knowledge of electrical engineering and control theory is also valuable. There is a good book about it if you wat to learn how cars works: Bosch Automotive Handbook.

If you go that path, bear in mind that you will not be involved in creating clever algorithm. When safety is involved, you need to program very simple and easy code. The complexity lies in how all the parts and ECUs are interacting with each other. On the upside, you do not need to constantly learn new languages and library, and you accumulate expert knowledge which is interesting for companies.