Live data from Hacker News

Ask HN: What should a systems/low-level software engineer know?

news.ycombinator.com

111–120 of 231 posts

Re: Ask HN: What should a systems/low-level software engineer know?

#111
I was also curious about choosing or creating a file format, when creating a low level program that outputs a lot of data.

Any recommendations or popular reading?

I realize some choices are for proprietary reasons, then compression, but that aside, there seems to be thousands of choices available.

Compared to the web, using json etc, and releasing a schema, but not necessarily creating a new file extension.

Re: Ask HN: What should a systems/low-level software engineer know?

#112
One important thing for an embedded engineer (just like any software engineer!) to know is how to select an appropriate solution for a given problem. This includes both hardware and software, and do it yourself vs off the shelf.

I think it can be summed up with some questions to ask at different points in a project:

- Should I use a microcontroller or a processor? If a microcontroller should I use a simple 8 bit or more featured 32 bit?

- Do I need an operating system like Linux, RTOS like FreeRTOS, or bare metal?

- Are there existing code modules out there to help kick start the project? Like SD card libraries, ethernet middleware, etc

- Should I design a custom PCB or look at development kits, or off the shelf electronics?

- Where should I design in flexibility in the project? What requirements can be solidified to simplify the design?

Knowing about what is out there helps pick appropriate solutions to problems, which will save the most time in the future.

Re: Ask HN: What should a systems/low-level software engineer know?

#113
some resources i find interesting and helpful learning low level system stuff: https://wiki.osdev.org/Expanded_Main_Page http://www.osdever.net/

That and documentation for the platform you are working on, in my case intel/amd manuals but not sure what system you are working on, i presume one of those though.

it's also interesting to look into assembly/ assemblers, disassemblers, perhaps look at capstone / keystone frameworks to learn about that. linkers & loaders is also interesting.

it really depends on where you want to go with low level/ system things what is relevant though, as a lot of it involves platform specific details.

Re: Ask HN: What should a systems/low-level software engineer know?

#114
post #82

Earlier quoted context omitted.

FWIW I think he's wrong. Basically, when writing C, you should write code the same as if you'd write it in C++, except that you're using C. Because sometimes that's annoying, things can get less type safe, but in your head, there's a C++ program that you're representing.

This seems like the wrong direction; C++ style projects are either more heavily indirected or make heavier use of compile-time reasoning with the type system. While you can pretend that a structure full of function pointers is a vtable (and the Linux kernel does a lot of this), it's not really the same thing. Treating C as a sort of "portable assembler" is a lot better, although it runs into UB problems (see DJB on t…

The view of C programming that I'm describing is mostly compatible with the concept of treating C as a portable assembler.

I think there is a world of wild and crazy C++ (like, boost::spirit-grade, or std::allocator-using) that you're imagining, that is not what I am thinking of. If you took C, and added vector and hash_table, added constructors and destructors so you don't have to call cleanup functions, you'd get a language which most sensible non-embedded C programs would map to, which then maps upward to C++.

Maybe some templated functions like std::min and std::max and add_checking_overflow would be nice to have too.

Edit:

An example of where I think properly written C diverges from portable assembler is things like here: https://github.com/Tarsnap/tarsnap/blob/master/libcperciva/d...

It depends on whether you think ELASTICARRAY_DECL is within the scope of portable assembler. (It gives you type safety!) (And I don't know what advanced assembly languages can offer in terms of that -- maybe they do too.)

Re: Ask HN: What should a systems/low-level software engineer know?

#115
The other posts are good, but I will give you the advice someone gave me (that I ignored) when I was considering the same thing: Consider doing it as a hobby. In general web stuff pays better. This seems crazy to me, but that's where the demand is.

Re: Ask HN: What should a systems/low-level software engineer know?

#116

From the top of my head, these are some areas I often would like other system programmers to know better: - Understanding how to write purely event-based programs using e.g. epoll since that will avoid lots of complexity and potential problems compared to throwing threads at everything. - Floating point representation and gotchas ("What Every Computer Scientist Should Know About Floating-Point"). - Unit-testing as pa…

One event driven programming framework to check out is Quantum Leaps QP framework, it models state machines easily.

The other points are solid, although I have to admit I have never used regular expressions on a microcontroller

Re: Ask HN: What should a systems/low-level software engineer know?

#117
post #111

I was also curious about choosing or creating a file format, when creating a low level program that outputs a lot of data. Any recommendations or popular reading? I realize some choices are for proprietary reasons, then compression, but that aside, there seems to be thousands of choices available. Compared to the web, using json etc, and releasing a schema, but not necessarily creating a new file extension.

My current go to for this is using SQLite. It's basically made for this purpose. If that doesn't serve, I like the idea of Apache Avro, but some of it's C++ bindings are a little lacking in my opinion.

Re: Ask HN: What should a systems/low-level software engineer know?

#118

Earlier quoted context omitted.

RAII is a disaster. Piecemeal allocation and wild jumping across the project to do all these little steps (to the point where the programmer cannot predict anymore what will happen) is not the way to go. Then all the implications like exceptions and needing to implement copy constructors, move constructors, etc. in each little structure. As to what C project doesn't just emulate RAII: Take any large C project and you…

I guess you're the yin to my yang, because I've got a compiler written in C that doesn't use any global variables at all: https://github.com/srh/kit/tree/master/phase1 It wasn't really implemented for performance, and maybe the language is more complicated -- no doubt it's a lot slower. On the other hand, I can look at any function and see what its inputs and outputs are.

My compiler isn't optimized for performance, either! I didn't do much other than expanding a linear symbol search into a few more lines doing binary symbol search. And I've got string interning (hashing).

I've mostly optimized for clean "mathematical" data structures - basically a bunch of global arrays. This approach is grounded on the realization that arrays are just materialized functions, and in fact they are often the better, clearer, and more maintainable functions. If you can represent the domain as consecutive integer values, of course. So I've designed my datastructures around that. It's great for modularity as well, since you can use multiple parallel arrays to associate diverse types of data.

But anyway, your language looks impressive I must say.

Re: Ask HN: What should a systems/low-level software engineer know?

#119
post #56

Earlier quoted context omitted.

Maybe it is just me, but I've seen this issue multiple times where a programmer used a regular expression thinking they were clever, only to have it backfire later when there was some corner case not covered by their regex (which likely would have obviously been caught if they had just taken the time to write out each case as an if statement). You're usually just gaining complexity in exchange for fewer lines of code…

Regardless of implementation (regex vs. conditionals), there should be sufficient unittesting to make sure that all the corner cases are tested. For embedded systems, the complexity tradeoff is relevant though, since a regex library will (probably) take up more code space than some conditionals.

In principle, I agree. In practice, it's easier to miss an edge or corner case in a regular expression than it is in a series of conditionals. That's just another consequence of the complexity tradeoff.

Re: Ask HN: What should a systems/low-level software engineer know?

#120

In embedded and systems-level programming, something I find indispensable is knowing what values fit in what types (for example, an 8-bit type can represent a maximum of 256 values; a 16-bit type can represent 65536 values). This comes up incredibly frequently, and having it be second nature will benefit you. A lot of the time, this gives you a starting place for how your inputs and outputs should look, what your met…

RAM is plentiful even on many microcontrollers now, but it becomes much less plentiful when you start storing thousands of data points and need to start bit packing boolean flags and reducing the integer size to squeeze as much as you can in the data structure! Or you need to stream as much of that data over a serial link as possible as quickly as possible!

So I agree, knowing how data is stored is critical.

Post reply on HN