I'm annoyed by the fact that I have to visit "new" reddit to be able to accept cookies, and then go back to the old design. This is on desktop btw.
Wrong thread. You probably want https://news.ycombinator.com/item?id=27641366.
Understanding thread stack sizes and how Alpine is different
71–80 of 81 posts
Re: Understanding thread stack sizes and how Alpine is different
#72This is one of the reasons why I'm no longer using Alpine as a base in Docker images. I ran into this limit specifically with node-sass. But in general, the difference in image size is negligible because of shared layers, and I just don't think enough testing happens on Alpine / musl in any given stack. Even if your app runtime is tested this way, how many dependencies are? Come to think of it, I'm not even sure why…
At $WORK, there's a process for automatically scanning docker images for packages that have CVEs against them. Any docker image that includes glibc instantly shoots to the top of the charts, mostly because of a boatload of high or critical severity CVEs relating to bugs in asm-implemented functions on platforms like ARM, POWER9, etc. Everything in our company runs on x86, but the CVE scanning tool is dumb, so a switch to alpine was heavily encouraged.
This broke teams that rely on python and on node, but the docker image guidelines come from a team whose ideal language is now go (and most of whose legacy code is in java), so they are not really sensitive to those concerns. Ironically we tried to move to distroless as implemented by google[1], but that's based on debian which includes glibc, so the un-nuanced CVE checker freaks out again. That effort was quietly dropped.
(I'm not actually disputing the proposition that alpine is better for security under certain circumstances, but I think a lot of "the push" comes from what might uncharitably be described as cargo culting, or with more insight as interpretations that make sense in one context [everything is a static binary, little to no reliance on traditional userland tools] being unquestioningly extended to other contexts.)
Re: Understanding thread stack sizes and how Alpine is different
#73Earlier quoted context omitted.
Unless you do alloca() or dynamically sized local arrays, you can measure your stack usage in the deepest call stack. Add some space in each frame for potential instrumentation and you have your minimum. Keep in mind that this is just for thread stacks - you can set the size for them yourself, so ideally you'd always do it. Then a guaranteed minimum size becomes irrelevant.
> Add some space in each frame for potential instrumentation and you have your minimum. On exit just scan from the maximum stack to minimum looking for non-zero. If you have tests it should be easy to get within a few bytes of max stack used, which is probably just as good as instrumenting everything.
It's possible, but you need to watch out for some cases. For example let's say your furthest function declares char foo[4096], but uses only a few bytes of it in your testing. Your measurement will be 4k short.
Re: Understanding thread stack sizes and how Alpine is different
#74Makes you wonder what else they can tweak that would be correct according to one spec or another but would still trip up people who assumed reasonable defaults, and why they don't do that as well.
Re: Understanding thread stack sizes and how Alpine is different
#75Earlier quoted context omitted.
Where's the problem? Compiler will tell you what amount of stack a function will use, if it's inlined it may not tell you for that function, but it will tell you for the function the function was inlined to, which is what matters. If the language is complicated and has generics or whatever, the programmer will have to do more work to understand it. It's not a huge issue in C.
> Compiler will tell you what amount of stack a function will use If you ask a compiler how much stack a function will use the answer for a non-trivial compiler for a complicated language is always going to be 'it depends...'
Original article is about C compiled programs, not about arbitrary languages. It's not a huge problem in C.
Re: Understanding thread stack sizes and how Alpine is different
#76> In general, it is my opinion that if your program is crashing on Alpine, it is because your program is dependent on behavior that is not guaranteed to actually exist, which means your program is not actually portable. When it comes to this kind of dependency, the typical issue has to deal with the thread stack size limit. The wording sounds as if it is trying to assign blame for the problem. What, then, is the guar…
> The wording sounds as if it is trying to assign blame for the problem.
Yes. It sound like someone that have been in a million discussion about why "that your program crashes is not a bug in our operating system".
Many people, even engineers, expect things to behave as they usually do instead of behave as specified. So, I can imagine how all that discussions go.
Re: Understanding thread stack sizes and how Alpine is different
#77Yeah, it sucks that programs crash on your system, but this is the way of things: The popular systems get targeted and tested in-depth, the less popular systems not so much. This is NOT the developer's fault; this is pragmatism . And so the mountain must come to Mohamed. Increase Alpine's default stack size to something more in-line with the big boys.
I would at least make it equal to MacOS, another very popular target where things are tested a lot. That’s 512KiB. 128 is teeny.
Windows is also popular. Maybe change the process and thread stack sizes to 1MiB each? That's probably just as bad of a solution though, IMO. This article shows that assumptions are being taken for granted; no developer should expect two libc implementations to behave identically. Packagers for distros should be aware of this, too.
Re: Understanding thread stack sizes and how Alpine is different
#78Earlier quoted context omitted.
> Add some space in each frame for potential instrumentation and you have your minimum. On exit just scan from the maximum stack to minimum looking for non-zero. If you have tests it should be easy to get within a few bytes of max stack used, which is probably just as good as instrumenting everything.
It's possible, but you need to watch out for some cases. For example let's say your furthest function declares char foo[4096], but uses only a few bytes of it in your testing. Your measurement will be 4k short.
It has to be a leaf function or there would be stack frames after, and no local variables after the array, and the function doesn't push anything onto the stack, and it has to be possible in ways not covered by tests to actually use that space.
Sure, that could happen.
But what the other guy was saying about being a compiler developer and being unsure how to calculate the maximum depth is that there are many, many ways to arrive at the wrong result. Resursion, argv/envp, varags, alloca, and so on. So unless you are going to spend a great deal of energy proving maximum depth you're going to be using an estimate of some sort. Thus, 'probably just as good'.
Re: Understanding thread stack sizes and how Alpine is different
#79Earlier quoted context omitted.
> Compiler will tell you what amount of stack a function will use If you ask a compiler how much stack a function will use the answer for a non-trivial compiler for a complicated language is always going to be 'it depends...'
As I mentioned in my other comment, AdaCore's GNATstack tool appears to be capable of reporting this information conservatively but with enough accuracy to be useful. https://www.adacore.com/gnatpro/toolsuite/gnatstack
The tool will simply freak out when there is any recursion or indirect call. It's not helpful since it happens everywhere in standard libraries and non trivial programs (forget about sorting, any regex, parsing source code, linting, etc...).
The only use case is for real-time embedded code -typically in aerospace- where the coding guidelines prohibits you from using any recursion at all and you have to prove the highest stack usage fits into the chosen microcontroller.
Re: Understanding thread stack sizes and how Alpine is different
#80Why would you overcomplicate your life and use something like the autofree example, that is not even portable, if you can use the heap which is simple to understand and do? I understand that if it is a hot function you may run into memory fragmentation/performance issues, but there are some many ways to deal with that with custom allocators if it truly is a problem . This is one of those perfect examples where simple…
The autofree example uses the heap. It just makes calling free() automatic when the function returns, regardless of where it does so. It's leak protection. It's also wrong since __attribute__((cleanup)) expects a function that takes an additional level of pointer. In this case, it'll call "free(&scratchpad);". Which doesn't get you a compiler warning in C because passing a char ** as a void * is perfectly fine. But y…
The proposed autofree example uses the heap with (unnecessary) preprocesor directives... but the problem is that it is not portable as written, vs simply using the heap like a normal person.