Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding thread stack sizes and how Alpine is different

ariadne.space

Post reply on HN
Print view View original

21–30 of 81 posts

Re: Understanding thread stack sizes and how Alpine is different

#21

Post by bsdetector »

viraptor wrote: post #10
chrisseaton wrote: post #7

How can you write a program that runs without at least some guaranteed stack size? Are you at fault if you program doesn't run in a 1kb stack? And how do you work out what stack size your program takes from looking at the source code? I guess make sure your required stack size is not a function of input, and test against a minimum stack size.

Unless you do alloca() or dynamically sized local arrays, you can measure your stack usage in the deepest call stack. Add some space in each frame for potential instrumentation and you have your minimum. Keep in mind that this is just for thread stacks - you can set the size for them yourself, so ideally you'd always do it. Then a guaranteed minimum size becomes irrelevant.

> Add some space in each frame for potential instrumentation and you have your minimum.

On exit just scan from the maximum stack to minimum looking for non-zero.

If you have tests it should be easy to get within a few bytes of max stack used, which is probably just as good as instrumenting everything.

Re: Understanding thread stack sizes and how Alpine is different

#22

Post by kosinus »

This is one of the reasons why I'm no longer using Alpine as a base in Docker images. I ran into this limit specifically with node-sass.

But in general, the difference in image size is negligible because of shared layers, and I just don't think enough testing happens on Alpine / musl in any given stack. Even if your app runtime is tested this way, how many dependencies are?

Come to think of it, I'm not even sure why there was a push for Alpine-based Docker images at some point. Maybe it was just hype.

Re: Understanding thread stack sizes and how Alpine is different

#23

Post by CodesInChaos »

viraptor wrote: post #16
chrisseaton wrote: post #13

Earlier quoted context omitted.

> Unless you do alloca() or dynamically sized local arrays, you can measure your stack usage in the deepest call stack. How does a normal working programmer calculate the size of each of their stack frames? I'm a compiler researcher and I'd struggle to do that. How are application developers going to do it? And how do you design a program to have a deterministic maximum call stack depth? I don't think these things ar…

You have 2 options: either your functions are recursive and you can hope and pray, or they're not and you can figure out which of your functions are the bottom of the call graph. In those leaf functions you can check &local_var and compare it to pthread_attr_getstack(pthread_getattr_np()). (Of course that's not precise for many reasons.) > And how do you design a program to have a deterministic maximum call stack dep…

Dynamic dispatch (e.g. function pointers/delegates, virtual methods) is another case that makes figuring out the call graph and thus the maximum stack size difficult (via whole-program data-flow analysis) to impossible (function pointers come from outside your codebase or are constructed in ways analysis can't handle).

Re: Understanding thread stack sizes and how Alpine is different

#24

Post by creata »

viraptor wrote: post #16
chrisseaton wrote: post #13

Earlier quoted context omitted.

> Unless you do alloca() or dynamically sized local arrays, you can measure your stack usage in the deepest call stack. How does a normal working programmer calculate the size of each of their stack frames? I'm a compiler researcher and I'd struggle to do that. How are application developers going to do it? And how do you design a program to have a deterministic maximum call stack depth? I don't think these things ar…

You have 2 options: either your functions are recursive and you can hope and pray, or they're not and you can figure out which of your functions are the bottom of the call graph. In those leaf functions you can check &local_var and compare it to pthread_attr_getstack(pthread_getattr_np()). (Of course that's not precise for many reasons.) > And how do you design a program to have a deterministic maximum call stack dep…

> You have 2 options: either your functions are recursive and you can hope and pray

Or you can try to figure out the maximum number of times it'll recurse: for example, the height of a red-black tree with less than 2^64 nodes is less than 128, iirc.

Re: Understanding thread stack sizes and how Alpine is different

#25

Post by nsajko »

moring wrote: post #6

> In general, it is my opinion that if your program is crashing on Alpine, it is because your program is dependent on behavior that is not guaranteed to actually exist, which means your program is not actually portable. When it comes to this kind of dependency, the typical issue has to deal with the thread stack size limit. The wording sounds as if it is trying to assign blame for the problem. What, then, is the guar…

> The wording sounds as if it is trying to assign blame for the problem.

Yes. Sadly, people often write incorrect programs.

> What, then, is the guaranteed thread stack size?

I can't be bothered to look up the POSIX guarantees, the gist of it anyway is that it depends on the application developer and system administrator.

> A developer would obviously need to know this (and other things such as the amount of stack size required by variables, parameters and frames) to not fall into this trap of writing non-portable programs.

If you're not going to calculate the exact requirements (probably unnecessary), guess/measure a number of bytes and allocate a stack that's 50 or 100 times greater than that. That's better than ignoring the existence of the stack, anyway.

Re: Understanding thread stack sizes and how Alpine is different

#27

Post by bradleyjg »

kosinus wrote: post #22

This is one of the reasons why I'm no longer using Alpine as a base in Docker images. I ran into this limit specifically with node-sass. But in general, the difference in image size is negligible because of shared layers, and I just don't think enough testing happens on Alpine / musl in any given stack. Even if your app runtime is tested this way, how many dependencies are? Come to think of it, I'm not even sure why…

A slimmer image is better from an attack surface point of view. “Distroless” with its tree shaking takes this to its logical conclusion but when images on alpine started getting popular that wasn’t available (at least to the general public).

Re: Understanding thread stack sizes and how Alpine is different

#28

Post by pjmlp »

IshKebab wrote: post #3
sys_64738 wrote: post #2

In other words, use heap space and not stack space. This is pretty elementary in C programming.

Sure, but 128 kB is really small even if you do that properly. Seems like it would be more sensible if the stack space could just grow when required. Surely not that difficult?

Enough for a full COM binary plus one overlay section. :)

Re: Understanding thread stack sizes and how Alpine is different

#30

Post by MaxBarraclough »

chrisseaton wrote: post #13
viraptor wrote: post #10

Earlier quoted context omitted.

Unless you do alloca() or dynamically sized local arrays, you can measure your stack usage in the deepest call stack. Add some space in each frame for potential instrumentation and you have your minimum. Keep in mind that this is just for thread stacks - you can set the size for them yourself, so ideally you'd always do it. Then a guaranteed minimum size becomes irrelevant.

> Unless you do alloca() or dynamically sized local arrays, you can measure your stack usage in the deepest call stack. How does a normal working programmer calculate the size of each of their stack frames? I'm a compiler researcher and I'd struggle to do that. How are application developers going to do it? And how do you design a program to have a deterministic maximum call stack depth? I don't think these things ar…

If we forbid things like recursion (including mutual recursion), function pointers, dynamic dispatch, and unbounded use of alloca, doesn't it then follow from the call graph and the per-function worst-case stack-usage numbers (which the compiler presumably knows)? Is that mistaken, or is the difficulty in generalising this approach to where those restrictions are lifted?

I tried googling for how SPARK Ada provides assurances against exceeding stack-size limits, but I couldn't find a decent answer. I presume it does so, though.

edit: forgot about alloca

edit 2: Turns out the AdaCore folks have a tool specifically for static analysis of stack-space requirements of Ada/C/C++ code: https://www.adacore.com/gnatpro/toolsuite/gnatstack

Post reply on HN