Live data from Hacker News

Understanding thread stack sizes and how Alpine is different

ariadne.space

11–20 of 81 posts

Re: Understanding thread stack sizes and how Alpine is different

#11
post #3

In other words, use heap space and not stack space. This is pretty elementary in C programming.

Sure, but 128 kB is really small even if you do that properly. Seems like it would be more sensible if the stack space could just grow when required. Surely not that difficult?

A typical stack frame is around a couple dozen words. Let's round that up to 32 words (256 bytes). 128K is enough for a 500 deep stack at that size. 128K is huge.

Re: Understanding thread stack sizes and how Alpine is different

#12
post #3

In other words, use heap space and not stack space. This is pretty elementary in C programming.

Sure, but 128 kB is really small even if you do that properly. Seems like it would be more sensible if the stack space could just grow when required. Surely not that difficult?

Since standard OS stacks are contiguous and unmovable, you can't grow them once they run out of space. However while the address space for the maximum stack size gets reserved, each page only requires backing memory once it's first used. So as far as physical memory consumption is concerned, typical stacks act as growable with a fixed maximum size. Since address space is huge on 64-bit systems, choosing a large stack size is cheap on such systems (at least of they allow over-commit).

For the main thread, a system can also try to keep other allocations far from the stack without committing to any particular size (heap grows upwards from the bottom of the address space, stack downwards from the top). But this doesn't scale to multiple threads and leads to an unpredictable maximum stack size, so I prefer the fixed reserved space approach.

Re: Understanding thread stack sizes and how Alpine is different

#13

How can you write a program that runs without at least some guaranteed stack size? Are you at fault if you program doesn't run in a 1kb stack? And how do you work out what stack size your program takes from looking at the source code? I guess make sure your required stack size is not a function of input, and test against a minimum stack size.

Unless you do alloca() or dynamically sized local arrays, you can measure your stack usage in the deepest call stack. Add some space in each frame for potential instrumentation and you have your minimum. Keep in mind that this is just for thread stacks - you can set the size for them yourself, so ideally you'd always do it. Then a guaranteed minimum size becomes irrelevant.

> Unless you do alloca() or dynamically sized local arrays, you can measure your stack usage in the deepest call stack.

How does a normal working programmer calculate the size of each of their stack frames? I'm a compiler researcher and I'd struggle to do that. How are application developers going to do it?

And how do you design a program to have a deterministic maximum call stack depth?

I don't think these things are as easy as you're making out.

Re: Understanding thread stack sizes and how Alpine is different

#14
I remember reading about arguments over whether Algol should permit recursive procedures, where one side of the argument was apparently claiming that they wouldn't be possible to implement.

That seems pretty strange to modern ears, but maybe the underlying point was that it isn't possible to statically know how much stack size would be required.

I suppose it wouldn't have been obvious then that if you fudge the issue for the first thirty years or so, everyone will just accept that this is the way the world is.

Still, it's a bit of a shame that there are still widely-used systems where if you exceed the available stack space you're likely to face a "weird crash" rather than a clean error message at runtime.

Re: Understanding thread stack sizes and how Alpine is different

#15

Earlier quoted context omitted.

Unless you do alloca() or dynamically sized local arrays, you can measure your stack usage in the deepest call stack. Add some space in each frame for potential instrumentation and you have your minimum. Keep in mind that this is just for thread stacks - you can set the size for them yourself, so ideally you'd always do it. Then a guaranteed minimum size becomes irrelevant.

> Unless you do alloca() or dynamically sized local arrays, you can measure your stack usage in the deepest call stack. How does a normal working programmer calculate the size of each of their stack frames? I'm a compiler researcher and I'd struggle to do that. How are application developers going to do it? And how do you design a program to have a deterministic maximum call stack depth? I don't think these things ar…

Not easy at all.

I know that in the small-embedded world, people do work on such things.

Eg https://github.com/japaric/cargo-call-stack

Re: Understanding thread stack sizes and how Alpine is different

#16

Earlier quoted context omitted.

Unless you do alloca() or dynamically sized local arrays, you can measure your stack usage in the deepest call stack. Add some space in each frame for potential instrumentation and you have your minimum. Keep in mind that this is just for thread stacks - you can set the size for them yourself, so ideally you'd always do it. Then a guaranteed minimum size becomes irrelevant.

> Unless you do alloca() or dynamically sized local arrays, you can measure your stack usage in the deepest call stack. How does a normal working programmer calculate the size of each of their stack frames? I'm a compiler researcher and I'd struggle to do that. How are application developers going to do it? And how do you design a program to have a deterministic maximum call stack depth? I don't think these things ar…

You have 2 options: either your functions are recursive and you can hope and pray, or they're not and you can figure out which of your functions are the bottom of the call graph.

In those leaf functions you can check &local_var and compare it to pthread_attr_getstack(pthread_getattr_np()). (Of course that's not precise for many reasons.)

> And how do you design a program to have a deterministic maximum call stack depth?

If you're running only your code - don't use recursion, or alloca. If you use external libraries, you have to research what they do and add some extra in case of updates.

Bounded stack size is also a common issue if you're targeting small microprocessors.

For non-critical apps it should be pretty easy to figure out the needed stack size. For cases when you want to guarantee it... that gets more tricky.

Edit: just learned that clang has the option -fstack-usage which should help a lot.

Re: Understanding thread stack sizes and how Alpine is different

#18

How can you write a program that runs without at least some guaranteed stack size? Are you at fault if you program doesn't run in a 1kb stack? And how do you work out what stack size your program takes from looking at the source code? I guess make sure your required stack size is not a function of input, and test against a minimum stack size.

If you have to work in an environment where stack size is very limited (typically a few KiB) you have to pay attention to certain things that you can brush away in more generous environment. In particular you need to be very careful with recursive functions and you probably want to use the heap or static storage for any object bigger than a couple dozens bytes.

But in my experience you don't really compute a "guaranteed" stack size, you use your experience and knowledge of the program to make an educated guess, and then you apply a reasonable multiplier to give you some security margin.

If you don't use (or severely limit) recursive calls you can usually just check that your deepest call stack fits within the bounds. Although finding the deepest call stack in the first place can be tricky given that compilers can aggressively inline function calls.

Re: Understanding thread stack sizes and how Alpine is different

#19

How can you write a program that runs without at least some guaranteed stack size? Are you at fault if you program doesn't run in a 1kb stack? And how do you work out what stack size your program takes from looking at the source code? I guess make sure your required stack size is not a function of input, and test against a minimum stack size.

In general, you just can't. This means that any function call in C can bust the stack, unfortunately. You can try to use heuristics to try to avoid using up large amounts of space (avoid alloca and large stack arrays, be careful about recursion) but other than that there isn't much you can do.

Re: Understanding thread stack sizes and how Alpine is different

#20

How can you write a program that runs without at least some guaranteed stack size? Are you at fault if you program doesn't run in a 1kb stack? And how do you work out what stack size your program takes from looking at the source code? I guess make sure your required stack size is not a function of input, and test against a minimum stack size.

GCC has “-fstack-usage”
Post reply on HN