Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding thread stack sizes and how Alpine is different

ariadne.space

Post reply on HN
Print view View original

51–60 of 81 posts

Re: Understanding thread stack sizes and how Alpine is different

#51

Post by arghwhat »

kosinus wrote: post #22

This is one of the reasons why I'm no longer using Alpine as a base in Docker images. I ran into this limit specifically with node-sass. But in general, the difference in image size is negligible because of shared layers, and I just don't think enough testing happens on Alpine / musl in any given stack. Even if your app runtime is tested this way, how many dependencies are? Come to think of it, I'm not even sure why…

> Come to think of it, I'm not even sure why there was a push for Alpine-based Docker images at some point.

The continuing push is due to the smaller footprint and better security properties. And no amount of sharing makes up for the difference between a single-MB image and a GB image.

Any application can just dictate its own thread stack size. What is discussed here is a default.

Re: Understanding thread stack sizes and how Alpine is different

#52

Post by ariadneconill »

stefan_ wrote: post #37
swiley wrote: post #31

Earlier quoted context omitted.

IMO: it's nice to have weird platforms. I've caught lurking UDB and memory corruption in my programs by trying to run them on weird OSes.

There is no UDB or memory corruption exceeding the stack size. It means your platform is too small to run the program. Of course, Alpine doesn't run on platforms that are too small, they just make nonsensical changes like these that cause incompatibilities but have zero benefits.

Alpine runs on all sorts of small platforms. It is possible to run it on OpenWRT type devices.

Re: Understanding thread stack sizes and how Alpine is different

#53

Post by megous »

chrisseaton wrote: post #13
viraptor wrote: post #10

Earlier quoted context omitted.

Unless you do alloca() or dynamically sized local arrays, you can measure your stack usage in the deepest call stack. Add some space in each frame for potential instrumentation and you have your minimum. Keep in mind that this is just for thread stacks - you can set the size for them yourself, so ideally you'd always do it. Then a guaranteed minimum size becomes irrelevant.

> Unless you do alloca() or dynamically sized local arrays, you can measure your stack usage in the deepest call stack. How does a normal working programmer calculate the size of each of their stack frames? I'm a compiler researcher and I'd struggle to do that. How are application developers going to do it? And how do you design a program to have a deterministic maximum call stack depth? I don't think these things ar…

You ask a compiler, since it knows the max stack requirements of every function it compiled, if it's fixed. If it's not fixed it may give you at least the minimum.

For total depth, keeping you program simple and predictable helps. People certainly manage to do it even for large programs like Linux itself, where stack size is like 16KiB or so. https://elixir.bootlin.com/linux/v5.2/source/arch/x86/includ... and less on other archs. 8 KiB on arm https://elixir.bootlin.com/linux/v5.13-rc7/source/arch/arm/i...

Re: Understanding thread stack sizes and how Alpine is different

#54

Post by chrisseaton »

megous wrote: post #53
chrisseaton wrote: post #13

Earlier quoted context omitted.

> Unless you do alloca() or dynamically sized local arrays, you can measure your stack usage in the deepest call stack. How does a normal working programmer calculate the size of each of their stack frames? I'm a compiler researcher and I'd struggle to do that. How are application developers going to do it? And how do you design a program to have a deterministic maximum call stack depth? I don't think these things ar…

You ask a compiler, since it knows the max stack requirements of every function it compiled, if it's fixed. If it's not fixed it may give you at least the minimum. For total depth, keeping you program simple and predictable helps. People certainly manage to do it even for large programs like Linux itself, where stack size is like 16KiB or so. https://elixir.bootlin.com/linux/v5.2/source/arch/x86/includ... and less on…

But the compiler may not compile simple 'functions' as the user understands them - it may compile loop bodies, functions with other functions compiled in them, individual branches of functions, multiple versions of functions based on where they're called from...

If I tell you as a compiler writer that this loop body from this function, but with this branch and this branch outlined, but only when called from this context, takes n bytes... I don't get what most working programmers are going to usefully do with that information.

Re: Understanding thread stack sizes and how Alpine is different

#55

Post by choeger »

arghwhat wrote: post #50
kstenerud wrote: post #34

Yeah, it sucks that programs crash on your system, but this is the way of things: The popular systems get targeted and tested in-depth, the less popular systems not so much. This is NOT the developer's fault; this is pragmatism . And so the mountain must come to Mohamed. Increase Alpine's default stack size to something more in-line with the big boys.

No, they should keep their low stack size to the benefit of everyone. Diversity helps discover what is fundamentally a broken and fragile assumption that a dynamic property will always have some value. An assumption that can fail anywhere, including on the OS that was initially targeted, and will fail the moment another OS is targeted. The developer should fix their broken assumption, but is entirely free to do so by…

I absolutely agree with your point about diversity. But us advocates need to understand this "pragmatic" view and counter it properly. Normally, the argument that works is: "Do you monitor the thread stack size on $POPULAR_PLATFORM so if that changes, you won't be bitte ?

Re: Understanding thread stack sizes and how Alpine is different

#56

Post by megous »

viraptor wrote: post #5
IshKebab wrote: post #3

Earlier quoted context omitted.

Sure, but 128 kB is really small even if you do that properly. Seems like it would be more sensible if the stack space could just grow when required. Surely not that difficult?

The main thread already does that. The thread stack does not. It probably made sense on 32b with relatively limited address space... But I'm curious why we're not applying that to all threads on 64b. Reserving a few 10MB of address space per thread shouldn't be a big issue, right? (Without actually mapping those pages)

Mapping a lot of pages has some overhead, too. So if you have large stacks per thread you'll take a hit.

If you use huge pages to alleviate that, you'll waste a lot of physical memory.

Re: Understanding thread stack sizes and how Alpine is different

#57

Post by megous »

chrisseaton wrote: post #54
megous wrote: post #53

Earlier quoted context omitted.

You ask a compiler, since it knows the max stack requirements of every function it compiled, if it's fixed. If it's not fixed it may give you at least the minimum. For total depth, keeping you program simple and predictable helps. People certainly manage to do it even for large programs like Linux itself, where stack size is like 16KiB or so. https://elixir.bootlin.com/linux/v5.2/source/arch/x86/includ... and less on…

But the compiler may not compile simple 'functions' as the user understands them - it may compile loop bodies, functions with other functions compiled in them, individual branches of functions, multiple versions of functions based on where they're called from... If I tell you as a compiler writer that this loop body from this function, but with this branch and this branch outlined, but only when called from this cont…

Where's the problem? Compiler will tell you what amount of stack a function will use, if it's inlined it may not tell you for that function, but it will tell you for the function the function was inlined to, which is what matters.

If the language is complicated and has generics or whatever, the programmer will have to do more work to understand it.

It's not a huge issue in C.

Re: Understanding thread stack sizes and how Alpine is different

#58

Post by chrisseaton »

megous wrote: post #57
chrisseaton wrote: post #54

Earlier quoted context omitted.

But the compiler may not compile simple 'functions' as the user understands them - it may compile loop bodies, functions with other functions compiled in them, individual branches of functions, multiple versions of functions based on where they're called from... If I tell you as a compiler writer that this loop body from this function, but with this branch and this branch outlined, but only when called from this cont…

Where's the problem? Compiler will tell you what amount of stack a function will use, if it's inlined it may not tell you for that function, but it will tell you for the function the function was inlined to, which is what matters. If the language is complicated and has generics or whatever, the programmer will have to do more work to understand it. It's not a huge issue in C.

> Compiler will tell you what amount of stack a function will use

If you ask a compiler how much stack a function will use the answer for a non-trivial compiler for a complicated language is always going to be 'it depends...'

Re: Understanding thread stack sizes and how Alpine is different

#59

Post by megous »

IshKebab wrote: post #3
sys_64738 wrote: post #2

In other words, use heap space and not stack space. This is pretty elementary in C programming.

Sure, but 128 kB is really small even if you do that properly. Seems like it would be more sensible if the stack space could just grow when required. Surely not that difficult?

Wait until you realize OS kernel thread stack size is 8 KiB or in that range, if you think 128 KiB is small. :))

Re: Understanding thread stack sizes and how Alpine is different

#60

Post by jerf »

mjw1007 wrote: post #14

I remember reading about arguments over whether Algol should permit recursive procedures, where one side of the argument was apparently claiming that they wouldn't be possible to implement. That seems pretty strange to modern ears, but maybe the underlying point was that it isn't possible to statically know how much stack size would be required. I suppose it wouldn't have been obvious then that if you fudge the issue…

"but maybe the underlying point was that it isn't possible to statically know how much stack size would be required."

Given the time frame you're talking about, remember to be thinking in terms of kilobytes, not gigabytes. And potentially low-single-digit numbers of kilobytes. It could be in the range of hundreds of bytes dedicated to stacks at the time. Even if you could compute your maximum size it's easy to imagine people balking a the results of such a computation and think it's not worth it to even consider the possibility because you'd blow your stack so quickly it's not like there'd be any benefit to it.

Post reply on HN