Understanding thread stack sizes and how Alpine is different • Hacker News Board

Re: Understanding thread stack sizes and how Alpine is different

#61

Post by MaxBarraclough » Sat, Jun 26, 2021, 5:58 PM UTC

chrisseaton wrote: post #58

megous wrote: post #57

Earlier quoted context omitted.

Where's the problem? Compiler will tell you what amount of stack a function will use, if it's inlined it may not tell you for that function, but it will tell you for the function the function was inlined to, which is what matters. If the language is complicated and has generics or whatever, the programmer will have to do more work to understand it. It's not a huge issue in C.

> Compiler will tell you what amount of stack a function will use If you ask a compiler how much stack a function will use the answer for a non-trivial compiler for a complicated language is always going to be 'it depends...'

As I mentioned in my other comment, AdaCore's GNATstack tool appears to be capable of reporting this information conservatively but with enough accuracy to be useful.

https://www.adacore.com/gnatpro/toolsuite/gnatstack

Re: Understanding thread stack sizes and how Alpine is different

#62

Post by ohazi » Sat, Jun 26, 2021, 6:48 PM UTC

The distinction that's being characterized as "GNU/Linux" vs. just "Alpine" is confusing.

Is Alpine using some new kernel? No, it's a Linux distribution that uses the Linux kernel, albeit with some unusual defaults.

Does Alpine not have any of the GNU userspace tools? Also no, there are plenty in the Alpine package repository.

Look, I get that GNU/Linux and ”GnU pLuS LiNuX" is a loaded term and has a lot of baggage, and that everyone would like to just be rid of the whole mess, but the characterization used here had me thinking that there was some other "Alpine kernel" experimental OS project that I had missed that had nothing to do with Alpine Linux.

The word "Linux" never once follows the word "Alpine" in this article, and it discusses overcommit mode as if it's a uniquely "GNU/Linux" thing. WTF does kernel overcommit have to do with GNU?

Please just call it what it is.

Re: Understanding thread stack sizes and how Alpine is different

#63

Post by eqvinox » Sat, Jun 26, 2021, 6:49 PM UTC

> Thread-local variables are referenced with the thread_local keyword. You must include threads.h in order to use it:

  #include 
  
  void some_function(void) {
    thread_local char scratchpad[500000];  
    memset(scratchpad, 'A', sizeof scratchpad); 
  }

As an important note, thread-local storage through this keyword still isn't supported on OpenBSD. It's a serious PITA.

[——— also, copied from a reply I posted below: ———]

The autofree macro is wrong since __attribute__((cleanup)) expects a function that takes an additional level of pointer. In this case, it'll call "free(&scratchpad);". Which doesn't get you a compiler warning in C because passing a char ** as a void * is perfectly fine. But your heap is f*cked after this.

Correct way to do it is

  void free2(char **p)
  {
    free(*p);
  }
  #define autofree __attribute__((cleanup(free2)))

Re: Understanding thread stack sizes and how Alpine is different

#64

Post by eqvinox » Sat, Jun 26, 2021, 6:55 PM UTC

rburhum wrote: post #32

Why would you overcomplicate your life and use something like the autofree example, that is not even portable, if you can use the heap which is simple to understand and do? I understand that if it is a hot function you may run into memory fragmentation/performance issues, but there are some many ways to deal with that with custom allocators if it truly is a problem . This is one of those perfect examples where simple…

The autofree example uses the heap. It just makes calling free() automatic when the function returns, regardless of where it does so. It's leak protection.

It's also wrong since __attribute__((cleanup)) expects a function that takes an additional level of pointer. In this case, it'll call "free(&scratchpad);". Which doesn't get you a compiler warning in C because passing a char ** as a void * is perfectly fine. But your heap is f*cked after this.

Correct way to do it is

  void free2(char **p)
  {
    free(*p);
  }
  #define autofree __attribute__((cleanup(free2)))

Re: Understanding thread stack sizes and how Alpine is different

#65

Post by lonjil » Sat, Jun 26, 2021, 7:20 PM UTC

ohazi wrote: post #62

The distinction that's being characterized as "GNU/Linux" vs. just "Alpine" is confusing. Is Alpine using some new kernel? No, it's a Linux distribution that uses the Linux kernel, albeit with some unusual defaults. Does Alpine not have any of the GNU userspace tools? Also no, there are plenty in the Alpine package repository. Look, I get that GNU/Linux and ”GnU pLuS LiNuX" is a loaded term and has a lot of baggage,…

So in your view, having even a single GNU tool installed, or even available for installation, means you're using "GNU/Linux"? Alpine uses musl and busybox rather than the more common GNU equivalents.

Kernel overcommit has nothing to do with GNU, but default stack size has a lot to do with which libc you use. Musl has a different default than glibc. Overcommit is mentioned because it is the justification for glibc having a large stack size by default. Musl has defaults that make fewer assumptions about how the system is configured.

Re: Understanding thread stack sizes and how Alpine is different

#66

Post by lonjil » Sat, Jun 26, 2021, 7:21 PM UTC

moring wrote: post #6

> In general, it is my opinion that if your program is crashing on Alpine, it is because your program is dependent on behavior that is not guaranteed to actually exist, which means your program is not actually portable. When it comes to this kind of dependency, the typical issue has to deal with the thread stack size limit. The wording sounds as if it is trying to assign blame for the problem. What, then, is the guar…

> What, then, is the guaranteed thread stack size?

If you need a big stack, you can just ask pthread to give you one.

Re: Understanding thread stack sizes and how Alpine is different

#67

Post by lonjil » Sat, Jun 26, 2021, 7:26 PM UTC

ynik wrote: post #43

sys_64738 wrote: post #2

In other words, use heap space and not stack space. This is pretty elementary in C programming.

It's a bit ridiculous to complicate recursive algorithms just because the stack sizes haven't been increased in the past 3 decades. Nowadays we have at least 48bit virtual address space available; what's the harm in giving each thread a full GB of stack?

If you need a bigger stack, you can get one. This stuff is merely the default.

Re: Understanding thread stack sizes and how Alpine is different

#68

Post by ohazi » Sat, Jun 26, 2021, 7:34 PM UTC

lonjil wrote: post #65

ohazi wrote: post #62

The distinction that's being characterized as "GNU/Linux" vs. just "Alpine" is confusing. Is Alpine using some new kernel? No, it's a Linux distribution that uses the Linux kernel, albeit with some unusual defaults. Does Alpine not have any of the GNU userspace tools? Also no, there are plenty in the Alpine package repository. Look, I get that GNU/Linux and ”GnU pLuS LiNuX" is a loaded term and has a lot of baggage,…

So in your view, having even a single GNU tool installed, or even available for installation, means you're using "GNU/Linux"? Alpine uses musl and busybox rather than the more common GNU equivalents. Kernel overcommit has nothing to do with GNU, but default stack size has a lot to do with which libc you use. Musl has a different default than glibc. Overcommit is mentioned because it is the justification for glibc hav…

Okay, that's fair. I was confused and responded in frustration. Maybe glibc vs. musl would have been more clear than "GNU/Linux" vs. "Alpine".

I still think only referring to it as "Alpine" and not once calling it "Alpine Linux" is weird.

Re: Understanding thread stack sizes and how Alpine is different

#69

Post by jlokier » Sat, Jun 26, 2021, 7:36 PM UTC

kstenerud wrote: post #47

sys_64738 wrote: post #2

In other words, use heap space and not stack space. This is pretty elementary in C programming.

That might have been true in the old days when memory wasn't the bottleneck, but in today's world where a cache miss is catastrophic, it makes MUCH more sense to use stack space where you can. This also has the side effect of facilitating idempotent functions and function purity in general. No sense in clinging to old world ideals when they no longer make sense.

I agree that using the stack instead of heap makes sense for cache reasons.

But in the contemporary world, the trend is increasingly to transform functions to "async" forms where much of the functions' local state including return address is stored in heap-allocated space instead.

Re: Understanding thread stack sizes and how Alpine is different

#70

Post by a1369209993 » Sat, Jun 26, 2021, 8:16 PM UTC

mjw1007 wrote: post #14

I remember reading about arguments over whether Algol should permit recursive procedures, where one side of the argument was apparently claiming that they wouldn't be possible to implement. That seems pretty strange to modern ears, but maybe the underlying point was that it isn't possible to statically know how much stack size would be required. I suppose it wouldn't have been obvious then that if you fudge the issue…

> it isn't possible to statically know how much stack size would be required.

It's not just that - if it were merely that you couldn't know the size statically, you could use dynamic memory allocation. The problem with that though, is that now every (not provably nonrecursive) function call can now fail with a memory allocation error, and if your language doesn't surface that to the caller, and (correctly) doesn't allow spurious errors to appear out of nowhere (cough every modern programming language cough cough), there's no way to handle that error.