Live data from Hacker News

Fedora 38 LLVM vs. Team Fortress 2

airlied.blogspot.com

61–70 of 111 posts

Re: Fedora 38 LLVM vs. Team Fortress 2

#61
post #18

Earlier quoted context omitted.

> If you don't mind wasting a bit of time, you could forward size+alignment to the allocator, return the aligned version and keep a record of aligned-to-allocation mapping. (For freeing later) I'm unsure what you're proposing here - the only methods you know in the replacement allocator are operator new(size_t) and operator delete(void ). The two possible failure paths are: a = ::operator new(some size) ... ::operato…

> and then just repeatedly allocate in the hope that you will eventually get a correctly aligned value out If you preload something that patches all the new/delete interfaces, you can do this without guesswork. new(size, alignment) -> res=alloc(size+alignment) res_aligned=res+... offsets[res_aligned] = res new(size) -> alloc(size) free(ptr) -> free(offsets[ptr] || ptr) offsets.del(ptr)

Haha, you've missed the issue. The question is what does the system do when someone overrides the builtin allocator functions, but does not override all of them.

You are absolutely correct that as a developer you can have your process override the allocator functions, and that is in fact what TF has done. The problem is that they have not overridden all of the allocation functions, and so they're crashing due to mismatching allocators being used. TF2 can "easily" fix this crash by implementing the aligned new, new[], delete, and delete[] operators in their custom allocator, or by simply removing their custom allocator's override of the global new & delete operators and using a common base class to get their faster allocator.

The question we're talking about is "how does the standard library respond to this scenario in a way that maximizes correctness?".

Re: Fedora 38 LLVM vs. Team Fortress 2

#62
post #35

Unfortunately this is exactly the type of stuff that makes supporting commercial apps on linux a nightmare. Weird crashes due to weird linking of system libraries. Common distros are very adamant about dynamic linking everything in order to support the use case of "core library has vulnerability, upgrade it in place without rebuilding consuming apps." Along with a desire to avoid "dll hell" and force a single canonic…

Can Linux not trivially do the same thing as windows with LD_PRELOAD? If so why is this more of an issue on Linux than Windows? Is it really less a technical challenge and more just a matter of Linux getting less support from upstream developers?

It can be done by setting rpath to origin, even post compilation using the patchelf tool. Works great with C shared libraries. Perhaps ABI issues with C++ shared libs introduces other problems.

Re: Fedora 38 LLVM vs. Team Fortress 2

#66
post #35

Earlier quoted context omitted.

Can Linux not trivially do the same thing as windows with LD_PRELOAD? If so why is this more of an issue on Linux than Windows? Is it really less a technical challenge and more just a matter of Linux getting less support from upstream developers?

It can be done by setting rpath to origin, even post compilation using the patchelf tool. Works great with C shared libraries. Perhaps ABI issues with C++ shared libs introduces other problems.

With the warning that rpath!=runpath, both are called rpath, and which you get depends on your linker and whether you also pass -Wl,--disable-new-dtags

Runpath is the default, and also the one that is non-transitive and overridden by environment variables.

Re: Fedora 38 LLVM vs. Team Fortress 2

#67
post #61

Earlier quoted context omitted.

> and then just repeatedly allocate in the hope that you will eventually get a correctly aligned value out If you preload something that patches all the new/delete interfaces, you can do this without guesswork. new(size, alignment) -> res=alloc(size+alignment) res_aligned=res+... offsets[res_aligned] = res new(size) -> alloc(size) free(ptr) -> free(offsets[ptr] || ptr) offsets.del(ptr)

Haha, you've missed the issue. The question is what does the system do when someone overrides the builtin allocator functions, but does not override all of them. You are absolutely correct that as a developer you can have your process override the allocator functions, and that is in fact what TF has done. The problem is that they have not overridden all of the allocation functions, and so they're crashing due to mism…

I was going for "ignore the issue, let's just re-patch all alloc/free pointers, built-in or external, new or old" which I think would still work. (As long as anticheat doesn't freak out) It wouldn't suffer from inconsistencies, because you'd control all the calls again. Or is there something missing in this approach?

Re: Fedora 38 LLVM vs. Team Fortress 2

#68
post #53

Earlier quoted context omitted.

LD_PRELOAD is too global to be useful, it's hard to scope it to one process (and not child processes). macOS is better in the sense that it clears DYLD_* variables when the dynamic linker has done its work and the process starts. (Although that can also be painful when you want to run a shell script and set DYLD_* outside)

You can compile binaries with additional relative library paths in to them that will take priority over /usr/lib64

How? Maybe this should be better documented & recommended. I suppose at some point you're just statically linking with more steps - though for a problem like this it might be worth it.

Re: Fedora 38 LLVM vs. Team Fortress 2

#69

Unfortunately this is exactly the type of stuff that makes supporting commercial apps on linux a nightmare. Weird crashes due to weird linking of system libraries. Common distros are very adamant about dynamic linking everything in order to support the use case of "core library has vulnerability, upgrade it in place without rebuilding consuming apps." Along with a desire to avoid "dll hell" and force a single canonic…

Completely off base. If you want to distribute your application to users yourself (instead of letting the distro take care of that), then distribute all dependencies together with it.

Re: Fedora 38 LLVM vs. Team Fortress 2

#70
post #69

Unfortunately this is exactly the type of stuff that makes supporting commercial apps on linux a nightmare. Weird crashes due to weird linking of system libraries. Common distros are very adamant about dynamic linking everything in order to support the use case of "core library has vulnerability, upgrade it in place without rebuilding consuming apps." Along with a desire to avoid "dll hell" and force a single canonic…

Completely off base. If you want to distribute your application to users yourself (instead of letting the distro take care of that), then distribute all dependencies together with it.

Which libGL.so should I be distributing alongside my application?
Post reply on HN