Swift Achieved Dynamic Linking Where Rust Couldn't (2019) • Hacker News Board

Re: Swift Achieved Dynamic Linking Where Rust Couldn't (2019)

#161

Post by eschaton » Mon, Jan 30, 2023, 7:47 AM UTC

alexvoda wrote: post #155

lamontcg wrote: post #143

Earlier quoted context omitted.

I did it for 10+ years at my last job, you need a build system that hammers on everything really hard to set the rpath on everything, you shouldn't need wrappers. It definitely isn't idiomatic though.

What is the philosophy behind why it's not done like this Linux? Also, what about Nix?

You assume there’s a philosophy or coherent reasoning behind it, rather than “This is the way we did it with static libraries, so when we adopted shared/dynamic libraries we didn’t change anything else.” Because near as I can tell that’s exactly what happened when BSD and Linux implemented Sun-style .so support in the early 1990s, and there hasn’t been any attempt to rethink anything since then.

Re: Swift Achieved Dynamic Linking Where Rust Couldn't (2019)

#162

Post by pjmlp » Mon, Jan 30, 2023, 7:50 AM UTC

greyw wrote: post #31

Dynamic linking is in my opinion not that useful anymore in this day and age where the few MB of RAM and disk space you save is not worth the hassle. The amount of dynamic linking issues I encountered on GNU/Linux was insane (fuck libstdc++). Not even glibc manages to keep forward compatibility working (breaking memcpy, breaking DT_HASH, ...)! It's much better to just statically link your binaries (unfortunately many…

Dynamic linking great for plugins and OS extensions, which otherwise will require tons of processes doing IPC, with serialized data.

Great if you have hardware to spare and enjoy Electron like experiences.

The only plus side is being safer to the host application.

Re: Swift Achieved Dynamic Linking Where Rust Couldn't (2019)

#163

Post by pjmlp » Mon, Jan 30, 2023, 7:53 AM UTC

josephg wrote: post #134

wtetzner wrote: post #109

Earlier quoted context omitted.

It depends on what kind of efficiency you care about more. Static linking can allow optimizations across library boundaries.

Yep. For hot call sites, these optimizations & inlining opportunities make a massive difference to performance. Static linking also allows for faster application startup time. (Though I don't have an intuition for exactly how slow dynamic linking is). The only argument for dynamic linking being more efficient is that each dynamic library can be shared between all programs that use it. But not a net win in all cases.…

Dynamic linking also allows for extensible applications that have to otherwise be implemented with slower OS IPC calls, and higher resource costs in process and CPU cores management.

Re: Swift Achieved Dynamic Linking Where Rust Couldn't (2019)

#164

Post by pjmlp » Mon, Jan 30, 2023, 7:55 AM UTC

pornel wrote: post #50

malkia wrote: post #39

Earlier quoted context omitted.

You can't ship reasonable plugin for most of the popular content creation tools - be it audio, video effects, 3d modelling, etc. without support for dynamic/shared libs. Okay, maybe you can - at the risk of performance cost - e.g. your out-of process plugin that need to move data in/out with a cost, and then it over complicates - because the plugin process might've died - how do you restart it, how do you get it back…

Apart from most perf critical cases, it's a good idea to run plugins in a separate process anyway. This way a bad plugin won't crash or corrupt your own process. You will be able to stop and unload it without risk of leaving locks locked or thread-locals leaking. And also there's WASM now which has even better sandboxing and OS/architecture independence.

WASM is nothing new, just a recyling of old ideas with refueled marketing.

Re: Swift Achieved Dynamic Linking Where Rust Couldn't (2019)

#165

Post by eschaton » Mon, Jan 30, 2023, 7:56 AM UTC

josephg wrote: post #140

giantrobot wrote: post #105

Earlier quoted context omitted.

> The vast majority of libraries on your system are used by only one program. I find this is a Linux-ism rather than the common case on Windows or macOS. With Windows and macOS a given version of the OS is a fixed set of libraries and services available for third party software. Even third party libraries tend to call into the system libraries for some things. On Linux the only fixed facility of the OS is going to be…

I think dynamic libraries which are only used by a single program are even more common on windows and macos. Look at how many DLLs ship with applications, and get installed inside the Program Files/MyApp directories! (Seriously - do a search for .dll files there. They're everywhere). Practically all of those DLLs are only be used by a single program. In each case, the price is being paid for dynamic linking (in start…

On Apple’s platforms the primary motivator for sharing code within an application is that an application isn’t just one component. You may have an application that has any number of app extensions in it that provide access to its features in various settings; if you build that functionality into a framework and use it from the app itself and any app extensions, it’ll still only be loaded into memory once even though there might be several processes mapping it.

Creating frameworks and libraries also helps the developer break down software into more testable components, which doesn’t necessarily require them to be dynamic but if you do go that route you remove an axis of variance that can result in different behavior.

For example, a static library on UNIX is just an indexed archive of object files, so it doesn’t have a single overall set of imports and exports or a single set of initializers to run at load, whereas a dynamic library is a fully-linked module with its own set of dependencies, imports, exports, and initializers. If you assume equivalence between them you’re eventually going to be wrong, often in hard-to-diagnose ways.

Re: Swift Achieved Dynamic Linking Where Rust Couldn't (2019)

#166

Post by pjmlp » Mon, Jan 30, 2023, 7:58 AM UTC

imron wrote: post #159

vrnvu wrote: post #156

Earlier quoted context omitted.

Sorry, does anyone mind to explain/reference what COM is?

Component object model. It’s a windows thing from the 90’s that still permeates the lower levels of windows programming.

Nope, it is the way modern Windows APIs are done since Vista.

Re: Swift Achieved Dynamic Linking Where Rust Couldn't (2019)

#167

Post by pjmlp » Mon, Jan 30, 2023, 8:00 AM UTC

zdimension wrote: post #160

vrnvu wrote: post #156

Earlier quoted context omitted.

Sorry, does anyone mind to explain/reference what COM is?

The result of Microsoft developers in the 90s drunkenly making up an object model on top of C with the goal of it being cross-language and cross-platform (in theory). The idea being that you can define an interface and implement it in any language and consume it in any language. A good idea in theory. In practice, DLL hell often made the experience unpleasant. A big part of Windows's APIs are COM-based. For historica…

COM is the main way Windows APIs are exposed since Windows Vista, as the .NET ideas for Longhorn were rebooted as COM.

While plain old bare bones Win32 APIs are still coming up in every release, they aren't the main ones.

Also COM as concept wasn't something designed by Microsoft, it rather build up on the ideas of what Sun, IBM and co were doing with distributed objects protocols.

It is no coincidence that COM IDL language is based on DCE RPC.

Re: Swift Achieved Dynamic Linking Where Rust Couldn't (2019)

#168

Post by silon42 » Mon, Jan 30, 2023, 8:03 AM UTC

darknavi wrote: post #128

greyw wrote: post #31

Dynamic linking is in my opinion not that useful anymore in this day and age where the few MB of RAM and disk space you save is not worth the hassle. The amount of dynamic linking issues I encountered on GNU/Linux was insane (fuck libstdc++). Not even glibc manages to keep forward compatibility working (breaking memcpy, breaking DT_HASH, ...)! It's much better to just statically link your binaries (unfortunately many…

Steam encourages devs to dynamically link SDL2 so that they can patch it for them [0]. Pretty neat IMO. Sorry for the terrible link, Google+ is deleted now but luckily a bot mirror the post to reddit. [0]: https://www.reddit.com/r/linux_gaming/comments/1upn39/commen...

"Platform" libraries, like UI toolkits, ... should always be dynamically linked...

Re: Swift Achieved Dynamic Linking Where Rust Couldn't (2019)

#169

Post by pjmlp » Mon, Jan 30, 2023, 8:06 AM UTC

jchw wrote: post #43

A lot of people who are invested in the Apple ecosystem have been trying to present Swift as a general competitor to Rust, but no matter how many bullet points you list, as far as I can tell, it hasn't made any progress at all on that front, and I haven't been convinced that it will change to become closer to that any time soon, either. As far as I can tell, there's really nothing wrong with Swift, and it probably ha…

I would say it is the reverse on what concerns Apple platforms, Rust isn't a competitor to Swift.

It will never have tooling parity with Swift.

Trying to use Rust instead of Swift on Apple platforms is only for those that enjoy the pain of creating ecosystems from scratch.

Re: Swift Achieved Dynamic Linking Where Rust Couldn't (2019)

#170

Post by pjmlp » Mon, Jan 30, 2023, 8:12 AM UTC

jchw wrote: post #74

galaxyLogic wrote: post #66

Earlier quoted context omitted.

Apple's goal may be is to create a SW ecosystem that mostly only works on Macs. If a Swift application could be easily compiled to run on Windows, it would be much easier for Apple users to switch to Windows or Linux. And if Swift apps would run happily on all platforms Apple would still have an incentive to make them run fastest on Macs. Therefore it is not a good idea to rely on a computer manufacturer to provide y…

Microsoft has plenty of conflicts of interest to contend with, but that hasn't stopped them from being able to make some inroads towards appealing to other platforms. Perhaps it would be unwise to trust that they've "changed" and invest in Microsoft tools or ecosystems, but it's beside the point: I think it's fair to say that Visual Studio Code and TypeScript have achieved widespread success across the board. I think…

Yeah, but if you want top tooling for .NET, the answer is still pretty much Visual Studio on Windows.

The cross platform version of .NET and VS4Mac/VSCode only support a subset of the whole development experience.

Many libraries on the ecosystem are still stuck in .NET Framework, wrappers for COM libraries, or plain Windows APIs, given its Windows focus between 2001 - 2016.

If you want a Visual Studio like experience without everything that is still missing from VS4Mac/VSCode then you need to shell out for a JetBrains Rider license.

Mostly because of the Windows/Visual Studio interests versus Azure folks interests.