Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Re: Integrating "safe" languages into OpenBSD? (2017)

marc.info

Post reply on HN
Print view View original

371–380 of 400 posts

Re: Re: Integrating "safe" languages into OpenBSD? (2017)

#371

Post by pjmlp »

lstamour wrote: post #202
legulere wrote: post #178

Earlier quoted context omitted.

Dynamic linking is pretty common in GUI applications (xlib, GTK/QT). Server-side there's also libssl, xml libraries, zlib, curl.

Outside of OS distro packaging, most Qt apps actually copy dynamically linked Qt binaries to the same app folder for re-distribution to end users within some form of installer or disk image. The same probably applies to Gtk apps too, as I think the last time I manually installed Gtk for Windows for a Gtk app was a decade ago. Dynamic linking only works if you can guarantee ABI stability and folks haven’t had to deal…

It is also one of the reasons why macOS is not welcomed by enterprise IT for large scale deployments.

You can have dynamic linking with ABI stability with stuff like COM and UWP.

It is also the only viable way to do plugins in scenarios where IPC is too costly.

Re: Re: Integrating "safe" languages into OpenBSD? (2017)

#372

Post by smnplk »

stcredzero wrote: post #257

BeOS/Haiku got itself into the 250 MB range of a full-featured distro, in part by emphasizing code reuse in C++. (Though my understanding is that they used GNU grep et. al.) The current macroeconomic climate is wrong for this, but what if someone funded an effort to base an implementation of an entire OS on Rust, let's say, using Linux as a bootstrap? So there would be no requirements for "base to build base," and th…

look into https://www.redox-os.org/

Re: Re: Integrating "safe" languages into OpenBSD? (2017)

#373

Post by pjmlp »

zozbot234 wrote: post #184
bsdubernerd wrote: post #103

It also doesn't help that Rust keeps on pushing on the idea that "static linking is the only way to go". This is another cargo-cult which I wish didn't end up being engrained so deep in the toolchain because while it has some merits, it also has significant drawbacks of a typical unix distribution. Static linking might be good for folks distributing a single server binary over a fleet of machines (pretty much like Go…

Dynamic linking is just not very useful for rust due to lacking a stable ABI. So yes, you can build shared libraries and link to them but you'll be using the C ABI at the interaction boundary, which some devs might find unintuitive. (Or else you have to make sure that everything you link to is compiled by the same version of rustc and llvm, which is not very realistic.) Static linking sidesteps this potential issue.

A systems language should support everything regardless how useful it might be.

Unless it wants to leave some scenarios open for the systems programming languages that offer tooling for them.

Re: Re: Integrating "safe" languages into OpenBSD? (2017)

#374

Post by pjmlp »

hedora wrote: post #155
em-bee wrote: post #79

Earlier quoted context omitted.

after you are done downloading all the source.

You can check that the source hasn’t been tampered with before starting the build, and know you’re building the right code base. Builds that curl junk and use a dozen language specific package managers don’t have that property.

That assumes every user is a developer, expert in security assessment, knowledgeable in tricks like trusting trust, and all the languages used in the code base.

Re: Re: Integrating "safe" languages into OpenBSD? (2017)

#375

Post by fluffything »

pjmlp wrote: post #370
fluffything wrote: post #338

Earlier quoted context omitted.

A GC only protects you from memory errors, not threading errors. In general, Rust allows you to write software that doesn't break silently. That's a quite good value proposition for large scale software, where other languages often require programers to be super careful with refactorings, while in Rust you can really refactor all the things. The reason people are afraid to do large refactorings in say C++ is often "s…

Rust's type system also only protects about threading errors if they happen to be data races for in-process data structures. If they are external resources, there is little it can do. While an improvement, it isn't a full solution, specially in the domain of distributed computing.

For multi-process data-structures using inter-process shared memory, you can still write Rust APIs that protect you from data-races as long as all processes involved use those APIs. If they don't, then protection is up to the operating system (you can lock and unlock shared memory, but whether processes can violate the locks is an OS thing).

For distributed computing, you typically use message passing of some form via network sockets, that's "safe" by design. You can also use RDMA, and there usually your process needs to opt into that, create an RDMA readable/writable region, and you are back to the mercy of your network stack up to which kind of protection you can enforce there.

From Rust point-of-view, if you make the wrong assumptions about shared memory or RDMA regions, your program has a bug. The fix is simple, use appropriate atomic (or atomic volatile) to read/write from those regions. Even if another process is writing while you are reading, as long as the operations are the right ones (e.g. This won't protect you from deadlocks or race conditions, etc. but that's not something that any programming language does. Rust has deadlock protection within a process, but that's not required for safety, so it is an extra feature on top.

Re: Re: Integrating "safe" languages into OpenBSD? (2017)

#376

Post by anewvillager »

ajross wrote: post #89
jcranmer wrote: post #8

> I wasn't implying. I was stating a fact. There has been no attempt to move the smallest parts of the ecosystem, to provide replacements for base POSIX utilities. This is in fact incorrect--there is a project aiming to build all of the coreutils in Rust ( https://github.com/uutils/coreutils ). More to the point: while I do concur in the conclusion that Rust shouldn't be a part of the OpenBSD base system, the gatekee…

> the gatekeeping implied here [...] is really toxic It's Theo de Raadt, toxic rhetoric is sort of his brand. But... he has a real point here. It's not about grep or cat or whatever really, those are just the use cases for which OpenBSD would care. It's that as Rust is reaching the second decade of its history, and despite some outrageous wins in press, evangelism, and general developer mindshare... Rust just hasn't…

> Where are the pervasively used compression libraries in Rust? Video and audio codecs? Network stacks? Database engines? System management utilities? PKI and encryption stacks? All that stuff is still in C. After ten years of Rust success!

It's not ten or 20 years, the first release of Rust was in 2015. Pre-1.0 Rust was a wildly different language, with green threads, segmented stacks, regular and wild breakage, not really a C replacement! Please understand this point.

But anyway here are some projects:

https://github.com/ctz/rustls a TLS library that uses https://github.com/briansmith/webpki a pki library

https://github.com/burntsushi/rust-snappy a compression library

https://github.com/tikv/tikv a database engine

https://github.com/hyperium/tonic a gRPC library

https://github.com/tock/tock an embedded OS

https://lib.rs/command-line-utilities lots of CLI utilities which include system management

You ask for "pervasively used" but this is not under control of Rust itself. It's not feasible to replace decades old setups in five years.

The most widely deployed Rust stuff is in Firefox and some Gnome libraries AFAIK.

> I mean, is it maybe fair to say that the window is closing for rust to take over a significant fraction the systems programming world?

I don't see why this should be the case.

Re: Re: Integrating "safe" languages into OpenBSD? (2017)

#377

Post by pas »

spion wrote: post #367
pas wrote: post #318

Earlier quoted context omitted.

yarn uses a global cache and symlink if I remember correctly. and npm has an "audit" feature, which is default on, so there's quite a bit of movement in JS land to address the disadvantages of having a myriad small packages.

Its not about the size of the packages, its about managing an entire system, for whatever definition of the word "system" In the "olden" days, you would patch the vulnerable dependency and be sure that all services running on the machine have been updated to the new version. (in theory - if everything works great and there is ABI compatibility etc etc). Since the library is installed in a well-known system location,…

Agreed. Yet it seems the ecosystem simply accepts this trade-off. For increased productivity, increased memory-management security, etc.

Re: Re: Integrating "safe" languages into OpenBSD? (2017)

#378

Post by pjmlp »

fluffything wrote: post #375
pjmlp wrote: post #370

Earlier quoted context omitted.

Rust's type system also only protects about threading errors if they happen to be data races for in-process data structures. If they are external resources, there is little it can do. While an improvement, it isn't a full solution, specially in the domain of distributed computing.

For multi-process data-structures using inter-process shared memory, you can still write Rust APIs that protect you from data-races as long as all processes involved use those APIs. If they don't, then protection is up to the operating system (you can lock and unlock shared memory, but whether processes can violate the locks is an OS thing). For distributed computing, you typically use message passing of some form vi…

That is exactly my point, while it is definitely good to have, it isn't the end solution that many assume at first thought.

Re: Re: Integrating "safe" languages into OpenBSD? (2017)

#379

Post by throwaway894345 »

akira2501 wrote: post #336
throwaway894345 wrote: post #72

Earlier quoted context omitted.

Yeah. Not sure about BSD, but I was wading into building the GNU coreutils and other GNU packages just yesterday. Fresh hell, they all seem to be build dependencies of each other. ‘sed’ is its own build dependency. The whole C ecosystem is a joke with respect to builds—all dependencies are implicit; you’re just expected to have the exact dependencies installed on your system at the exact versions and in the exact loc…

> Fresh hell, they all seem to be build dependencies of each other. ‘sed’ is its own build dependency. Well, they are the "core" utilities after all. This becomes less of an issue the further out in the ecosystem you get. > at the exact versions and in the exact locations on the filesystem that the build tooling is willing to look. This isn't strictly true.. you just need _compatible_ versions to be available, and mo…

I don’t know man, I’ve written a lot of C and C++ and other languages and the sheer effort to get your average C project to compile is much greater than others. The dependencies aren’t listed anywhere, they are rarely compatible with the versions available via your package manager (so now you’re building more things from source), ./configure regularly runs into problems (“couldn’t guess which platform you were on”, “your version of grep is incompatible”, etc). It’s just a big mess of yak shaving. If you think I’m making too big a deal, I invite you to spend some time in the Go or Rust ecosystems where you just “cargo build” or “go build” and you’re done. You don’t need to understand anything about the project, there are no shitty DSLs to grok (looking at you, CMake). Things just work.

Re: Re: Integrating "safe" languages into OpenBSD? (2017)

#380

Post by throwaway894345 »

cycloptic wrote: post #358
throwaway894345 wrote: post #315

Earlier quoted context omitted.

Yeah, Nix is really neat. It's the right direction for sure, but it's mostly used as a wrapper around autotools and cmake projects by third parties and not so much as a first-class build tool by the maintainers of C/C++ libs (this isn't going to change because Nix prefers to be "innovative" and unfamiliar in just about every design decision--its core approach to reproducible builds is right on, but the execution leav…

Nix is a package manager, not a build tool. It's never been in-scope for it to figure out how to invoke a particular compiler or linker. All it can do is make it easier for you to invoke another build tool, of which there are quite a few. I would not recommend the use of autotools in new projects, but in my experience CMake with Ninja is fine for distro maintainers to deal with — All other modern build tools I've see…

> Nix is a package manager, not a build tool. It's never been in-scope for it to figure out how to invoke a particular compiler or linker. All it can do is make it easier for you to invoke another build tool, of which there are quite a few.

I agree that Nix is focused on package management and not building, and that’s a bummer because Nix is perfectly capable technically. In particular, Nix can trivially invoke a compiler or linker; its issues are that it assumes that it will only be used by a relatively small number of package maintainers instead of a much larger number of developers, so it prefers to be novel and innovative instead of familiar and straightforward. This makes it really hard to get developers to buy in, but there’s no technical reason why it can’t be used as a build system.

> I would not recommend the use of autotools in new projects, but in my experience CMake with Ninja is fine for distro maintainers to deal with — All other modern build tools I've seen are roughly the equivalent of it. If you find that it's hard to get over the hump with that, then I hate to say it but I don't think solutions will come easily elsewhere on any OS or platform. The state of things on free software operating systems is constantly improving but for now it's something you have to get used to. There are no silver bullets here.

The silver bullets are getting away from the C ecosystem to the greatest extent possible, otherwise you will have to build way more autotools and CMake projects and this isn’t a good use of anyone’s time. As fun as C is, for most applications, we have better alternatives nowadays. They aren’t perfect, but they let you move a lot faster especially with respect to building software.

Post reply on HN