Re: Integrating "safe" languages into OpenBSD? (2017) • Hacker News Board

Re: Re: Integrating "safe" languages into OpenBSD? (2017)

#331

Post by saagarjha » Sun, May 03, 2020, 9:28 PM UTC

riquito wrote: post #301

spion wrote: post #250

Earlier quoted context omitted.

Why does a "default" need to exist at all at the system level? Why not just have various versions available and have version decisions be local to the software being developed/used.

Security. By having defaults the operative system can share the most secure minor version of that library (or multiple major if necessary). When you have a copy of a library for each program then most of your programs will end up using obsolete or unsecure dependencies.

Plus it also cuts down on disk space usage.

Re: Re: Integrating "safe" languages into OpenBSD? (2017)

#332

Post by saagarjha » Sun, May 03, 2020, 9:33 PM UTC

Arch-TK wrote: post #309

roblabla wrote: post #163

Earlier quoted context omitted.

> Python's build ecosystem IS sane and well-designed. It's the pinnacle of package handling so far. Big oof. Python packaging has a lot of problems, and defaulting to a single shared location when installing, that might conflict with your default package manager, is very high on the "What the fuck are you even doing" list of issues. You also very easily get into version hell. Unless you start using virtualenv, at whi…

> Python packaging has a lot of problems Sure, but I think that what systems like cargo provide is worse across the board. > defaulting to a single shared location when installing, that might conflict with your default package manager, is very high on the "What the fuck are you even doing" list of issues. This is easily solved if pip was to have a way of listing all dependencies which it's about to install. Then I co…

Rust not having a stable ABI is not universally bad; it allows for breaking changes obviously. Swift has stabilized its ABI with generics, but its implementation is not quite zero-cost so I am not sure if Rust could go that direction.

Re: Re: Integrating "safe" languages into OpenBSD? (2017)

#333

Post by saagarjha » Sun, May 03, 2020, 9:35 PM UTC

kzrdude wrote: post #186

adrianN wrote: post #138

Earlier quoted context omitted.

No, I don't think so. If LLVM is at fault then the issue is definitely not inherent to Rust the language or Rust the development culture.

Interesting point. Does anyone know how much memory is needed to compile llvm+clang? GCC?

That varies widely on your linker and how many threads you use. I have compiled LLVM on systems with 4 GB of RAM, though; but have been forced to fall back on single-threaded compilation.

Re: Re: Integrating "safe" languages into OpenBSD? (2017)

#334

Post by burntsushi » Sun, May 03, 2020, 9:38 PM UTC

rurban wrote: post #293

staticassertion wrote: post #61

This is less "On Rust" and more "On accepting a rewrite of any tool into OpenBSD, on the merits of memory safety alone". There isn't really much of a statement or judgment on Rust. At most there's an interesting point on it's value proposition: > However there is a rampant fiction that if you supply a new safer method everyone will use it. For gods sake, the simplest of concepts like the stack protector took nearly 1…

A lot is missing also. First he doesn't know about ripgrep, which is far better than GNU or BSD grep. Second,none of the known grep's can find unicode strings. Redhat carried along the uni patches for a while, but when people complained about performance on the new utf8 locales, they dropped it. coreutils is still missing unicode support, so we don't find equivalent strings with different bytes. No normalization, no…

> First he doesn't know about ripgrep

ripgrep isn't POSIX compliant. Never was and never will be. So I don't think it's really applicable here. ripgrep is maybe an existence proof that a competing tool can be written, but it is certainly not a suitable POSIX compliant grep replacement. Building a fully POSIX compliant grep tool with good performance like GNU grep is pretty difficult. It could be done. It would probably take me a couple months (and only by leveraging my existing work). I just don't have any incentive to do it, personally.

> Second,none of the known grep's can find unicode strings.

That's definitely not true. GNU grep can do this just fine:

    $ echo 'Δ' | LC_ALL=en_US.UTF-8 grep '\w'
    Δ
    $ echo 'Δ' | LC_ALL=en_US.UTF-8 grep 'δ' -i
    Δ

GNU grep does pay more of a performance penalty for Unicode support than ripgrep does. It's easy to see this with a somewhat pathological case by comparing `rg '\w{42}'` with `LC_ALL=en_US.UTF-8 grep -E '\w{42}'`. (Not all cases are pathological, but ripgrep and GNU grep are both very good about literal optimizations, so it's easiest to demonstrate with a pathological case.) I'm not sure if one can attribute this to better library support in the Rust ecosystem though. It's really about how the regex machinery itself is built.

Re: Re: Integrating "safe" languages into OpenBSD? (2017)

#335

Post by saagarjha » Sun, May 03, 2020, 9:42 PM UTC

busterarm wrote: post #238

eximius wrote: post #173

Earlier quoted context omitted.

I don't think the position of the language teams is that static linking is better than dynamic linking. I think that static linking is a significantly easier target for a relatively new language whose _novel_ features make it difficult to define a stable ABI. Under the very specific constraints that Rust is operating in, static linking is currently the best option. That said, I wouldn't mind seeing some kind of ABI g…

Deployability really should be king. I hinted at this in my other post, but I don't think quite enough people think about this. Static linking makes your deployability worries go away. That's not to say that isn't at least possible with dynamic linking, but the complexity and gymnastics will sink you. Everyone gets bit by it eventually.

> Deployability really should be king.

Not everyone values deployability as much as you (or employers, I suspect) do. On my personal computer I care about getting security patches and disk usage.

Re: Re: Integrating "safe" languages into OpenBSD? (2017)

#336

Post by akira2501 » Sun, May 03, 2020, 9:43 PM UTC

throwaway894345 wrote: post #72

staticassertion wrote: post #61

This is less "On Rust" and more "On accepting a rewrite of any tool into OpenBSD, on the merits of memory safety alone". There isn't really much of a statement or judgment on Rust. At most there's an interesting point on it's value proposition: > However there is a rampant fiction that if you supply a new safer method everyone will use it. For gods sake, the simplest of concepts like the stack protector took nearly 1…

Yeah. Not sure about BSD, but I was wading into building the GNU coreutils and other GNU packages just yesterday. Fresh hell, they all seem to be build dependencies of each other. ‘sed’ is its own build dependency. The whole C ecosystem is a joke with respect to builds—all dependencies are implicit; you’re just expected to have the exact dependencies installed on your system at the exact versions and in the exact loc…

> Fresh hell, they all seem to be build dependencies of each other. ‘sed’ is its own build dependency.

Well, they are the "core" utilities after all. This becomes less of an issue the further out in the ecosystem you get.

> at the exact versions and in the exact locations on the filesystem that the build tooling is willing to look.

This isn't strictly true.. you just need _compatible_ versions to be available, and most build tools and even the compiler itself will check several commonly used locations for code.

There are a few corner cases, but by and large, this isn't as big a problem as it might be for other languages that make writing incompatible interfaces orders of magnitude easier than with C.

> Specifically, hasten the day where my entire application dependency tree doesn’t bottom out in some Autotools, CMake, shell script, project.

Each those solve the same problem at different levels. Perhaps this is because there is no _one true_ way to replace them, and the advantage is that each project uses the tool most suited to their case.

Perhaps the built-in tools with these "modern" languages are a good general tool for the majority of developers, perhaps they aren't. If they aren't, then now you have the same problem all over again in a new ecosystem.

Re: Re: Integrating "safe" languages into OpenBSD? (2017)

#337

Post by raggi » Sun, May 03, 2020, 9:43 PM UTC

Working on an operating system that uses some of these modern / safer alternatives, the concerns around bootstrapping, lack of spec, lack of formal memory model, long compile times, and larger output binaries are valid. They are real and present challenges for us, that we choose to work against. It is currently extremely difficult to produce a binary of the size of bsd cat using these technologies, and very large amounts of work are needed to close that gap, if it even can be.

In a new system you may not need cat though, but if you're a unix system with a heavy shell and textual basis, you do need lightweight versions of these things - light to build, light to run, and well integrated into the environment.

What's the path out for unix systems? I have no idea, but at least as of today I have not seen a modern safe language that is ready to replace unix use cases. Maybe zig? It's hello world is at least under 10kb.

Re: Re: Integrating "safe" languages into OpenBSD? (2017)

#338

Post by fluffything » Sun, May 03, 2020, 9:44 PM UTC

staticassertion wrote: post #76

fluffything wrote: post #70

Earlier quoted context omitted.

> Developers in particular don't generally care about security, Bad developers don't, but many developers do. The Rust project itself has hundreds of contributors, to the point that it feels that it has more contributors than LLVM itself (I work on both, and this is an unbacked feeling I get from the velocity of the contributions). Point being, if developers wouldn't care about Rust, they wouldn't be developing it.

Maybe I shouldn't have said that, because it's contentious, but I do believe it to be true. I believe Rust's success has less to do with memory safety, which I think most developers (anyone coming from a GC'd language) consider table stakes, and much more to do with great documentation and incredibly powerful primitives and ecosystem such as the type system, cargo, crates.io, etc.

A GC only protects you from memory errors, not threading errors.

In general, Rust allows you to write software that doesn't break silently. That's a quite good value proposition for large scale software, where other languages often require programers to be super careful with refactorings, while in Rust you can really refactor all the things.

The reason people are afraid to do large refactorings in say C++ is often "security-related": fear of introducing segfaults, memory errors, undefined behavior, threading errors, etc. but one can also see these fears as "productivity-related" (hours and hours of debugging), or through many other lenses (shipping bugs to users, having a segfault in the middle of a demo that costs you a client, introducing a segfault one day before the release of your game, etc.).

In general, every programmer wants to have a certain degree of security that their software "works" for some definition of "works". For some this security might be actual network security, but for others it might mean that they don't want their data-science app that has been running for a week burning thousands of dollars to crash due to a segfault while writing the results.

Re: Re: Integrating "safe" languages into OpenBSD? (2017)

#339

Post by geofft » Sun, May 03, 2020, 9:45 PM UTC

a1369209993 wrote: post #323

geofft wrote: post #306

Earlier quoted context omitted.

Granting your framing for the sake of argument, then we've got one remote hole in a heck of a long time in Ubuntu desktop, and still nothing in macOS or Amazon Linux. I'm certainly not claiming that any of these OSes is better than OpenBSD. I'm claiming they're all about the same, and the "OpenBSD cares more about security than everyone else" narrative isn't actually based in evidence. OpenBSD has a particular view o…

> we've got one remote hole Are you forgetting the time they sent local filesystem searches to some spyware company? And that's just things that were a: deliberate, and b: public enough that I remember them off the top of my head despite not having used Ubuntu in years? (I forget which specific problem made me drop it, or I'd probably have a third example.) OSX is a toxic, vendor-supplied-malware infested cesspit tha…

> Are you forgetting the time they sent local filesystem searches to some spyware company?

That was not a remote hole.

> OSX is a toxic, vendor-supplied-malware infested cesspit that I've never used

Sure, but is any of the vendor-supplied malware in that cesspit a remote hole?

I'm happy to have broad, open-ended arguments about who sucks more in new and innovative ways, but let's finish the argument we're already having first. Is OpenBSD meaningfully more secure than other operating systems on the axis they are choosing to advertise, namely "remote holes in the default install," than other OSes?

In particular, whatever you believe about deliberate backdoors, toxic cesspits, being actively opposed to security, etc., none of that is something the choice of programming language is in any way relevant to. If Theo's argument were "We don't need Rust because we are the only operating system that isn't actively opposed to security, so everyone else has lost the game already," we'd be having a very different conversation. But it's not and we aren't.

Re: Re: Integrating "safe" languages into OpenBSD? (2017)

#340

Post by lonelappde » Sun, May 03, 2020, 9:45 PM UTC

dathinab wrote: post #74

cycloptic wrote: post #59

Earlier quoted context omitted.

There is no gatekeeping. To actually achieve POSIX-compliance is not an easy task and requires lots of testing. If you don't do it then your replacement will break everyone's scripts. A distro maintainer will also want to retain compatibility with their supported GNU/BSD extensions so that's more work to add on. I would still agree with his statement at least as far as BSD is concerned. Outside of Redox I haven't see…

I don't think you miss anything. POSIX compliance is a headache which is not only hard to get right but also strongly limits your interfaces and internal tooling. Many scripts still will run with non full compliance, but that doesn't help if you try to build for OpenBSD. Also I'm not sure how they ended up with 400 deps for coreutils. I would have expected much less. But yes this means at least until better code sign…

The author is from the 1990s era of Linux / FOSS, before kindness was part of the culture, when the culture had a tough competitive streak where mistakes and humility were seen as signs of weakness and disrespected. High expectations wee tightly bound with low levels of patience. And many people were scared off from contributing.