Re: Integrating "safe" languages into OpenBSD? (2017)

> I don't know what project this is but a blanket "add a test which fails without your patch for EVERY PATCH no exception" rule is a bit cargo-culty.

We had (and still have) 100% code coverage on Windows MSVC, Windows GNU, MacOS, iOS, Android, Linux, FreeBSD, NetBSD, DragonflyBSD, Solaris, WASM, and many other platforms, as well as all possible hardware (ARM32/64, x86/x64, ppc32/64/be/le, sparc64, mips32/64/be/le, mips64, riscv32...).

The OpenBSD parts of our code base weren't tested at all; they weren't even compiled.

In practice that meant that pretty much every PR broke the OpenBSD support in some subtle way, and people would only discover those failures during the release process, which took a lot of time to fix, usually from somebody that did not care about OpenBSD at all, requiring them to create a VM to be able to develop and test from OpenBSD itself.

So yeah, we decided to unanomously require that code that cannot be tested be removed, and gave the OpenBSD devs the chance to fix it. Starting with requiring OpenBSD contributor on their PRs to add testing support.

It wasn't practicable because their platform is designed to prevent non-OpenBSD users to develop for it. So we removed the code, and this became some OpenBSD user group problem.

This solution actually worked better for them, because it became up to them to decide when new versions of our project got released for OpenBSD, so they could get the latest release, apply patches, test that it compiled locally and passed tests, etc. before doing a release for OpenBSD, something that we did not care about doing.

We believe that code that isn't compiled doesn't compile, and that code that isn't tested doesn't work correctly. Our OpenBSD parts were living proof that this is 100% true.

I find it quite ironic that this project is security related, and some of the bugs that hit OpenBSD were CVEs that were completely preventable. This CVEs only hit OpenBSD because it was the only untested platform. In a sense, OpenBSD users actually were quite lucky, in that most releases broke compilation for OpenBSD, and that saved them from many CVEs. Unluckily for them, not all bugs result in compilation errors.

> Finally, surely you get plenty of patches which do small useful things like cleanup but which it would likely be impossible to write a test for.

We have 100% code coverage. All our functionality is heavily tested and fuzzed. The modifications in a cleanup PR are exercised by many tests, and therefore tested.

If someone submits a bugfix, that means we were missing a test, so we require users to add one. OpenBSD devs weren't able to do that, so we stopped accepting their bugfixes, and at some point removed all the code.

If you don't think that bugfixes should be accompanied by tests that check that the fix works and the bug cannot be reintroduced again, then I disagree with you, and hope I never have to use your software.

> we've got one remote hole

Are you forgetting the time they sent local filesystem searches to some spyware company? And that's just things that were a: deliberate, and b: public enough that I remember them off the top of my head despite not having used Ubuntu in years? (I forget which specific problem made me drop it, or I'd probably have a third example.)

OSX is a toxic, vendor-supplied-malware infested cesspit that I've never used and don't pay much attention to, and I've never even heard of Amazon Linux, so I wouldn't expect to have examples for those.

> the "OpenBSD cares more about security than everyone else" narrative

Actually, my claim was that Ubuntu (and maybe a significant fraction of "everyone else", but that wasn't really my point) is actively opposed to security.

> even if we don't admit a difference between unintentional remote holes and "deliberate backdoors,"

There is a difference; there's a huge difference; deliberate backdoors are much, much worse. This kind of shit is something I would expect of Microsoft (Windows) or Google (Chrome).

> For example outside of rustc-dev hardly anyone compiles rust them self.

Yep, and it shows (as many other threads point out). Gentoo users are one of the populations that is more likely to compile rust from scratch, but many likely use rust-bin because the build times, toolchain dependencies, etc. are all completely insane non-starters. If the rust evangelists want people to take them seriously, they need to have serious answers to serious engineering questions and concerns and not simply respond with "but memory safety!"

Amusingly this recalls the kernel mailing list thread where Linus says "I think the OpenBSD crowd is a bunch of masturbating monkeys" because they focus too much on security. Well damn if Theo didn't just effectively say about the rust evangelists what Linus said about him. You have to be severely out of touch to not understand, much less dismiss, the basic features that OSes require of language ecosystems. I mean, these are so basic that if you show up without those features you will be rightly shown the door because you haven't done the simplest requirements gathering much less the actual engineering work that anyone would be expected to do if they wanted to be part of core infrastructure.

I've installed hundreds of pieces of software from source, and I have a fairly decent idea of how linking and loading works. I still have to spend time figuring out why something doesn't build when it should due to something strange about my system.

> Makefiles have to work

And yet I get GNU Makefiles all the time that don't work when I type "make" at my computer, since they use things that aren't compatible with my version of Make. It's clearly more of a problem than you think it is.