Can’t say I’m a fan of Nix evangelists pointing their finger at any problem and yelling how it would be solved better by using Nix, but in this case, one could pin a nixpkgs version and all the sources for llvm, gcc and ocaml, and thus have a reproducible bootstrap. Ultimately, it wouldn’t do anything different to what was done manually here, but pinning commits will save the archaelogical burden for the next bootstrapper.
Retrobootstrapping Rust for some reason
11–20 of 54 posts
Re: Retrobootstrapping Rust for some reason
#12The difficulty in reproducing builds and steps even from a time as recent as 2011 is somewhat disturbing; will technology stabilize or is this going to get even worse? At what point do we end up with something in-use that we can’t make anymore?
I think we must have some software in use for which the compiler or the source code just isn’t around anymore. It probably isn’t a massive problem. There’s just a slow trickle of tech we can’t economically reproduce, but we replace it with better stuff. Or, if it was really crucial, it would become worth paying for, right?
I think with advances in AI-assisted decompilation, we may soon end up in the situation where given a binary you can produce realistic-looking source (sane variable and function names, comments even) which compiles to the same binary, even though non-identical to the original source code
Re: Retrobootstrapping Rust for some reason
#13I've done this project myself, based on Ubuntu 20.04 and a whole lot of patchsets [0]. I got up to the 2014-01-20 snapshot before running into weird LLVM stack issues that I couldn't figure out how to resolve. One big annoyance is that the snapshot file refers to some commit hashes that do not appear to point to any surviving public repo, so it takes a fair bit of effort to reconstruct which commits must have been included in the missing commits.
Re: Retrobootstrapping Rust for some reason
#14> Modern clang and gcc won't compile the LLVM used back then (C++ has changed too much) Is this due to changing default values for the standard used, and would be "fixed" by adding "std=xxx" to the CXXFLAGS? I've successfully built ~2011 era LLVM with no issues with the compiler itself (after that option change) using gcc last year - there were a couple of bugs in the llvm code though that I had to workaround (mainly…
Even -fpermissive is no longer sufficient for some of the things that appear in the old LLVM codebase. It's mostly related to syntax issues that older compilers accepted even though the standard never permitted them.
Re: Retrobootstrapping Rust for some reason
#15Not sure why, but I am getting 403 Forbidden, so if you are getting the same here's an archive.is link https://archive.is/UH5fg
You're not the only one getting blocked. I emailed dreamwidth about this in the past and they say it's something their upstream network host does and they cannot even fix it if their site users wanted to fix it. They're a somewhat limited and broken host partially repackaging some other company's services. >Dreamwidth Studios Support: I'm sorry about the frustrations you're having. The "semi-randomly selected to solv…
This needs to be a catchy name, but I don't have a good one. CloudFlaritis? CloudFlareup? (CloudFlareDown?)
Regardless of whether Cloudflare is the particular infra company, the company who uses them responds to blocked people: "We don't know why some users can't access our Web site, and we don't even know the percentage of users who get blocked, but we're just cargo-culting our jobs here, so sux2bu."
The outsourced infra company's response is: "We're running a business here, and our current solution works well enough for that purpose, so sux2bu."
Re: Retrobootstrapping Rust for some reason
#16> Debian has maintained both EOL'ed docker images and still-functioning fetchable package archives at the same URLs as 14 years ago.
Debian FTW.
Re: Retrobootstrapping Rust for some reason
#17> Modern clang and gcc won't compile the LLVM used back then (C++ has changed too much) Is this due to changing default values for the standard used, and would be "fixed" by adding "std=xxx" to the CXXFLAGS? I've successfully built ~2011 era LLVM with no issues with the compiler itself (after that option change) using gcc last year - there were a couple of bugs in the llvm code though that I had to workaround (mainly…
Well, one thing I've noticed about LLVM is that it blatantly and intentionally relies on UB. The particular example I encountered probably isn't what causes the version breakage, but it's certainly a bad indicator.
That said, failures in building old software are very often due to one of:
* transitive headers (as you mentioned)
* typedef changes (`siginfo_t` vs `struct siginfo` comes to mind)
* macros with bad names (I was involved in the zlib `ON` drama)
* changes in library arrangement (the ncurses/tinfo split comes to mind, libcurl3/4 conditional ABI change, abuse of `dlopen`)
Most of these are one-line fixes if you're willing to patch the old code, which significantly increases the range of versions supported and thus reduces the number of artifacts you need to build for bootstrapping all the way to a modern version.
Re: Retrobootstrapping Rust for some reason
#18The difficulty in reproducing builds and steps even from a time as recent as 2011 is somewhat disturbing; will technology stabilize or is this going to get even worse? At what point do we end up with something in-use that we can’t make anymore?
I'd imagine that it's going to end up both getting somewhat better and somewhat worse.
2011 is around the time that programmers start taking undefined behavior seriously as an actual bug in their code and not in the compiler, especially as we start to see the birth of tools to better diagnose undefined behavior issues the compilers didn't (yet) take advantage of. There's also a set of major, language-breaking changes to the C and C++ standards that took effect around the time (e.g., C99 introduced inline with different semantics from gcc's extension, which broke a lot of software until gcc finally switched the default from C89 to C11 around 2014). And newer language versions tend to make obsolete hacky workarounds that end up being more brittle because they're taking advantage of unintentional complexity (e.g., constexpr-if removes the need for a decent chunk of template metaprogramming that relied on SFINAE, a concept which is difficult to explain even to knowledgeable C++ programmers). So in general, newer code is likelier to be substantially more compatible with future compilers and future language changes.
But on the other hand, we've also seen a greater tend towards libraries with less-well-defined and less stable APIs, which means future software is probably going to have a rougher time with getting all the libraries to play nice with each other if you're trying to work with old versions. Even worse, modern software tends to be a lot more aggressive about dropping compatibility with obsolete systems. Things like (as mentioned in the blog post) accessing the modern web with decade-old software is going to be incredibly difficult, for example.
Re: Retrobootstrapping Rust for some reason
#19Can’t say I’m a fan of Nix evangelists pointing their finger at any problem and yelling how it would be solved better by using Nix, but in this case, one could pin a nixpkgs version and all the sources for llvm, gcc and ocaml, and thus have a reproducible bootstrap. Ultimately, it wouldn’t do anything different to what was done manually here, but pinning commits will save the archaelogical burden for the next bootstr…
Does re bootstrapping Rust like this actually work? How much work is it?
Re: Retrobootstrapping Rust for some reason
#20I imagine you just need to update CA certs and the known_hosts file to get GitHub communication working again.
A few more hurdles might involve expectations of SHA-1 cert signing, and TLS1.0 deprecation