Live data from Hacker News

The GitHub Arctic Code Vault

github.blog

81–90 of 96 posts

Re: The GitHub Arctic Code Vault

#81
post #64
post #18

As David Rosenthal (formerly of Sun, NVIDIA, and Stanford) explains, the actual Arctic Code Vault is a PR stunt, and has almost no chance of helping anyone in any kind of realistic disaster scenario: https://blog.dshr.org/2019/11/seeds-or-code.html That said, the rest of the project, which focuses on preserving several independent copies of repositories hosted on GitHub with a handful of partner organizations, is qui…

Archive Program director here - it's really not a PR stunt, we genuinely believe it will be of significant historical value and quite a good chance it will be of practical value. Much of that is "if we forget technology which we realize somewhere down the road we actually might want to use again." History provides plenty of examples of this, and it's particularly important with a technology which mostly lives on ephe…

People have some legitimate and some less legitimate criticisms here, in the HN comments section of course, but I for one think this is a fantastic effort and I'm pleasantly surprised to read what the new badge I saw on my profile yesterday is actually about.

There will always be "negative Nancies" -- especially here, they are everywhere -- but personally I'd just like to say thanks for having some vision outside of the normal day-to-day of making money for shareholders and keeping regular customers happy. More of this, please.

Re: The GitHub Arctic Code Vault

#82

> The next morning, it traveled to the decommissioned coal mine set in the mountain, and then to a chamber deep inside hundreds of meters of permafrost, where the code now resides fulfilling their mission of preserving the world’s open source code for over 1,000 years. What is the probability that we still have the required tech to read that code in 1,000 years?

Depends on whether the Great Filter is before or behind us.

Re: The GitHub Arctic Code Vault

#83
post #23

This is so awesome, but the most surprising to me is that all the public source code on GitHub only totals 21 TB. I forget that they do fundamentally host text, and not video etc. I somehow thought it would be petabytes. The private repos might be more than that but those are historically paid.

On the topic of size, I wonder how small it would be if you were able to deduplicate all repositories against each other. I sometimes suspect there is a tremendous amount of copy/paste code out there masquerading as someone else’s. Even a naive deduplication might yield some very interesting results Reminds me of a time I caught someone using someone else’s code in an interview and passing it off as their own. (Using…

I work at Software Heritage, where we archive all source code we can find, including all GitHub repositories, and deduplicate them internally.

The size of all file contents (including older versions of files) is a few hundreds TBs, and everything else (directory structures, revision history, etc.) is under 10TB.

So for GitHub alone it would be a little under that

Re: The GitHub Arctic Code Vault

#85

Where can we find the list of the 6000 repos ? On my profile it just shows 3 "and more", would like to get the full list. TYIA ;)

This was corrected by @rezendi below:

> Archive Program director here - the 6,000 repos were on the single proof-of-concept reel we archived last autumn. The full archive consists of millions of repos, including all repos with at least one star with any commits in the year leading up to 02/02/2020.

Re: The GitHub Arctic Code Vault

#87

Earlier quoted context omitted.

Rather more eloquently asked than by the other person I saw querying this[0]! I suspect it's covered under Github's TOS - specifically[1], only public repositories were included and these are all effectively just backups. Especially in the case of the vault in Svalbard. But you can opt out of the 'warm storage'[0]. [0] https://github.com/github/archive-program/issues/36 [1] https://docs.github.com/en/github/site-poli…

I recognize they wouldn't have done it unless they felt confident of having the legal right, but it's just bad manners not to ask first. If that's the case, this not-a-PR stunt degraded my impression of them. I'm quite certain this isn't what their customers contemplated when reading "backup" in their ToS. EDIT: Interestingly it says " This license does not grant GitHub the right to sell Your Content or otherwise dis…

>Also what happens if it turns out one of those repos had personal information in it and the subject makes a GDPR right-to-forget demand? Are they going to drag it out and purge that bit of tape?

I believe GDPR has exemptions for archives ([0] section 28) so that's less of a concern for them I imagine. I recognise what you're saying, but I think anyone _very_ opposed would have a difficult time in court arguing GitHub should remove their work/name/etc. My (very loose) understanding of the law is that they would have to demonstrate some kind of loss. That being said, GitHub could just have sent a notification email with very little effort. Maybe 'no harm, no foul' applies here?

[0] https://www.legislation.gov.uk/ukpga/2018/12/schedule/2/part...

Re: The GitHub Arctic Code Vault

#88
post #64
post #18

As David Rosenthal (formerly of Sun, NVIDIA, and Stanford) explains, the actual Arctic Code Vault is a PR stunt, and has almost no chance of helping anyone in any kind of realistic disaster scenario: https://blog.dshr.org/2019/11/seeds-or-code.html That said, the rest of the project, which focuses on preserving several independent copies of repositories hosted on GitHub with a handful of partner organizations, is qui…

Archive Program director here - it's really not a PR stunt, we genuinely believe it will be of significant historical value and quite a good chance it will be of practical value. Much of that is "if we forget technology which we realize somewhere down the road we actually might want to use again." History provides plenty of examples of this, and it's particularly important with a technology which mostly lives on ephe…

Hi Jon, Congratulations on moving forward with this. Thank you! If you ever think about what might come next in terms of being able to re-make computers and so on from scratch, here is a concept website I put up around 1999 (when I was trying to get NASA to support the work for space settlements). I still work on the general idea on-and-off in my spare time (generally at a more abstract level of software for sensemaking and organizing information) but so many other distractions get in the way: https://www.kurtz-fernhout.com/oscomak/goals.htm

From there: "The OSCOMAK project is an attempt to create a core of communities more in control of their technological destiny and its social implications. No single design for a community or technology will please everyone, or even many people. Nor would a single design be likely to survive. So this project endeavors to gather information and to develop tools and processes that all fit together conceptually like Tinkertoys or Legos. The result will be a library of possibilities that individuals in a community can use to achieve any degree of self-sufficiency and self-replication within any size community, from one person to a billion people. Within every community people will interact with these possibilities by using them and extending them to design a community economy and physical layout that suits their needs and ideas. As the internet has grown, it has enabled collaborative work which has created many success stories, including Linux, Python, GCC, Squeak and other projects. We want to harness that power and apply it to organizing technological knowledge in concert with many interested individuals. The main project goal is to develop an on-line library of technology ideas, techniques, and tools, including a range from high-tech processes like plastics to medium-tech like ceramic houses to low-tech like spinning wheels. Also included will be biotechnology processes, like perennial agriculture, companion planting, sheep farming, and eventually cloning and DNA synthesis. One process to be included is a way to convert the high-tech computerized library to a low-tech paper one as desired. Key to the whole endeavor will be to present everything in a how-to fashion. Also needed is a way to map out and simulate the interrelations of processes; for instance, sheep raising requires veterinarians, antibiotics, feed, fencing, and shears; shears require a blacksmith, metal, and a furnace. This latter feature also would be used to keep track of the product flows into, out of, and within a community's entire economy."

Re: The GitHub Arctic Code Vault

#89

Only disappointed that the new badge does not show the 2 open source projects I contributed to in the last 10 years of my work for open source :( They are not super big, but also not super small. Seems organisation work is ignored and only individual username fork/PRs respected (is this a bug?). Software is teamwork ;) I mean awesome-react, tldr-pages or homebrew-cask are probably not unimportant but that's not where…

I am not a huge GitHub user and have only contributed some code to a single repo that was merged. I was surprised to see I had the badge in my profile.

Re: The GitHub Arctic Code Vault

#90

Earlier quoted context omitted.

Right, so it wouldn't be a 2020 computer. It would be whatever new computer they've built.

By then, why would they need code from GitHub? Given that they will won't even be able to run it in any shape or form.

To study history and culture. And who knows, there may well be algorithms we came up with but which no one ever re-discovered.
Post reply on HN