Live data from Hacker News

Mirroring US climate data

climatemirror.org

151–160 of 166 posts

Re: Mirroring US climate data

#151
post #88

If you want to help towards archiving climate data and other gov't public datasets, join the ArchiveTeam effort. All you need is to run their ftp-gov-grab script and a bit of bandwidth. VPSs work very well for this sort of thing, so if you have a box lying around, just run the script. More info at the wiki: http://archiveteam.org/index.php?title=Ftp-gov Ftp-gov-grab: https://github.com/ArchiveTeam/ftp-gov-grab Join u…

Correction: it seems to be #cheetoflee on efnet.org

oops, thanks for that.

Re: Mirroring US climate data

#154
post #25

Can I be the first to claim that the people running the mirror are changing the data for their own nefarious purposes? Not that I believe this or anything, but come on - you know that accusation is going to go around. It might make sense to use a remote server to timestamp files as they're uploaded, or something like that.

I would say the same thing. Do you think you will really convince a climate denier with this data that you copied off of the Internet that, was copied from a gov server, on to another server and then torrented and copied to another server and then downloaded onto you PC... doubt it.

I work for a digital archives project and we're concerned about similar issues of provenance. Our way of dealing with this has been to structure our archives as JSON-formatted text in Git repositories, with binaries managed by git-annex.

Git uses hashes for everything: files are placed in .git/objects/ by their hash, each commit lists the hashes of files in the working directory, and each commit points to the hash(es) of its parent(s)

Using Git it's possible to verify the integrity of the entire repository and its history, making it impossible to tamper with without leaving traces. It's also possible to have multiple copies (clones) of the data and to verify that they are exactly the same.

If you have a copy of such a collection you can compare it against a copy held by the originating institution. If that becomes impossible for some reason, then if you can track down any of the original files you could prove that that portion of the data is correct, which lends trust to the rest of the collection.

Of course, it's possible this would not convince a die-hard trump supporter. /s

Re: Mirroring US climate data

#155
post #38

Another useful tactic is to archive news and commentary (left and right) about current political events, both for accountability and perhaps, to prevent the rewriting of history in the era of "alternative facts".

It is not sufficient to have a record of facts when others have alternative facts. To your signed record of facts they would just introduce alternate facts with alternative signatures and claim you had made it up. Normal people don't have the tools or experience to differentiate, and the alternative facts support a story they want to believe, rather than hard truths. News media won't even mention your so-called 'real…

Angry_octet I love you. this was one of the best comments..

Re: Mirroring US climate data

#156
post #59
post #54

Earlier quoted context omitted.

Is there any good way to timestamp things like this?

Hash the content and store the hash in the Bitcoin blockchain.

Or store it in a Git repository, the "stupid content tracker". If you have binaries use git-annex.

Re: Mirroring US climate data

#157
post #13
post #11

Having backups is obviously always a good idea but that aside, how likely is it, that mirroring this data is actually necessary? There are obviously at least some people believing that it is necessary but then there are always people convinced that the most silly things are necessary. I am really unable to tell whether this is more of an overreaction or an urgent necessity, especially as someone not living in the US.…

Besides redundancy, mirroring also helps spread the cost around. These groups tend to have pretty tight budgets.

Lots of copies keeps stuff safe.

Re: Mirroring US climate data

#158
post #76

I heard that Trump has given NOAA 100 days to justify their existence, or face heavy cuts. Does anyone know the details? Here is the closest article I could find: http://www.salon.com/2016/11/23/politicizing-climate-change-...

Why shouldn't government agencies have to justify their budgets and operations periodically?

It's called a budget review. It's Congress's job to do this, not the Administration. And it seems like a total waste of time compared to all of the other more important issues the Administration should be covering, IMO.

Re: Mirroring US climate data

#159

Earlier quoted context omitted.

But find the image of the prop half out of the water. Thats the super-secret thing they were so embarrased about. Those images have been surgically removed from the internet. The full story involves a dilema between forcing the boat off the ground, and risk stripping its super-secret rubber coatings that might then be collected by russian agents, or letting the tide go out and expose the prop to the media.

Still, I'd expect media companies to be more likely to comply with orders to remove military secrets than to remove scientific data. But you're right, censorship happens. How do we prevent it?

The UK in particular doesn't seem to have ever been overly enamoured with the idea of an intrinsic right to unlimited free speech.

https://en.m.wikipedia.org/wiki/Censorship_in_the_United_Kin...

Post reply on HN