Having backups is obviously always a good idea but that aside, how likely is it, that mirroring this data is actually necessary? There are obviously at least some people believing that it is necessary but then there are always people convinced that the most silly things are necessary. I am really unable to tell whether this is more of an overreaction or an urgent necessity, especially as someone not living in the US. Is the future just [perceived as] so unpredictable, that better safe than sorry is the right mindset?
Mirroring US climate data
11–20 of 166 posts
Re: Mirroring US climate data
#12Awesome. Is there the same thing but for all public datasets? I saw recently someone showed a federal site had put up a (hastily hacked in) link to download all data on the day of the inauguration. I bet there are a lot of federal employees who would like a well-resourced site to upload to.
Re: Mirroring US climate data
#13Having backups is obviously always a good idea but that aside, how likely is it, that mirroring this data is actually necessary? There are obviously at least some people believing that it is necessary but then there are always people convinced that the most silly things are necessary. I am really unable to tell whether this is more of an overreaction or an urgent necessity, especially as someone not living in the US.…
Besides redundancy, mirroring also helps spread the cost around. These groups tend to have pretty tight budgets.
Re: Mirroring US climate data
#14I'm sincerely dismayed that this is necessary.
Re: Mirroring US climate data
#15Having backups is obviously always a good idea but that aside, how likely is it, that mirroring this data is actually necessary? There are obviously at least some people believing that it is necessary but then there are always people convinced that the most silly things are necessary. I am really unable to tell whether this is more of an overreaction or an urgent necessity, especially as someone not living in the US.…
Think of it this way: servers cost money, bandwidth power costs money.
When your budget is cut, servers will shut down at some point in the future. It isn't a "the gov says rm -rf everything"
Re: Mirroring US climate data
#16Having backups is obviously always a good idea but that aside, how likely is it, that mirroring this data is actually necessary? There are obviously at least some people believing that it is necessary but then there are always people convinced that the most silly things are necessary. I am really unable to tell whether this is more of an overreaction or an urgent necessity, especially as someone not living in the US.…
Think of it this way: servers cost money, bandwidth power costs money. When your budget is cut, servers will shut down at some point in the future. It isn't a "the gov says rm -rf everything"
I totally see how data could become unavailable, I am more asking whether it just really became more likely to happen. It is certainly no mistake not to rely on single sources, but if that is the motivation, then there is nothing special about now and climate data and mirroring all kinds of data would have been as good an idea a decade ago as now.
Re: Mirroring US climate data
#17I'm sincerely dismayed that this is necessary.
It's not.
Re: Mirroring US climate data
#18Having backups is obviously always a good idea but that aside, how likely is it, that mirroring this data is actually necessary? There are obviously at least some people believing that it is necessary but then there are always people convinced that the most silly things are necessary. I am really unable to tell whether this is more of an overreaction or an urgent necessity, especially as someone not living in the US.…
If you only create redundancy when you are sure it's necessary, you're too late.
Re: Mirroring US climate data
#19Can I be the first to claim that the people running the mirror are changing the data for their own nefarious purposes? Not that I believe this or anything, but come on - you know that accusation is going to go around. It might make sense to use a remote server to timestamp files as they're uploaded, or something like that.
Before publishing, they could do a checksum, insert it into the blockchain and then voila. Easy to validate if the data has changed.
Re: Mirroring US climate data
#20Is this project collaborating with Archive Team? They have quite a lot of experience with this sort of scraping+archiving project (e.g. Geocities, Google Reader, Coursera, etc.).