Live data from Hacker News

Verizon/Yahoo Blocking Attempts to Archive Yahoo Groups – Deletion: Dec. 14

modsandmembersblog.wordpress.com

351–360 of 424 posts

Re: Verizon/Yahoo Blocking Attempts to Archive Yahoo Groups – Deletion: Dec. 14

#351

disclaimer: I'm a Member of Archive Team who's helping coordinate the joining of Yahoo Groups in preparation for archival. Yahoo's banning of a large amount of the accounts we were using is a huge setback for us. In total we lost over access to over 55,000 Yahoo Groups, many of these will now not be archived and will be lost when Yahoo deletes everything on December 14. Particularly disastrous was the loss of access…

Shoutout to https://github.com/dessant/buster by the way!

`Buster is a browser extension which helps you to solve difficult captchas by completing reCAPTCHA audio challenges using speech recognition. Challenges are solved by clicking on the extension button at the bottom of the reCAPTCHA widget.`

Re: Verizon/Yahoo Blocking Attempts to Archive Yahoo Groups – Deletion: Dec. 14

#353

disclaimer: I'm a Member of Archive Team who's helping coordinate the joining of Yahoo Groups in preparation for archival. Yahoo's banning of a large amount of the accounts we were using is a huge setback for us. In total we lost over access to over 55,000 Yahoo Groups, many of these will now not be archived and will be lost when Yahoo deletes everything on December 14. Particularly disastrous was the loss of access…

Shoutout to https://github.com/dessant/buster by the way! `Buster is a browser extension which helps you to solve difficult captchas by completing reCAPTCHA audio challenges using speech recognition. Challenges are solved by clicking on the extension button at the bottom of the reCAPTCHA widget.`

That's nice, but it doesn't scale. Google only let you solve a few (5 or so) audio captchas in quick succession before you're banned for a while, so it's no good for us.

Re: Verizon/Yahoo Blocking Attempts to Archive Yahoo Groups – Deletion: Dec. 14

#355
Just thinking out loud: This makes me wonder if we can learn from this and prepare to backup other (similar) platforms that hold such an amount of data and might go away some day. Building the backup tools today and ideally starting to backup now, making the process incremental so you can run it every now and then and only scrape the new stuff.

Re: Verizon/Yahoo Blocking Attempts to Archive Yahoo Groups – Deletion: Dec. 14

#356

Earlier quoted context omitted.

Mark it inaccessible and write the new data over it in the next write cycle.

That doesn't work on SSDs, and the data might be even theoretically recoverable on HDDs : https://security.stackexchange.com/questions/12503/can-wiped... > Therefore, you should assume there is no reliable way to securely erase individual files on a SSD; you need to sanitize the whole drive, as an entire unit. There's a reason why when security is deemed important, the storage is physically destroyed instead.

I meant for the sake of data protection, not for forensics. You start with all ones and gradually deplete your ability to write ones over time in electron charge memories such as SSDs.

Re: Verizon/Yahoo Blocking Attempts to Archive Yahoo Groups – Deletion: Dec. 14

#357

Earlier quoted context omitted.

1. Terms of use can change at any time. 2. Technical capabilities have expanded massively. When Yahoo Groups launched, enterprise storage of more than a few hundred GB was highly unusual. I worked for a Very Impressive Service Agency which was lucky to claim two Sun Starfire servers, only one of which was Large File (> 2 GB) at about the time, for analytic use. By the late 2000s, AOL were deploying massive-RAM based…

Social cost would be at best very difficult to quantify, though, making it quite hard to handle. "Increased partisan tensions" due to social media, for instance, is not the sort of thing the cost of which one can quantify and mitigate. Your point that the things which can be done with information collected are constantly in flux, and I agree the ability to retroactively change terms of service to cover previously-col…

My own thinking on this has evolved very considerably over the past five years or so. That's included a comprehensive and ongoing exploration of the fields of media, communications, epistemology, and several others, related to this. I'd long seen computers as technology, largely independent of social implications. I now see these as utterly inextricably linked, and with implications that are anything but predictably benign.

Costs being difficult to assess does not mean impossible, and the notions of probability and risk are central to all finance, investment, and insurance. Uncertainty is NOT an absolute lack of knowledge.

Among the principles that becomes apparent is that changes in informational regimes have profound impacts upon societies, and that this is a pattern which can be traced back through history to the invention of writing itself, and via indirect anthropological evidence likely to the emergence of speech.

The principle transcends humans themselves -- a leading theory for the Cambrian Explosion is that it was a consequence, effecively, of structuring and communications mechanisms within organisms developing, and allowing the creation of complex body plans, and not merely single-celled organisms or masses or colonies of cells.

For media, see especially Elizabeth Eisenstein's The Printing Press as an Agent of Change and Marshall McLuhan's The Gutenberg Galaxy. The link between mass media and totalitarian, fascist, authoritarion, and nationalist sentiments has long been observed (Hannah Arendt, Dwight MacDonald, the Frankfurt School, Edward Herman & Noam Chomsky, Adam Curtis).

I've been impressed by the insight, or occasionally, lack, of awareness of the potential perils of comprehensive data archives by pioneers within the data field.

Paul Baran, co-inventer of packet-based networking, wrote "On the Engineer's Responsibility in Protecting Privacy" (https://www.rand.org/pubs/papers/P3829.html) in 1968, some 51 years ago. In it he remarked on both the risks, and industry attitudes:

There are many amongst us who would not hesitate to build equipment to compromise the privacy of any given individual provided the price is right. These are the whores of industry. They would not hesitate building systems and devices contrary to the public interest; their only concern is the buck.

The full paper, and in fact, all of Baran's RAND publications, are online in full-text, following my request to RAND. I remain grateful to them for this.

Baran was also interviewed for a 1966 BBC documentary:

"Well, he who has access to information controls the game. This is very dangerous. I think both your country and mine have never trusted the government completely. We do so for good reason. Here we have a mechanism that could be abused. Here we have a mechanism that would allow the creation of a dictator. . .

I've yet to see an expression by anyone in Congress about this new type of danger. In fact, we see proposals for centralizing information, we see proposals for rushing ahead into new, more efficient computer information systems, and very little thought is being given to the dangers of the misuse of these systems. . . I ask a lot of people about privacy, why they valued it, and I was surprised by the number of people who said "Well, I don't do anything wrong. Why should I worry about privacy?" And then, on the other hand, I think there's a more wise group that says, 'Privacy is really the right to be wrong, then go on and live the rest of your life, without having it mark you forever.' I tend to think this latter view is the view we should hold.

https://invidio.us/watch?v=FwaDvJYZTVk&t=29m31s

Another view was expressed by AI pioneer and Nobel Laureate (economics) Herbert Simon:

"The privacy issue has been raised most insistently with respect to the creation and maintenance of longitudinal data files that assemble information about persons from a multitude of sources. Files of this kind would be highly valueable for many kinds of economic and social research, but they are bought at too high a price if they endanger human freedom or seriously enhance the opportunities of blackmailers. While such dangers should not be ignored, it should be noted that the lack of comprehensive data files has never been the limiting barrier to the suppression of human freedom. The Watergate criminals made extensive, if unskillful, use of electronics, but no computer played a role in their conspiracy. The Nazis operated with horrifying effectiveness and thoroughness without the benefits of any kind of mechanized data processing."

https://pdfs.semanticscholar.org/a9e7/33e25ee8f67d5e670b3b7d....

There is, of course, one slight problem with Simon's argument: The Nazis did make heavy use of mechanised data processing, provided and supported by IBM. Edwin Black documents this meticulously in his book IBM and the Holocaust:

https://ibmandtheholocaust.com

Re: Verizon/Yahoo Blocking Attempts to Archive Yahoo Groups – Deletion: Dec. 14

#358
This is very sad to hear. One particular piece of Yahoo groups that I particularly care about is the LTSpice group, which contained a lot of simulation models I've not been able to find anywhere else. Luckily, this particular one seems to have been migrated to https://groups.io/g/LTspice

Re: Verizon/Yahoo Blocking Attempts to Archive Yahoo Groups – Deletion: Dec. 14

#359
post #267
post #265

Don't use free corporate services for shit you care about. Or think you may care about later. Don't use any service that suffers from a single point of control. How much anguish when Facebook inevitably either goes away or pivots entirely? Or HN, for that matter?

I don't believe that point is necessarily up for debate. At this point we are just trying to save the data that we know will be lost.

It might not be up for debate, but it's a good reminder.

Maybe Hacker News should be mirrored on Usenet...

Re: Verizon/Yahoo Blocking Attempts to Archive Yahoo Groups – Deletion: Dec. 14

#360

Earlier quoted context omitted.

Shoutout to https://github.com/dessant/buster by the way! `Buster is a browser extension which helps you to solve difficult captchas by completing reCAPTCHA audio challenges using speech recognition. Challenges are solved by clicking on the extension button at the bottom of the reCAPTCHA widget.`

That's nice, but it doesn't scale. Google only let you solve a few (5 or so) audio captchas in quick succession before you're banned for a while, so it's no good for us.

It's been working for me instead of clicking on all the little busses or crosswalks, even if it doesn't work at scale. Thought it might help some other users of the extension.
Post reply on HN