Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Instagram kept deleted photos and messages on its servers for more than a year

theverge.com

Post reply on HN
Print view View original

191–200 of 205 posts

Re: Instagram kept deleted photos and messages on its servers for more than a year

#191

Post by notacoward »

endtime wrote: post #144
holidayacct wrote: post #116

Whenever someone publishes an article like this, I want them to find out what its like when you reach the threshold of 100000 individual delete requests per second which ends up being five million actual deletes when you factor in all of the associated references to the item being deleted and its metadata. Then I want them to find out what happens when you have to propagate those deletes across geographically distrib…

Google has an internal standard for how long it takes to wipe out user data after a deletion request, and compliance with it is taken very seriously. It's a lot less than 1 year.

...and I do not believe for one second that they have never discovered a copy of data lying around somewhere after they thought it was deleted. The choice is to admit it, or lie about it.

Re: Instagram kept deleted photos and messages on its servers for more than a year

#192

Post by lozf »

bryan_w wrote: post #179
lozf wrote: post #122

Earlier quoted context omitted.

What makes you think even deleting one's account will truly wipe data? These companies profit from personal info whether one is a customer or not. I would imagine everything exists with a "user de-activated since $DATE" entry or similar in their database. Perhaps I'm too cynical?

Actually I think the incentives are aligned here. If they can't show the pictures that they have claimed to have deleted then they can't use it to bring in eyes for advertisers, which means that the data is a net loss for them, causing them to have to buy more storage sooner. They should want to delete data as soon as possible

Hmm yes, they probably free up the bulk of space and just keep extensive metadata instead: "We don't have the picture, but we know you took one at date, time, location, make & model, shutter speed & aperture, and we regognized the faces of these 2 users X & Y, and another non-user for whom we have shadow profile Z wwas also tagged."

Re: Instagram kept deleted photos and messages on its servers for more than a year

#193

Post by areyousure »

jacquesm wrote: post #189
areyousure wrote: post #184

Earlier quoted context omitted.

So you have per-row or perhaps per-user keys? Do you back them up?

Not currently running a user visible service. Fortunately. The one I did run was started in 1998 and had turned into a very large pile of super insecure PHP and was shut down. If I were to start a user visible service today I would think really long and hard about whether or not the kind of superstructure required would be offset by the potential gains. The web is no longer a place for amateur projects with a focus o…

Sorry, what I meant was to clarify your proposal, not to ask what you literally do now.

I'm wondering, more or less, whether the key management is another case of backups and deletion being hard.

Re: Instagram kept deleted photos and messages on its servers for more than a year

#194

Post by Traster »

holidayacct wrote: post #116

Whenever someone publishes an article like this, I want them to find out what its like when you reach the threshold of 100000 individual delete requests per second which ends up being five million actual deletes when you factor in all of the associated references to the item being deleted and its metadata. Then I want them to find out what happens when you have to propagate those deletes across geographically distrib…

Woe is me. My poor little (looks at notes) trillon dollar company can't cope! Instragram seems to be able to cope with 100000 new photos per second - that they can do. Across geographies and data centres. Ever seen a user facing error due to a newly uploaded photo?

No one expects you to just rm -rf. They just expect you to expect the same effort to remove the file that they put in to create the file. It's not like the file got propogated to 10 different data centres by accident. There was a decision that an uploda needed to be everywhere in 5 seocnds and a deletion was an advisory hint that could be ignored for a year.

Re: Instagram kept deleted photos and messages on its servers for more than a year

#195

Post by jacquesm »

areyousure wrote: post #193
jacquesm wrote: post #189

Earlier quoted context omitted.

Not currently running a user visible service. Fortunately. The one I did run was started in 1998 and had turned into a very large pile of super insecure PHP and was shut down. If I were to start a user visible service today I would think really long and hard about whether or not the kind of superstructure required would be offset by the potential gains. The web is no longer a place for amateur projects with a focus o…

Sorry, what I meant was to clarify your proposal, not to ask what you literally do now. I'm wondering, more or less, whether the key management is another case of backups and deletion being hard.

Ah, apologies.

Key management is indeed just another - hopefully simpler - version of the same problem. The reason why it simplifies things is because a single key can invalidate a lot of data stored in places that are out of reach such as cold backups.

But you still have the same essential problem.

Re: Instagram kept deleted photos and messages on its servers for more than a year

#196

Post by jacquesm »

holidayacct wrote: post #116

Whenever someone publishes an article like this, I want them to find out what its like when you reach the threshold of 100000 individual delete requests per second which ends up being five million actual deletes when you factor in all of the associated references to the item being deleted and its metadata. Then I want them to find out what happens when you have to propagate those deletes across geographically distrib…

I find it very interesting that given a chance to hover up a billion records we see no technical challenge at all but when asked to get rid of them suddenly you're demanding unobtanium.

If you can't reliably get rid of the data you administer, don't collect it in the first place.

Re: Instagram kept deleted photos and messages on its servers for more than a year

#197

Post by jacquesm »

notacoward wrote: post #191
endtime wrote: post #144

Earlier quoted context omitted.

Google has an internal standard for how long it takes to wipe out user data after a deletion request, and compliance with it is taken very seriously. It's a lot less than 1 year.

...and I do not believe for one second that they have never discovered a copy of data lying around somewhere after they thought it was deleted. The choice is to admit it, or lie about it.

It has actually happened and they did come clean about it. Some data was discovered in log files that shouldn't have been there. They owed up and fixed it pronto even if the chances that the data was leaked outside of Google were nil. There is plenty wrong with Google, but that isn't one of those.

Re: Instagram kept deleted photos and messages on its servers for more than a year

#198

Post by FridgeSeal »

ben509 wrote: post #73
FridgeSeal wrote: post #33

> When you delete something from Instagram you expect it to be gone for good Ohh do I have some bad news for you. Seemingly everyone in this industry can’t or won’t design databases and applications to actually allow for data to be deleted.

Deletion is effectively a failure mode, and engineering tends to focus on the happy path.

Only if you treat it like one.

Arguably a well designed system should be built to handle having certain user data deleted without interrupting or breaking anything.

Re: Instagram kept deleted photos and messages on its servers for more than a year

#199

Post by gjs278 »

holidayacct wrote: post #116

Whenever someone publishes an article like this, I want them to find out what its like when you reach the threshold of 100000 individual delete requests per second which ends up being five million actual deletes when you factor in all of the associated references to the item being deleted and its metadata. Then I want them to find out what happens when you have to propagate those deletes across geographically distrib…

you’re just a bad programmer if you can’t do this. I could work for your company for a day and solve this

Re: Instagram kept deleted photos and messages on its servers for more than a year

#200

Post by jandrewrogers »

jacquesm wrote: post #190
jandrewrogers wrote: post #187

Earlier quoted context omitted.

Encryption at record granularity introduces two technical problems that no one has come up with a tractable solution for, and a solution likely doesn't exist. It is actually a discussion of fundamental tractability, not "efficiency". The law can declare that we should be able to break AES encryption too but that doesn't manufacture plausibility. First, encryption has a block size. Data field storage in databases is t…

The tricky bit to me is that fundamentally deletion should not be harder than insertion, but because we have historically only focused on the happy path nobody cares at all about the deletion mechanism. Whereas a secure erase-in-place of a field option alone would already take you to 80% or more of a workable solution. Perfection, the way you describe it is for now unattainable. But the bigger problem as far as my pr…

Sure, you can trivially design a data structure where insert and delete have the same cost. This works if you don’t care about query performance; most people care about query performance a great deal. This was litigated in the marketplace decades ago. Even every open source database rejects designs that produce rubbish query performance. It also does not reflect the real-world distribution of operations between insert and delete — we do a lot more inserts.

The “erase-in-place” operations you mention were common in databases a few decades ago and abandoned because the typical performance was terrible compared to the alternative. I am old enough I even implemented a few. It isn’t like these designs didn’t exist, they were deeply flawed for reasons that apply today.

This has nothing to do with perfection. Database engineers would love if deletes where inexpensive even in the absence of hard delete requirements, as that would make update operations, which people want, dramatically cheaper. But that isn’t the reality. You are essentially re-litigating settled database kernel engineering without understanding why they are designed the way they are.

If you thinking there is “low hanging fruit dragging on the ground” then I encourage you to prove it by designing a useful database kernel that can scale deletes while preserving insert and query performance, the two operations that drive all economics in databases. You’ll be instantly famous as a computer scientist because there are some nasty theoretical computer science problems in there.

Post reply on HN