Live data from Hacker News

Facebook engineers: we have no idea where we keep all your personal data

theintercept.com

101–110 of 137 posts

Re: Facebook engineers: we have no idea where we keep all your personal data

#101
post #11

In many ways you can think of large, long-living tech companies not unlike old cities like, say, London or Paris. The buildings and roads you see are built on top of older buildings and ruins. The streets are weirdly shaped and intersect at odd angles because they were made hundreds of years before and adapted over time as needs evolved. There are catacombs underneath sidewalks and no one genuinely understands it all…

Maybe, but one can find out. You can follow the chain of calls of any software, and they have access to everything, as well as the whole history. It's like when in an interview Buffets says it's very hard to know where all the money is in the financial system. Sure, it's a complex system, but it's more of a matter of incentive.

As someone who's arguably professionally employed to do things like tell people where money is in the financial system, no, you can't.

In the same way you can't enumerate the atoms or molecules in a human body. sure, you can come up with good estimates and summaries that are useful, but not the actual full and truthful state of affairs.

In addition, you are a self-referential participant in the system: so acting in an attempt to observe such likely immediately makes your information at least partially incorrect.

Additionally, if we can't get full/transparency or introspection into machine learning models or neural networks (and we can't because i don't believe we even possess the theoretical knowledge of how to do so), it should also be wilfully apparent that we can't actually know the true lineage or dependencies of systems that implement such in production.

Not to mention the difficulties as the inputs/outputs of such pass in and out of the notional entity or system being observed.

Re: Facebook engineers: we have no idea where we keep all your personal data

#102
I assume it's the same in every company after a particular amount of time. They start clean and then everything grows organically because you spend more money in innovation than cleaning up and documentation. It's not even a thing of competition or capitalism, but limited resources.

Re: Facebook engineers: we have no idea where we keep all your personal data

#103
post #73

Earlier quoted context omitted.

Counterpoint, in my career, I've seen a lot of Security Audits that seem on paper to be very thorough and detailed but when you dig deeper, you can see that it only touched the surface and the auditor missed a lot of things.

I'd guess there's a difference between an external audit performed for regulatory of financial reasons, and an audit by the internal legal team because of a pending legal case or regulatory scrutiny.

There's a very wide gap in what needs to be covered as a result of this consideration!

Disaster recovery/business continuity? Absolutely, you need it documented so that someone 'off the street' could be helpful.

Implementing yet another service gateway? Probably going to take a lot of conversations with people familiar with the streets involved.

Auditors won't care until it's realized. They'll want things like interconnection diagrams -- painting a picture so they can follow your adherence.

We have very high compliance requirements where I work -- about half of my time is spent on it.

They're less worried about how the bread is made, more that you can keep making bread the way you say you do.

Re: Facebook engineers: we have no idea where we keep all your personal data

#104
post #52
post #44

Earlier quoted context omitted.

> In that case, can I sue anyone using Stable Diffusion for stealing my data because the billions of images in its training set included something I created? No, because of the landmark Authors Guild, Inc. v. Google, Inc. case which found that ML training can be considered fair use in some circumstances.

So why do we expect Meta to internally track this kind of data usage? Seems like it’s both fair use and covered by the terms of service that FB users have agreed to.

Copyright and privacy law are two different animals.

Copyright law covers things that are expected to be published - i.e. creative works. We decide that authors get to control how and when they are published. Privacy law is different: the subject is information that is held in confidence and should not be published under any circumstance.

Fair use would circumvent the "copyright my nudes so I can DMCA them off Facebook" kind of action, but not, say, the GDPR. GDPR does not care about fair use, and its skeptical of TOS agreements.

Re: Facebook engineers: we have no idea where we keep all your personal data

#105
post #22

Earlier quoted context omitted.

The cynic in me says it's building your own job security. Documentation only really gets created if leadership forces the issue in my experience. You also get to sell the refactor/rewrite because "nobody understand the existing code".

I've been a programmer for 15 years and I've never seen it done for "job security". I think this is mostly about low tolerance for boredom of most programmers: writing code is exciting, documenting it is boring, so you do not do it unless forced to.

Ive seen people "neglect" to offer training peers or document things before to protect their own position. Anecdotes and all that.

I mean people "usually" wont say that out loud of course.

Re: Facebook engineers: we have no idea where we keep all your personal data

#106
post #33
post #11

In many ways you can think of large, long-living tech companies not unlike old cities like, say, London or Paris. The buildings and roads you see are built on top of older buildings and ruins. The streets are weirdly shaped and intersect at odd angles because they were made hundreds of years before and adapted over time as needs evolved. There are catacombs underneath sidewalks and no one genuinely understands it all…

I work for a large long-living tech company. Where legal compliance and security are concerned, "it is just layering and organic growth" is not something you get to say. If we were speculating about such a thing on a coffee break, maybe we'd get the kind of answer given here, but if a single reference doesn't exist for something that is necessary for compliance , people get paged until it does. (There may be other th…

> if a single reference doesn't exist for something that is necessary for compliance, people get paged until it does.

And you will trust the reference like if it perfectly describes the reality. Because, really, there is no other way to deal with it. You can test a thing or two and improve it, but you can't test for the absence of necessary information.

The Facebook situation is inexcusable. But no company has perfect knowledge about this either ("perfect" here carrying a lot of weight).

Re: Facebook engineers: we have no idea where we keep all your personal data

#107

What are we talking about here? Data associated with specific users? Well in order for your grandma to log into Facebook her user account must have a primary key associated with it so she sees her info when she logs in and not someone else’s. We are talking about computers and databases. When did using a computer to search a database become a difficult, nigh impossible thing to do? Even if design documents and flow c…

It became difficult when your 'database' turned into exabytes of random files in s3.

Re: Facebook engineers: we have no idea where we keep all your personal data

#108

Earlier quoted context omitted.

> In the end, you can't force competence through rules. I mean, we have in all other forms of engineering. Why is software different?

Sounds like a pretty good argument for 'software engineering is not engineering'.

I think we can all (mostly?) agree that most of what's called software engineering doesn't live up to the standards (including responsibilities of the developers/engineers) of the other formalised engineering disciplines.

Re: Facebook engineers: we have no idea where we keep all your personal data

#109

I recall when Windows 10 came out some company had demanded to know all of the things it sent back to Microsoft over the wire. Microsoft had some guy basically run wireshark and a bunch of network scans while using it and put that in a report. It blew my mind because it implied nobody at Microsoft actually knew what it was collecting and when anymore than your average security researcher, possibly because there are s…

When I worked in Redmond, they pressed us heavily to collect telemetry where possible. Each line of business basically operated as a separate company with many subsystems and no effective way of connecting customers or users between them. I have been at many big companies and seen similar. If these big companies with expertise and money can't manage their data well, it should be no surprise to anybody how poorly it is handled at smaller companies.

Re: Facebook engineers: we have no idea where we keep all your personal data

#110
post #35
post #17

After reading some more of the transcript, I think the article does such a bad job of describing what was being asked for that it makes the court seem incompetent and Facebook actually rather reasonable. My original comment is below: Government contracted construction workers: We Have No Idea Where Your Tax Money Goes When tasked with answering the simple question "Which specific bricks did my tax money buy this year…

That’s an absurd analogy that doesn’t fit at all. FB uses the data it can’t find or tell you anything about to successfully sell ad space to third parties targeted back at you. Let’s make it concrete. We know FB keeps track of which webpages you visit. We also know they use that data as a way to help target ads. That data isn’t in the data export they give you, as far as I can tell from a brief search. Is there data…

But it is not absurd. Facebook doesn't target ads at you, Bob Jones.

It takes all of Bob's activities and pushes them into bins based on how they characterize Bob; Male, lives in Iowa, 18-25, etc. a bin (Moves to LA, for example), his data will contribute to different bins. This activity is disconnected from Bob at this point; and the data is aggregated away from single interaction events. "Bob visited foo.com" as a single event is gone at this point.

The models grind on these aggregate data bins.

Then when ads are targeted at Males who live in Iowa, aged 18-25 -- those ads get shown to Bob, because he is tagged with those tags.

They don't "keep track of which webpages you visit", not for more than a day. Those events get pushed in large aggregate stores of activities pretty fast. These aggregate stores are vastly smaller than if you kept all the individual data, hence much cheaper.

Post reply on HN