In many ways you can think of large, long-living tech companies not unlike old cities like, say, London or Paris. The buildings and roads you see are built on top of older buildings and ruins. The streets are weirdly shaped and intersect at odd angles because they were made hundreds of years before and adapted over time as needs evolved. There are catacombs underneath sidewalks and no one genuinely understands it all…
Facebook engineers: we have no idea where we keep all your personal data
91–100 of 137 posts
Re: Facebook engineers: we have no idea where we keep all your personal data
#92Earlier quoted context omitted.
If more industries did carry this governance burden, we'd see a lot less cases of companies getting hacked left and right. For way too many companies, "IT" is just another cost center that doesn't bring in any profits and so is barely kept at life support levels. And when something inevitably happens, it's rarely the C-level that gets to go to the unemployment office, it's more one poor soul who has complained for ye…
Hah, I might have agreed before I worked on a credit card processing system. There were plenty of rules alright, but it's possible to abide by all the rules without actually making a secure system. Some of the rules made it harder to create a secure system because they did not reflect the state of the art, or they made adding additional security too burdensome to implement. In the end, you can't force competence thro…
I mean, we have in all other forms of engineering. Why is software different?
Re: Facebook engineers: we have no idea where we keep all your personal data
#93In many ways you can think of large, long-living tech companies not unlike old cities like, say, London or Paris. The buildings and roads you see are built on top of older buildings and ruins. The streets are weirdly shaped and intersect at odd angles because they were made hundreds of years before and adapted over time as needs evolved. There are catacombs underneath sidewalks and no one genuinely understands it all…
I work for a large long-living tech company. Where legal compliance and security are concerned, "it is just layering and organic growth" is not something you get to say. If we were speculating about such a thing on a coffee break, maybe we'd get the kind of answer given here, but if a single reference doesn't exist for something that is necessary for compliance , people get paged until it does. (There may be other th…
In my experience with the above, people get paged until a "socially plausible fiction or resolution" can be created.
There's several versions of these, one of which is to engage in enough rituals and processes so that we all pretend the task has been completed and the requisite outputs and points of reference exist. An archaeologist knows they don't and they can't and that we lay on bedrocks of speculation, history, ignorance and myth, but we don't have to worry too much about the fiction we create because we also know that disproving such mechanically is also extremely impossible/unlikely.
And we have a host of other social functions to get out of the cases where it could, such as:
- throwing people under the bus/human sacrifice/ finger pointing
- contrition
- plausible deniabilty
- continued non-compliance and non-enforcement
- bribery
- bullshit jobs
Re: Facebook engineers: we have no idea where we keep all your personal data
#94In many ways you can think of large, long-living tech companies not unlike old cities like, say, London or Paris. The buildings and roads you see are built on top of older buildings and ruins. The streets are weirdly shaped and intersect at odd angles because they were made hundreds of years before and adapted over time as needs evolved. There are catacombs underneath sidewalks and no one genuinely understands it all…
This is a seductive notion, but it's indeed possible for a tech company to understand how a single piece of data goes through its systems. It might take a while, and it might involve weeks of code spelunking and dozens of conversations with engineers — but it is indeed possible. A codebase is not encumbered with the physical constaints of an old city. It's not like trying to figure out the physical state of a certain…
You might have two independent datasets that can't be stored together due to privacy or security regulations. The teams who work on that know and understand that. But some other team working on something completely unrelated might join those datasets for some non-nefarious purpose, which then creates a new dataset with those data combined. Now other teams who also don't know about the regulations might build on top of that combined dataset. They might even generate new data fields from the data that's not supposed to be joined. This stuff can propogate through layers of systems, queries, service calls, analytical engines, offline spreadsheets, etc. It's very difficult to keep track while keeping your engineering teams independent and nimble.
Re: Facebook engineers: we have no idea where we keep all your personal data
#95Earlier quoted context omitted.
Counterpoint, in my career, I've seen a lot of Security Audits that seem on paper to be very thorough and detailed but when you dig deeper, you can see that it only touched the surface and the auditor missed a lot of things.
I mean, sure, security is complicated, lots of judgment calls, but within this mindset, if a team/org/company said "we have no idea where we're using log4j" it was then their job to have someone working around the clock until they figured that out and fixed it . Figuring out how to meet new requirements that weren't around when systems were begun is, IMO, fully just part of the job. For FB engineers to be throwing up…
- reality
- the socially acceptable fiction in the report
the exercise would be to create an output that allows people to pretend we know where all instances of "log4j" were used, or that sufficient depth and resources have been expended on such.
but in a sufficiently large and complex organisation, we (and by we I mean technically able, knowledgeable or sufficiently aware individuals) know that the content of such is likely to be as factually true as reading a historian's report entitled: "enumeration of all points in past century where cutlery has been used".
Re: Facebook engineers: we have no idea where we keep all your personal data
#96Earlier quoted context omitted.
Hah, I might have agreed before I worked on a credit card processing system. There were plenty of rules alright, but it's possible to abide by all the rules without actually making a secure system. Some of the rules made it harder to create a secure system because they did not reflect the state of the art, or they made adding additional security too burdensome to implement. In the end, you can't force competence thro…
> In the end, you can't force competence through rules. I mean, we have in all other forms of engineering. Why is software different?
Re: Facebook engineers: we have no idea where we keep all your personal data
#97Earlier quoted context omitted.
Yes and it’s a good thing that most industries do not carry this governance burden.
I think specifically any industry working in personal data collection of minors should absolutely have this governance burden. If Meta knows enough about their users to know Instagram is giving teenage girls mental illnesses, Meta engineers should absolutely be hoisted out of their beds if they don't have precise, well-defined, secure data stores.
If the issue is about teenagers being too immature for the product X, then typically we ban it or age restrict it using legislation (eg tobacco, alcohol, porn, etc)
I think we as a society need to figure out where social media fits in and how we want to contain it and it's still early days for that. As much as I like the idea of an age restriction, I look around the world and see 6 month olds playing with iPads, 6 year olds on tiktok, etc. as the more 'average' way to parent and wonder if I'm in the minority. My kid is 4 and has never touched a device as entertainment, we taught him how to eat out at restaurants/be in public instead of pacifying him. But that seems like an old fashioned parenting method.
Ok, he's technically used it briefly as a viewfinder for the camera and to facetime his grandparents
Re: Facebook engineers: we have no idea where we keep all your personal data
#98In many ways you can think of large, long-living tech companies not unlike old cities like, say, London or Paris. The buildings and roads you see are built on top of older buildings and ruins. The streets are weirdly shaped and intersect at odd angles because they were made hundreds of years before and adapted over time as needs evolved. There are catacombs underneath sidewalks and no one genuinely understands it all…
You can hire one or more data privacy people whose whole job it is to track this. They get an inventory of all the systems and talk to the managers of every system, find the answers and document them. Meta is choosing not to know or pretending not to know. They have the resources to know where your data goes.
Re: Facebook engineers: we have no idea where we keep all your personal data
#99In many ways you can think of large, long-living tech companies not unlike old cities like, say, London or Paris. The buildings and roads you see are built on top of older buildings and ruins. The streets are weirdly shaped and intersect at odd angles because they were made hundreds of years before and adapted over time as needs evolved. There are catacombs underneath sidewalks and no one genuinely understands it all…
Processes to access personal data vary wildly among tech companies. At Apple, when a machine learning engineer wants to store and use data on the server side, they need to go through layers of approval from lawyers. Even when approved, it often comes with serious constraints about what can be done with the data. Meta is a lot more lax.
Re: Facebook engineers: we have no idea where we keep all your personal data
#100In many ways you can think of large, long-living tech companies not unlike old cities like, say, London or Paris. The buildings and roads you see are built on top of older buildings and ruins. The streets are weirdly shaped and intersect at odd angles because they were made hundreds of years before and adapted over time as needs evolved. There are catacombs underneath sidewalks and no one genuinely understands it all…
This is a seductive notion, but it's indeed possible for a tech company to understand how a single piece of data goes through its systems. It might take a while, and it might involve weeks of code spelunking and dozens of conversations with engineers — but it is indeed possible. A codebase is not encumbered with the physical constaints of an old city. It's not like trying to figure out the physical state of a certain…
If it takes a while it will be out of date by the time it is complete.
A single request might directly impact a dozen separate systems, which in turn call other services. You won't be able to find anyone who knows much past the layer they directly interact with. All of these services are being worked on. Some older ones are in the process of being deprecated in favor of newer ones. Someone is already working on the design of the replacements for the newer ones. Changes are being pushed weekly if not daily. If you send a request and then send it again a minute later, it might go through a completely different set of services based on a feature toggle.
You can't step into the same river twice.