I worked at Facebook for most of 2017 and 2018. In the first week, they made it clear that you would be fired instantly for any improper access of user data. They further said that if you need to access any sensitive personal data, or if you need to log in as a user in order to debug a problem, you need to have approval from your manager _before_ the access, not after. Also, you are not allowed to access the data of…
>They further said that if you need to access any sensitive personal data, or if you need to log in as a user in order to debug a problem, you need to have approval from your manager _before_ the access, not after. But were you still able to just look at the data or login as the user without the permission? I think that's the key question. Talk is cheap. As a user it's not good enough for me that people are being tol…
A Facebook engineer abused access to user data to track down woman
81–90 of 108 posts
Re: A Facebook engineer abused access to user data to track down woman
#82Earlier quoted context omitted.
I know an engineer, a security engineer at Google who is pretty well-known, who went to work at Google specifically so he could get at peoples personal data. I don’t know if he actually does it, but he boasted quite openly for years that he wanted to be the “architect“ and see everything and know everyone’s secrets. He is now a highly placed Google security employee.
There are no “architect” roles at google. Most people outside anti-abuse roles have no access to user data, and even the abuse people use audited frontend tools that formalize the policy that the viewer must reference the ticket they are working on and the limited data they need to see. People with direct access to production data streams mostly see encrypted data. There’s a big, annoying technical scheme in place to…
In this case I suspect "The Architect" is a reference to the character in the Matrix films. The character who has visibility into every corner of the Matrix and full knowledge about everything going on inside it.
Could be wrong. Either way I very much doubt it is related to a specific role at Google.
Re: A Facebook engineer abused access to user data to track down woman
#83> At the time, more than 16,000 employees had access to users’ private data, according to the book. > Stamos suggested tightening access to fewer than 5,000 employees and fewer than 100 for particularly sensitive information like passwords. I'm sorry, what? I can tell you the number of legitimate engineers that should have access to user's passwords. It's a nice, round number. It's zero.
It’s not zero. Hashed passwords are still passwords and should be treated as such. “Zero” implies that hashed passwords are not passwords, since otherwise you won’t get to zero. Just because passwords are hashed doesn’t mean you can give access to them willy nilly and happily claim that “zero” people have password access.
Agreed.
> “Zero” implies that hashed passwords are not passwords, since otherwise you won’t get to zero.
You can get to zero:
- No humans in the serving path servers' ACLs.
- Diagnostic/recovery servers for humans which require the person submit a justification that links to a ticket/bug/outage, wait for a second person to approve, perform high-level operations that affects sensitive data ("restore user from backup at timestamp T") rather than exposing direct access ("read from backup", "write live user"), and keep an audit record for later.
Everything is about trade-offs. This approach takes more engineering time to set up and if not done well can really slow down common tasks. And there are certainly reasons there might be exceptions—eg allowing the primary on-call to have unilateral access can speed recovery over waiting for a second person to be available. But zero is possible, and stories like this remind us of its value.
Re: A Facebook engineer abused access to user data to track down woman
#84Earlier quoted context omitted.
This is completely unethical and unreasonable. It's like arguing that police don't need more accountability because it makes it harder for them to do their jobs, and most of them aren't bad people, so who cares about a few bad apples? Yeah it sucks, but it's part of the job. Start thinking about the people you're supposedly serving instead of yourself first. I'm pretty sure that the overwhelming majority of facebook…
They are well audited already. Does every step possible need to be taken to ensure that no data can be leaked ever? No. You can walk out your door right now and hop on a bus. That driver has a CDL, a good first step. But how do we know that the driver isn't drunk? Through threat of possible audit (breathalyzer) after any incident. We don't test them before handing them the keys every day. We trust people all the time…
Re: A Facebook engineer abused access to user data to track down woman
#85Earlier quoted context omitted.
This is completely unethical and unreasonable. It's like arguing that police don't need more accountability because it makes it harder for them to do their jobs, and most of them aren't bad people, so who cares about a few bad apples? Yeah it sucks, but it's part of the job. Start thinking about the people you're supposedly serving instead of yourself first. I'm pretty sure that the overwhelming majority of facebook…
They are well audited already. Does every step possible need to be taken to ensure that no data can be leaked ever? No. You can walk out your door right now and hop on a bus. That driver has a CDL, a good first step. But how do we know that the driver isn't drunk? Through threat of possible audit (breathalyzer) after any incident. We don't test them before handing them the keys every day. We trust people all the time…
I work at a competitor to Facebook in a user-facing service and have these kind of restrictions in place (must request access with justification, otherwise I literally don't have ACLs to see the data). It's a non-issue because I run into it at most 1-2 times a month, usually far less.
Re: A Facebook engineer abused access to user data to track down woman
#86Earlier quoted context omitted.
Disclaimer: was at FB in 2014 You could at the time start trying to log in as a user and MULTIPLE red warnings came up that proceeding further would automatically notify your manager and skip of access and a reminder of data policies. Now at that point I did not go further but I did know that content moderation and security teams had special access so I imagine they did both, heavily warn avg FB eng AND restrict acce…
How about people with direct database access?
Re: A Facebook engineer abused access to user data to track down woman
#87Earlier quoted context omitted.
Is it not possible to only have the hashes or does it have to get persisted somewhere in the process?
Usually it's in the logs. So small number of SREs can sometimes access them (if there are logged). And even if they are not logged, they can always show up during tcpdump debugging of network issues and such. Client side hashing could solve this, but almost no one does it.
That is because it doesn't solve the problem. If you implement client-side hashing and then let your engineers see what arrives from the client, they will be just as able to log in as the snooped user; the client-side hash has added nothing.
What a client-side password hash does is to conceal the user's password from the user. There are a few types of password-related attacks:
1. If I learn your password, I can log in as you.
2. If I learn your password, I can log in as you, on other sites where you use the same password (and I can find your account, perhaps because you also use the same username or email address).
3. If I learn your hashed password, I can try to crack it.
4. If I dump the site's hashed password database, I can learn who else has the same password as you. (Because the hashes are the same.) Cracking one of them will crack everyone else's -- efficiency!
I might learn your password in a few different ways. Maybe it's "ncc1701d". Maybe I snooped your network traffic. Maybe I found the hash and cracked it.
So what are the solutions?
#1. There is no solution; this is working as intended. That said, HTTPS operates in this area, by making it more difficult for people to learn your passwords by snooping your network traffic.
#2. You, the user, would have to use different passwords for different accounts.
#3. I, the website operator, should process your password with a hashing function that is tuned to be slow.
#4. I, the website operator, should salt the passwords so that two identical passwords on two different accounts on my site produce different salted hashes.
Hashing the password on the client side and sending the hash over the wire does not address attacks 1, 3, or 4. They all work just as well regardless. It might show up if I try to perform attack #2.
This attack involves me (1) learning your password on one site, and then (2) using it to log in to your account on a second site. We will compare strategy (A) -- you enter your password, the site hashes it on your side, an intermediate hash is transmitted to the server, the server hashes it again, and finally the server stores the final hash in a database -- with strategy (B): you enter your password, the site transmits it in plain text to the server, the server hashes it, and finally the server stores the hash in a database.
In strategy (A), you have three passwords, plaintext, intermediate, and final; for purposes of parallelism, the same is true in strategy (B), except that your intermediate password is identical with your plaintext password.
In the case where I learn your final hashed password by dumping the site's database, there is no difference. In both cases, I will attempt to crack that password by guessing a likely password and running it through the hashing process.
In the case where I learn your plaintext password, there is also no difference. That's just attack #1.
In the case where I learn your intermediate password, perhaps by snooping network traffic or server logs, I can immediately perform attack #1 against your account on the particular site, because -- for purposes of that site only -- I have learned your plaintext password. I can also perform attack #3, cracking your hashed password, against your accounts on any and all other websites. But, in strategy (A), I cannot immediately perform attack #2 -- my attempt to crack your password would have to succeed first. If you have a weak password, it will. If you have a strong password, it probably won't.
This is very weak tea, which is why, working as a security consultant, we considered the defense you describe, "passing the hash", to be a red flag, and always recommended against it.
In the case being discussed here, where Facebook is explicitly worried about engineers viewing users' Facebook accounts, as opposed to their eBay accounts, client-side hashing has literally nothing to add.
Re: A Facebook engineer abused access to user data to track down woman
#88Earlier quoted context omitted.
Usually it's in the logs. So small number of SREs can sometimes access them (if there are logged). And even if they are not logged, they can always show up during tcpdump debugging of network issues and such. Client side hashing could solve this, but almost no one does it.
> Usually it's in the logs. That is definitely not usual
Re: A Facebook engineer abused access to user data to track down woman
#89Earlier quoted context omitted.
> Usually it's in the logs. That is definitely not usual
It's pretty common; a lot of places have blanket logging and it hasn't occurred to them to disable it for login attempts. It is obviously undesirable.
By default nether Apache nor Nginx log any post data. So with the 2 most popular options you actually have to go out of your way to enable this.
On the application side I mostly know Rails and it redacts even password hashes.
Re: A Facebook engineer abused access to user data to track down woman
#90I worked at Facebook for most of 2017 and 2018. In the first week, they made it clear that you would be fired instantly for any improper access of user data. They further said that if you need to access any sensitive personal data, or if you need to log in as a user in order to debug a problem, you need to have approval from your manager _before_ the access, not after. Also, you are not allowed to access the data of…
Yet TFA contains a quote about how abusing personal data is "against Mark's DNA". Horseshit.
Facebook is the enemy.
[0] https://www.esquire.com/uk/latest-news/a19490586/mark-zucker...