Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Former employees say Lyft staffers spied on passengers

techcrunch.com

Post reply on HN
Print view View original

241–250 of 253 posts

Re: Former employees say Lyft staffers spied on passengers

#241

Post by daenney »

ryan_j_naughton wrote: post #122

Having seen this at too many companies, we at fair.com decided to adopt stronger policies to prevent this, viz: - all inbound API requests first go to our API proxy in the secure layer. - the API proxy encrypts all PII using the encryption service in the secure layer - then API proxy sends the request on to the appropriate service, having swapped all PII for tokens. - all services in the general layer are not able to…

Would you please consider open sourcing some of this? If not the code then perhaps a more comprehensive design doc, explaining the different components involved, some API samples and flows so people that want to can recreate this?

Re: Former employees say Lyft staffers spied on passengers

#242

Post by telchar »

jacquesm wrote: post #233
telchar wrote: post #174

Earlier quoted context omitted.

Exactly, barriers to looking up a patient's information can be fatal. I need to give someone medication now to stabilize them. What medications are they on now? Can't look it up? Better give it to them and hope there's no adverse reaction. Better to avoid that situation and implement auditing while making sure people know the rules are enforced.

"override authentication" -> You have chosen to override the authentication protocol, you are logged in as John Doe, all your actions will be subject to internal affairs review, continue yes / no?

That tends to happen when the user logs in, so they'll probably see it multiple times per day. I think that's pretty standard. The system doesn't have to be the wild west. But if someone can click through an authentication override then it's not really doing anything.

Re: Former employees say Lyft staffers spied on passengers

#243

Post by misterbowfinger »

marshallbrekka wrote: post #180
misterbowfinger wrote: post #173

Earlier quoted context omitted.

Would you consider open sourcing it? It seems quite interesting and valuable

It's so specific to our architecture that there would be little value in open sourcing it. That being said, there are some lessons we learned along the way that I do think are worth sharing. As I mentioned above the API proxy supports REST and gRPC API calls. For REST based APIs, we developed a YAML syntax for declaratively specifying (per route) the input/output keys that needed to be encrypted/decrypted. This is ac…

I suppose it's difficult to answer what's specific to your architecture..... but it does seem interesting. I really wonder if there can be a open-source solution that makes it easier to do what you're suggesting, but for anyone.

Re: Former employees say Lyft staffers spied on passengers

#244

Post by dboreham »

ryan_j_naughton wrote: post #122

Having seen this at too many companies, we at fair.com decided to adopt stronger policies to prevent this, viz: - all inbound API requests first go to our API proxy in the secure layer. - the API proxy encrypts all PII using the encryption service in the secure layer - then API proxy sends the request on to the appropriate service, having swapped all PII for tokens. - all services in the general layer are not able to…

To add to this (which is excellent btw), we have taken the following measures:

Rate-limiting on admin interfaces and APIs (so a "rogue" human with admin rights can't just suck out PII in bulk).

Access controls on sensitive data and dangerous operations (such as account deletion) that allow these things to be limited to a small number of admin staff.

Ability to mark specific accounts (e.g. celebrities, senior management) as inaccessible by admins, overridable only by special privileges that require sign off.

Regular audit of log files and raw database content to verify that no PII is leaking into uncontrolled areas.

There are probably other measures I can't remember but the basic idea is to throw sand in the face of a potential internal attacker while still allowing legitimate activities to be carried out. In a larger organization you might be able to use ML to detect anomalous admin activity.

Update : I remembered another thing -- don't collect sensitive data that you don't need. I've had several discussions over the years along the lines of "we found that we can get xxx (PII field of some sort , e.g. the user's cell phone number even if they didn't know they gave it to us), where should we store that?". Answer : don't.

Re: Former employees say Lyft staffers spied on passengers

#245

Post by organsnyder »

milofeynman wrote: post #47
mpolichette wrote: post #5

When I did an internship at a national lab, a lot of the hard rules about security relied on the fact that you had gone though their hiring process and would follow the rules. There were different access levels, for sure, but only like 2 or 3. You might have "had access" but you shouldn't be anywhere you didn't have a good reason for being. Lyft should be checking on this, running audits and whatnot, but they also sh…

In hospitals in the U.S. the way it works in some is nurses can view a lot of the patients charts (including VIP). And then someone is supposed to audit who viewed those VIP patients (celebrity or what not) but every hospital is different and it's a mess.

My wife is a music therapist, and her previous employer had a contract with a local hospital system. During one visit, she (or a colleague—I don't recall) was working with someone with the same last name, but no relation; this set off an automatic alert in the EMR, leading to a follow-up inquiry. VIP records have similar alerts configured to verify that the accessors have legitimate business reason.

Re: Former employees say Lyft staffers spied on passengers

#246

Post by organsnyder »

pookeh wrote: post #54
milofeynman wrote: post #47

Earlier quoted context omitted.

In hospitals in the U.S. the way it works in some is nurses can view a lot of the patients charts (including VIP). And then someone is supposed to audit who viewed those VIP patients (celebrity or what not) but every hospital is different and it's a mess.

Here in Canada if you have access to the central medical records you can look up anyone but (a) if you are not a doc and are not assigned to the case or (b) you are looking up yourself or a family member, you immediately get a call and get fired on the spot. (Source: Wife works at the hospital and has seen some people get fired shortly after unauhorized access.)

My hospital system has similar policies in place. I like to tell people, "If I look at your record, I'll probably get fired. If I look at my own record, I will absolutely get fired."

Of course, both of them are prohibited (without a valid business reason), but the latter is easier to detect in an automated fashion.

Re: Former employees say Lyft staffers spied on passengers

#247

Post by organsnyder »

mattmanser wrote: post #201
djsumdog wrote: post #144

Earlier quoted context omitted.

I agree with you. I worked at a BlueCross and when debugging front-end apps, and I'd often dig through the database to check to see what is in each table and what could be throwing an error. I'd often bright up a debugging tool called TeaLeaf that let me watch the entire web session (this was back in 2009; so the full DOM tracking we complain about now has been available for quite some time). Technically, we couldn't…

That you casually talk about breaking the law with incredibly sensitive data is shocking. And worse still, you built in a system that circumvented the official policy, simply because you weren't capable of doing your job. You basically built in an illegal back door. If you were my employee I would fire you on the spot for gross negligence when I found that. Or maybe if you were a junior you'd be given a massive dress…

> This is just bad engineering, if you can't recreate bugs without having to constantly resort to live data, get someone in who can. It's not particularly hard, it's a learn-able skill of our profession.

In any large enterprise, you're probably working with data spanning multiple in-house databases, vendor applications, message buses, responsible teams... I can develop a greenfield system that makes identifying bugs trivial without accessing production data. Unfortunately, the real world is rarely that simple—especially for any enterprise that has grown organically over time (i.e., all of them).

My employer recently retired some systems that were older than me (I'm in my mid 30's), and the organization is simply too large to be understood by any one individual.

Re: Former employees say Lyft staffers spied on passengers

#248

Post by zaroth »

cornholio wrote: post #197
ryan_j_naughton wrote: post #122

Having seen this at too many companies, we at fair.com decided to adopt stronger policies to prevent this, viz: - all inbound API requests first go to our API proxy in the secure layer. - the API proxy encrypts all PII using the encryption service in the secure layer - then API proxy sends the request on to the appropriate service, having swapped all PII for tokens. - all services in the general layer are not able to…

The key problem with both Lyft and Uber is that employees who had the right to view the data abused the privileges. While encryption can help to enforce the privileges against technically capable employees, the main problem is the privilege system itself, or lack thereof. 95% of the abuse would be eliminated with no encryption, just proper design of the internal interfaces and queries talking to the plain text databa…

Agree completely. Encryption or tokenization is great but a much easier first step which covers the first order problem of employees abusing access privileges is simply a secure audit trail and someone to actually look at the logs and discipline or terminate employees who misuse their access.

There are probably innumerable reasons to have access to a particular ride’s exact route, or a particular users ride history, or feedback history, etc. Just like the police have many perfectly valid reasons to run a plate.

Audit logging and request throttling are the low hanging fruit. If your system can tie each request to the service ticket which prompted it, even better.

Re: Former employees say Lyft staffers spied on passengers

#249

Post by zaroth »

organsnyder wrote: post #246
pookeh wrote: post #54

Earlier quoted context omitted.

Here in Canada if you have access to the central medical records you can look up anyone but (a) if you are not a doc and are not assigned to the case or (b) you are looking up yourself or a family member, you immediately get a call and get fired on the spot. (Source: Wife works at the hospital and has seen some people get fired shortly after unauhorized access.)

My hospital system has similar policies in place. I like to tell people, "If I look at your record, I'll probably get fired. If I look at my own record, I will absolutely get fired." Of course, both of them are prohibited (without a valid business reason), but the latter is easier to detect in an automated fashion.

Everyone should be able to look at their own record...

Re: Former employees say Lyft staffers spied on passengers

#250

Post by bowbles »

marshallbrekka wrote: post #238
hoorayimhelping wrote: post #228

Earlier quoted context omitted.

> I don't get why more companies don't follow our approach. How much latency does this process add to the rendering of a web page? How many people did it take to design, implement and now maintain?

For an API call that requires decrypting data it can add about 5-10ms, most of that comes from reading the values from the token database. Encrypting will incur a similar performance penalty (again from the database). The core pieces (API Proxy, Encryption Service, secure/general infrastructure divide) we're done by me in the first 2-4 months. We now have a team of three (myself included) who maintain those systems (…

Productive this and sell it at a low cost if it’s that simple. I’d love to use something like this but building it from scratch in house is cost prohibitive.
Post reply on HN