Live data from Hacker News

Toyota suffered a data breach by accidentally exposing a secret key on GitHub

blog.gitguardian.com

231–240 of 272 posts

Re: Toyota suffered a data breach by accidentally exposing a secret key on GitHub

#231
post #222
post #213

Earlier quoted context omitted.

There probably is but ISO certs are just paperwork filed. Reality is different.

Sure. But then again you need a framework to approximate reality. ISO isn’t perfect by any means. But it’s a start. What do you use?

ISO 9001 + 27001 :(

Re: Toyota suffered a data breach by accidentally exposing a secret key on GitHub

#232

Earlier quoted context omitted.

Rant time: this isn’t directed at you. I am just replying to your comment because you said something that triggered me. Also the “you” below is the generic you - not you personally. Disclaimer: I work at AWS in Professional Services, all rants are my own. Now with that out of the way, I hate the fact that there are way too many code samples floating around on the internet that have you explicitly put your access key…

> I hate the fact that there are way too many code samples floating around on the internet that have you explicitly put your access key and secret key in the initialization code for the AWS SDK. See, I thought that was a big strength of a lot of the AWS documentation over Google Cloud. An AWS example for, say, S3 would show you where to insert the secrets, and it would work. The Google Cloud Storage examples, though?…

> See, I thought that was a big strength of a lot of the AWS documentation over Google Cloud.

Just to clarify, I’ve never seen a code sample published by AWS that has you explicitly specifying your credentials. (Now I await 15 replies showing me samples hosted on Amazon)

Re: Toyota suffered a data breach by accidentally exposing a secret key on GitHub

#233

Earlier quoted context omitted.

A super simple way is to have a script in your home directory - far away from your repos - that set environment variables that you read in your configuration.

That makes sense. I could do something like that. [UPDATE] I ended up doing something even simpler. I have issues with running scripts during the build process, unless really necessary (I have done it, and will, again). Since this is Xcode, I simply needed to store the file in a directory (still with the global ignored name) far out of my dev tree, and dragged the file into the IDE.

That’s basically the idea - get your credentials out of your dev tree. I’m not dogmatic about how it’s done.

Re: Toyota suffered a data breach by accidentally exposing a secret key on GitHub

#234

Earlier quoted context omitted.

I actually had something similar happen to me last month. I accidentally published a discord API key to GitHub and within minutes I got a nice message from “Safety Jim” to my personal discord account letting me know they’ve found my key on a public repo and have gone ahead and revoked it. I felt like a bit of a dope but it was neat to have it happen to me. Lesson learned for sure.

GitHub PM here. Glad that was a good experience! We work with ~50 partners (details in the link below) to notify them when tokens for their service are exposed in public repos, so that they can notify you. https://docs.github.com/en/code-security/secret-scanning/sec...

What are the thoughts around capabilities like this for private/enterprise customers? Is the code available in an action that could be connected to private runners perhaps?

Re: Toyota suffered a data breach by accidentally exposing a secret key on GitHub

#235

Earlier quoted context omitted.

> I hate the fact that there are way too many code samples floating around on the internet that have you explicitly put your access key and secret key in the initialization code for the AWS SDK. See, I thought that was a big strength of a lot of the AWS documentation over Google Cloud. An AWS example for, say, S3 would show you where to insert the secrets, and it would work. The Google Cloud Storage examples, though?…

> See, I thought that was a big strength of a lot of the AWS documentation over Google Cloud. Just to clarify, I’ve never seen a code sample published by AWS that has you explicitly specifying your credentials. (Now I await 15 replies showing me samples hosted on Amazon)

For Java they used to demonstrate putting a .properties file in among your source code [1] although admittedly not literally hardcoding a string. The PHP examples suggested putting your code into a config php include [2] (although they did also suggest putting them in your home directory).

But I can't understate how important it was that the AWS getting started guides said "Go to this URL, copy these values into this file" while Google's examples and getting started guides... didn't.

[1] https://web.archive.org/web/20120521060506/http://aws.amazon... [2] https://github.com/amazonwebservices/aws-sdk-for-php/blob/ma...

Re: Toyota suffered a data breach by accidentally exposing a secret key on GitHub

#236

Earlier quoted context omitted.

> See, I thought that was a big strength of a lot of the AWS documentation over Google Cloud. Just to clarify, I’ve never seen a code sample published by AWS that has you explicitly specifying your credentials. (Now I await 15 replies showing me samples hosted on Amazon)

For Java they used to demonstrate putting a .properties file in among your source code [1] although admittedly not literally hardcoding a string. The PHP examples suggested putting your code into a config php include [2] (although they did also suggest putting them in your home directory). But I can't understate how important it was that the AWS getting started guides said "Go to this URL, copy these values into this…

Wow, that’s some old code :).

But here is the newest documentation for PHP

https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/s...

Re: Toyota suffered a data breach by accidentally exposing a secret key on GitHub

#237
post #148

Earlier quoted context omitted.

Yes, seriously. They need to service the vehicle, OK, but communicate to and through the vehicle.

These are the services, according to the article: “T-Connect enables features like remote starting, in-car Wi-Fi, digital key access, full control over dashboard-provided metrics, as well as a direct line to the My Toyota service app. The servers that control these options contain unique customer identification numbers and customer emails.”

I don't see why any of that should require the email address. They can communicate with the customer through the app or through the car UI.

In general, apps and sites these days hoover up more info than they need simply because they can, not because it adds to the customer experience (and often doesn't help the company either). There is no incentive to be in any way judicious about what to collect and the frequent breaches show that even the companies don't value PII as something worth protecting because it's not core to their business.

Re: Toyota suffered a data breach by accidentally exposing a secret key on GitHub

#238

Earlier quoted context omitted.

My company has monitoring for this, but it still seems to be a law of nature that, 1. someone adds new service/server/infra in a submarine manner 2. it goes to prod 3. the cert expires and outage begins 4. my team is asked what to do, because "we're the cert experts" 5. we add it to the monitoring So it only happens once … per service. Which isn't great. But how do you get people to slow down and do simple shit, like…

I think the real answer is to only issue limited-duration certs and only via automated means (ACME or similar), thus requiring automation be in place from day 1. This still doesn't protect against the vector where somebody else in the company has managed to prove themselves to be responsible parties to another CA/issuer.

Oh I agree! Everything would be ACME if I could. A ridiculous amount of stuff still doesn't support it, though.

And, like I said, usually it's someone who doesn't grok certs doing it without asking for help in the first place, so they're not going to get why ACME. (Because I am tired of doing cert renewals. I've had enough for a lifetime…)

Re: Toyota suffered a data breach by accidentally exposing a secret key on GitHub

#239

Earlier quoted context omitted.

My company has monitoring for this, but it still seems to be a law of nature that, 1. someone adds new service/server/infra in a submarine manner 2. it goes to prod 3. the cert expires and outage begins 4. my team is asked what to do, because "we're the cert experts" 5. we add it to the monitoring So it only happens once … per service. Which isn't great. But how do you get people to slow down and do simple shit, like…

“If you keep smelling shit, look at your own shoe”. Your processes are failing your development teams, and you need to fix them, rather than blaming your teams, which achieves nothing.

I agree, but that level of authority is in a different level of the foodchain. I'm on duck tales, Larry.

Re: Toyota suffered a data breach by accidentally exposing a secret key on GitHub

#240

Earlier quoted context omitted.

My company has monitoring for this, but it still seems to be a law of nature that, 1. someone adds new service/server/infra in a submarine manner 2. it goes to prod 3. the cert expires and outage begins 4. my team is asked what to do, because "we're the cert experts" 5. we add it to the monitoring So it only happens once … per service. Which isn't great. But how do you get people to slow down and do simple shit, like…

Pit of success. Make it so that the Right Thing™ is super easy, whereas the Wrong Thing™ is frustrating and keeps pushing people towards the Right Thing. Humans are lazy, use that to your advantage. For example it's one line for me to configure a new machine infrastructure built to have a certificate for myservice.myorg.example, and there's a Wiki page reminding me what the line is, or I can look at lots of services…

People are really good at avoiding the pit of success! We run most of our infra on k8s, and if you want a cert, with ACME & auto-renew all managed automatically for you, you just create a Certificate object.

But then we get some vendored product that manages to be completely unable to run in k8s, devs avoid the automation for $reasons, etc.

> Does your company have a Wiki page saying how to do it wrong?

Sometimes we do! I've found a few of these after really pressing the point of "why are you doing it this way?" hard enough. But you have to a.) get an answer to that and b.) the answer has to reveal they followed some shadow docs.

> If you have firewall rules or a proxy, block random unauthorised stuff.

Your rouge service implementer just creates their own VPC; they are in control of the firewall.

Should my security team either set appropriate privileges or delegate that to my team? Perhaps. I have to get them to adopt like RBAC and ABAC first; they fervently believe our industry's regulations forbid "job function begets access" (i.e., RBAC) type policies. (They desire that, even if a job function begets access, that if you're not needing to exercise that access, it should be revoked until such a time that it is required to be exercised. But this means that you end up with "all security reqs. must flow through security team" style thing, and there are then a lot of them (because they are so ephemeral) and so any process must inherently be ignorant of whether the request is right. So your rouge implementor's request for "I need to implement $high level service" is basically carte blanche.

The thing about shadow-docs and shadow-services is that they're hard to find out about in a timely manner. A lot of these comments are fighting the very core of human nature.

(We used to be better about this as a company, back when we were very engineer heavy, and filled with good engineers — most better than me. The quality bar definitely fell at some point, and we've hired a lot of not engineers doing things that really would be better served by an engineer. Y'all are working at rainbow companies, and I don't know how to keep a company in that state or move it to that state as a bottom-of-rungs eng.)

Post reply on HN