Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heroku: We’ve Heard Your Feedback

blog.heroku.com

Post reply on HN
Print view View original

91–100 of 151 posts

Re: Heroku: We’ve Heard Your Feedback

#91

Post by oxff »

chillfox wrote: post #88
throwawa3238273 wrote: post #46

Earlier quoted context omitted.

> 2. Customers sign up at "heroku.com", the platform is called "Heroku", the CLI is "heroku", everything's heroku, so don't send emails from a parent company (Salesforce), send them from "Heroku". They're working on something called "Project Periwinkle" that is intended to remove all Heroku branding and make everything Salesforce branded. Periwinkle being a colour between blue (salesforce) and purple (heroku). No mor…

Well, that’s sad. I guess it’s Fly or Digital Ocean for all future projects then.

There is some comparison of DO's App platform and Google's Cloud offerings here if you are interested (not my blog, just found it helpful: https://blog.verygoodsoftwarenotvirus.ru/posts/greener-cloud...)

Re: Heroku: We’ve Heard Your Feedback

#92

Post by bin_bash »

tomatowurst wrote: post #79
steffanA wrote: post #73

Sorry, but this is a joke of a response. When they started sending out password reset emails, they should have explained why. Not only when people started complaining, and the media picked up on the lack of transparency.

can somebody fill me in what happened? is this related to the oAuth vulnerability through github a while back?

They got hacked about a month ago and have been extremely cagey about what exactly was hacked ever since.

Turns out the master database with encrypted username/passwords got leaked and encrypted environment variables were also leaked but it was like pulling teeth to get them to answer whether or not these happened or even admit that it might have been possible. Presumably more than this was also leaked but so far they haven't said anything on that. Env vars were the biggest concern on everyone's mind.

They gave the absolute least amount of information over the longest period they could muster.

The problem wasn't the hack really, it was the lack of transparency in the response.

See: https://twitter.com/jacobian/status/1522782890957819906

Re: Heroku: We’ve Heard Your Feedback

#93

Post by your_username »

chillfox wrote: post #88
throwawa3238273 wrote: post #46

Earlier quoted context omitted.

> 2. Customers sign up at "heroku.com", the platform is called "Heroku", the CLI is "heroku", everything's heroku, so don't send emails from a parent company (Salesforce), send them from "Heroku". They're working on something called "Project Periwinkle" that is intended to remove all Heroku branding and make everything Salesforce branded. Periwinkle being a colour between blue (salesforce) and purple (heroku). No mor…

Well, that’s sad. I guess it’s Fly or Digital Ocean for all future projects then.

No post body was provided.

Re: Heroku: We’ve Heard Your Feedback

#94

Post by remram »

ezekg wrote: post #71
ezekg wrote: post #67

> Additionally, we have no evidence that the attacker has accessed any customer accounts or decrypted customers’ environment variables. Now the attackers had access to encrypted environment variables?

More context, from the latest email notification: > We also wanted to address a question regarding impact to environment variables. While we confirmed that the threat actor had access to encrypted Heroku customer secrets stored in config var, the secrets are encrypted at rest and the threat actor did not access the encryption key necessary to decrypt config var secrets.

Where do I find those notifications? I only got a single email, stating I should "reset my user password as mentioned in our previous notification". Nothing prior, nothing since, and their password reset form shows nothing but "Internal Server Error". I've been getting some details from HN but I think I'm missing a lot of the picture since they obviously don't send everyone every email.

Re: Heroku: We’ve Heard Your Feedback

#95

Post by BeefWellington »

charcircuit wrote: post #6

>The Heroku team and their colleagues have worked around the clock, including nights and weekends Can someone more familiar with an event like this tell me what they are working so hard on? I imagine securing the vulnerable service and resetting various credentials doesn't take that much work.

Proper incident response involves a lot more than just "securing the vulnerable service and resetting various credentials" and it will in this case probably involve a lot of different teams.

For starters, you have to document everything. At the very minimum the legal team for the company should insist on this, if no other measures, just as a CYA move.

If an attacker had broad access, it's entirely possible an all hands on deck approach is required to help identify (and document) what systems were compromised. Yes, you definitely want a team working on patching the hole ASAP. You also need a team hunting for any possible persistence. Another team probably involving standing up brand new "safe" systems and failing over client systems over to those running the patched software. While that happens yet more people may start doing audits of what was compromised on the original systems.

I've seen incidents where 50-80 people were pulled in to work on an incident at a company of about 150 employees. Depending on how well-funded your SoC is they can cut that number down substantially.

Re: Heroku: We’ve Heard Your Feedback

#96

Post by tomatowurst »

bin_bash wrote: post #92
tomatowurst wrote: post #79

Earlier quoted context omitted.

can somebody fill me in what happened? is this related to the oAuth vulnerability through github a while back?

They got hacked about a month ago and have been extremely cagey about what exactly was hacked ever since. Turns out the master database with encrypted username/passwords got leaked and encrypted environment variables were also leaked but it was like pulling teeth to get them to answer whether or not these happened or even admit that it might have been possible. Presumably more than this was also leaked but so far the…

> Turns out the master database with encrypted username/passwords got leaked and encrypted environment variables were also leaked

WTF!!!!

That alone is disastrous enough, they should be reprimanded for this. Are there I'm sure, class action lawsuits happening?

How much of an impact will this have on Salesforce? I mean imagine the data from that alone would be immensely valuable.

Re: Heroku: We’ve Heard Your Feedback

#97

Post by gabereiser »

chillfox wrote: post #88
throwawa3238273 wrote: post #46

Earlier quoted context omitted.

> 2. Customers sign up at "heroku.com", the platform is called "Heroku", the CLI is "heroku", everything's heroku, so don't send emails from a parent company (Salesforce), send them from "Heroku". They're working on something called "Project Periwinkle" that is intended to remove all Heroku branding and make everything Salesforce branded. Periwinkle being a colour between blue (salesforce) and purple (heroku). No mor…

Well, that’s sad. I guess it’s Fly or Digital Ocean for all future projects then.

Don’t forget render.com

Re: Heroku: We’ve Heard Your Feedback

#98

Post by adrr »

jolux wrote: post #12
charcircuit wrote: post #6

>The Heroku team and their colleagues have worked around the clock, including nights and weekends Can someone more familiar with an event like this tell me what they are working so hard on? I imagine securing the vulnerable service and resetting various credentials doesn't take that much work.

You have to make sure the attackers aren’t still in your network, you have to get them out if they are, you have to fully scope out what they messed with, and restore anything that’s plausibly connected. Even in the best case with all the right monitoring systems in place there is a lot of manual work involved from owners of the various different effected services and just managing the overall response adequately. I…

Wouldn’t you hire an incident response team who are experts in figuring this stuff out?

Re: Heroku: We’ve Heard Your Feedback

#99

Post by mh- »

toyg wrote: post #20
darknavi wrote: post #4

> I started as Heroku GM a few weeks ago with intense enthusiasm to be a part of such a storied team. Wow talk about terrible timing.

Maybe he's the guy who posted a couple of days ago about pulling a disk out of a RAID array on his first day in a leadership position...

Hah. Got a link to that?
Post reply on HN