Posting this at 5pm on a Friday sounds about right, yeah?
[deleted]
Heroku: We’ve Heard Your Feedback
11–20 of 151 posts
Re: Heroku: We’ve Heard Your Feedback
#12>The Heroku team and their colleagues have worked around the clock, including nights and weekends Can someone more familiar with an event like this tell me what they are working so hard on? I imagine securing the vulnerable service and resetting various credentials doesn't take that much work.
You have to make sure the attackers aren’t still in your network, you have to get them out if they are, you have to fully scope out what they messed with, and restore anything that’s plausibly connected. Even in the best case with all the right monitoring systems in place there is a lot of manual work involved from owners of the various different effected services and just managing the overall response adequately. I would say the amount of effort involved is comparable to managing a novel, ongoing SEV1.
Re: Heroku: We’ve Heard Your Feedback
#13Posting this at 5pm on a Friday sounds about right, yeah?
That isn’t a very charitable take. It’s been 24x7 for them, the acknowledgement is most welcome, in my opinion. I’m not sure waiting until Monday 9AM Pacific just to avoid these type of comments would be the right choice. The internet is 24x7 after all.
There doesn't seem to be anything different now than 6 hours ago. So posting it after 8pm EDT/5pm PDT Friday seems very intentional.
Re: Heroku: We’ve Heard Your Feedback
#14Posting this at 5pm on a Friday sounds about right, yeah?
That isn’t a very charitable take. It’s been 24x7 for them, the acknowledgement is most welcome, in my opinion. I’m not sure waiting until Monday 9AM Pacific just to avoid these type of comments would be the right choice. The internet is 24x7 after all.
[deleted]
Re: Heroku: We’ve Heard Your Feedback
#15As a former herokai, communication was always the #1 point of discussion internally. This idea that they "can" do better is a half truth. They DID do better. I leave it as an exercise to the reader to determine what the limiting resource was here.
By the way, how many senior devs and cofounders are left at Heroku Bob? Why doesn't it show up at dreamforce anymore?
Re: Heroku: We’ve Heard Your Feedback
#16>The Heroku team and their colleagues have worked around the clock, including nights and weekends Can someone more familiar with an event like this tell me what they are working so hard on? I imagine securing the vulnerable service and resetting various credentials doesn't take that much work.
I imagine they are examining their whole attack surface area. Probably bringing in consultants to discover what could possibly be attacked.
Re: Heroku: We’ve Heard Your Feedback
#17Curious why they would need to keep the GitHub integration down for a while longer. Are they afraid someone could still grab credentials?
Re: Heroku: We’ve Heard Your Feedback
#18> We’ve heard your feedback on our communications during this incident. You want more transparency, more in-depth information, and fewer “we are working on it” posts.
Well, those, and:
1. Speed. It took days for heroku customers to be told about this.
2. Customers sign up at "heroku.com", the platform is called "Heroku", the CLI is "heroku", everything's heroku, so don't send emails from a parent company (Salesforce), send them from "Heroku".
3. Unambiguous info on what customers need to do. I had to guess based on HN comments whether config vars were accessed. Config vars are 100x more sensitive than code. Comms should be unambiguous and complete, and if incomplete for any reason, explain that (e.g. we don't know yet).
4. I still don't know whether having 1 Github Deploy on my Heroku account allowed unauthorized access to all heroku applications on my heroku account (i.e. those using other deploy methods, like `git push heroku main`). Were all my apps' repositories able to be accessed, or just the one(s) deployed via Github Deploys?
5. I still don't know whether unauthorized access was gained to all other GitHub respositories on my GitHub account, i.e. the repos that aren't heroku apps.
These said, I still really appreciate that security incidents happen and aren't easy to deal with, and there's no obligation for anyone at a profitable company to actually care about semi-captive customers, so thanks to Heroku for the efforts; it's genuinely appreciated.
Re: Heroku: We’ve Heard Your Feedback
#19“I have a lifelong enthusiasm for developers and the experience they have building software together”
And then drops a link to contact them, via LinkedIn…
LinkedIn is the polar opposite of GitHub. It’s the worst example of social media, from its news feed, to spam invites. And it’s broken every rule in the “be a good netizen” play book, from constant spam, to slurping your email contacts and surveillance to the extreme.
I struggle to imagine a developer saying “I’d like to contact xxx, and I’d love to do it via LinkedIn”
Why not drop your email? Or a GitHub profile with a public email, and readme containing other contact methods, would have been more dev centric.
Re: Heroku: We’ve Heard Your Feedback
#20> I started as Heroku GM a few weeks ago with intense enthusiasm to be a part of such a storied team. Wow talk about terrible timing.
Maybe he's the guy who posted a couple of days ago about pulling a disk out of a RAID array on his first day in a leadership position...