Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Heroku Security Notification

status.heroku.com

Post reply on HN
Print view View original

121–130 of 162 posts

Re: Heroku Security Notification

#121

Post by jeremymcanally »

jeremyjh wrote: post #115
jeremymcanally wrote: post #114

Earlier quoted context omitted.

The bigger question to me is how did they leverage a GitHub OAuth token to gain access to an internal database unless they're storing that config in their codebase. If that's the case...yikes.

They didn’t say that happened. I’m reading it as their DB was compromised and it’s contents included GH auth tokens.

How do you read this as the database had tokens in it?

> Separately, our investigation also revealed that the same compromised token was leveraged to gain access to a database...

EDIT: Ah yep you're right. Two tokens in play there: one Heroku API token, one GitHub token. Phew.

Re: Heroku Security Notification

#122

Post by bpicolo »

samwillis wrote: post #68

Well, it only seems to be getting worse on this one. I’m keen to get off Heroku, but waiting for one of the newer alternatives (Render/Fly+others) to implement WAL point in time restore for Postgres. It’s the only thing keeping me on Heroku now, but is indispensable. Anyone here from them have any update on when we could see that feature made available?

I recently spun up Google Cloud Run as an alternative. GCP has PITR for postgres.

The only weird hitch with Google Cloud Run (and their other serverless products) is you either need to either use public IPs for e.g. memory store cache or other things in your network (minus your database), or set up a VPC Access Connector [0]. Admittedly, that was easy once I realized I needed it, but was very annoying to figure out (because django's default redis socket timeout is never...).

You also don't get SSH access, as it's fully managed. K8s is still the easiest version where you get that, unfortunately.

[0]: https://cloud.google.com/vpc/docs/configure-serverless-vpc-a...

Re: Heroku Security Notification

#123

Post by martinald »

ryanSrich wrote: post #76
samwillis wrote: post #68

Well, it only seems to be getting worse on this one. I’m keen to get off Heroku, but waiting for one of the newer alternatives (Render/Fly+others) to implement WAL point in time restore for Postgres. It’s the only thing keeping me on Heroku now, but is indispensable. Anyone here from them have any update on when we could see that feature made available?

I’m also interested in this. I have 4 decent sized applications I want to migrate, but I don’t have the resources to self host on GCP or AWS. K8s is nice in theory, but I know from experience it requires too much hand holding (let alone other managed services like a database and cache). From what I’ve seen in the market, fly, render, and railway are the only real Heroku competitors. All seem to be missing a few criti…

Azure web apps is pretty similar to heroku and I find it very easy to use. It has had a Linux version (instead of Windows) for quite a while now and it's pretty solid. Haven't had many real issues with it in many years of use.

They also have managed postgres db with very powerful tools.

Re: Heroku Security Notification

#124

Post by jeromegv »

srinathkrishna wrote: post #37
craigkerstiens wrote: post #28

"a Heroku database" was what was known as core-db internally for the longest time. I'm not sure if still the case or not today. But at one point was the source of everything for Heroku. Over time things were moved out, so this isn't an everything that exists has been leaked, but it is not a guarantee that attacker didn't move from one area to another. As someone with some apps on Heroku, having worked there, but no k…

I feel for the team working on this at this time. I hope this doesn't end up accelerating the culling off of Heroku by Salesforce. One of the smartest and nicest bunch of folks I've worked with.

Heroku is heavily connected with Salesforce now with Heroku Connect, I doubt this is part of the plan.

Re: Heroku Security Notification

#125

Post by sfdcsecthrow »

lovelearning wrote: post #49

> On April 7, 2022, a threat actor obtained access...GitHub identified the activity on April 12, 2022, and notified Salesforce on April 13, 2022, at which time we began our investigation. Can some experienced security professionals weigh in on the cultural and organizational factors that allow this kind of major breach to go unnoticed for a week, that too in a reputed company like Heroku? I'm not asking this rhetoric…

Throwaway for obvious reasons. But in my experience Salesforce security org is plagued with incompetent leaders who chase arbitrary metrics that does not improve security at all. At one point in time, security team at Salesforce was stellar and did some awesome work. Dont get me wrong, there are many many smart security engineers still around but their population has been dwindling. This all started when a bunch of new leaders were hired. Instead of promoting tenured smart people, the security leaders decided to bring in their own gang of coworkers from previous employers. At the same time they slowly started pushing out tenured leaders.

These new leaders are typical VPs who have long lost any technical chops and its a huge task to explain any complex technical topic to them. On top of that they don't bother understanding the fundamental business model and just want to push their agenda on to everyone. So they have added "security processes" which requires checking boxes. The more boxes you check the more metric it generates the better the leader looks. These security leaders are so disconnected from the ground reality that they don't even realize that all they are doing is adding hurdles in the path of engineers without improving any security.

Re: Heroku Security Notification

#127

Post by sfdcsecthrow »

adrr wrote: post #106
lovelearning wrote: post #49

> On April 7, 2022, a threat actor obtained access...GitHub identified the activity on April 12, 2022, and notified Salesforce on April 13, 2022, at which time we began our investigation. Can some experienced security professionals weigh in on the cultural and organizational factors that allow this kind of major breach to go unnoticed for a week, that too in a reputed company like Heroku? I'm not asking this rhetoric…

Because Heroku/Salesforce doesn't have real security. Requiring special characters in passwords and sending out emails that have http and not https links to a password reset page. Their security is a joke.

This is not true. Heroku actually had a pretty decent security team. They have lost most of that talent though.

Re: Heroku Security Notification

#128

Post by sfdcsecthrow »

nameless912 wrote: post #111
adrr wrote: post #106

Earlier quoted context omitted.

Because Heroku/Salesforce doesn't have real security. Requiring special characters in passwords and sending out emails that have http and not https links to a password reset page. Their security is a joke.

As a former Herokai, let me color this a bit: Heroku _used_ to have their own security team which was quite good and had some scary talented people on it. However, over the last 3 years or so Salesforce has been forcing Heroku to adopt Salesforce's operations practices, and this has not only wrecked productivity but completely destroyed morale and caused many, many of those talented people to quit. I for one decided…

The integration of Heroku stuff into SFDC would have probably been easier if the Heroku folks got off their high horse.

Re: Heroku Security Notification

#129

Post by nameless912 »

sfdcsecthrow wrote: post #128
nameless912 wrote: post #111

Earlier quoted context omitted.

As a former Herokai, let me color this a bit: Heroku _used_ to have their own security team which was quite good and had some scary talented people on it. However, over the last 3 years or so Salesforce has been forcing Heroku to adopt Salesforce's operations practices, and this has not only wrecked productivity but completely destroyed morale and caused many, many of those talented people to quit. I for one decided…

The integration of Heroku stuff into SFDC would have probably been easier if the Heroku folks got off their high horse.

I don't want to start a flamewar, but it wasn't an integration; it was a hostile takeover. Heroku was doing just fine without SFDC's interference, and when Salesforce not only refused to believe that employees might have negative opinions about integrating but actively prevented us from speaking up about it, people got rationally angry and left. I remember talking with an architect on the Heroku side about how he felt about all of it, and he told me that it wasn't presented to him as a choice, and senior leadership was convinced that Herokai would see this as a good thing despite his (and others') warnings.

Re: Heroku Security Notification

#130

Post by onphonenow »

DethNinja wrote: post #102
onphonenow wrote: post #4

Ugh - I got the password reset email w silly password complexity - never a good sign.

That email made it clear that Heroku lacks fundamental knowledge about security. I’m sure they lost some enterprise customers, I know I don’t open accounts on websites with silly password complexity requirements.

Google allowed 6 character passwords for a while, and didn't expire them when they increased minimum to 8 for google workspace accounts. This has been fantastic, as users can remember their password forever even if its higher complexity (google does a password strength eval). No rotations either.

I'm pretty confident google will pick-up someone trying to brute force a 6 character password. That google will notice connections from new / different IPs or browsers. That's because google asks for my 2FA in various situations but doesn't annoy me by asking for 2FA all the time.

I use one govt system that has something like a 14 character password requirement. For even more security if you don't log in for 90 days your account goes inactive and the password EXPIRES! Very secure you say? Well, to regain access you have to provide the answer to a security question - favorite pet! That's a 5 letter word that doesn't change (and is probably pretty guessable).

Here is another example:

"(b) Information systems must be designed to require passwords to be changed not less frequently than every sixty (60) days." - SBA IT Security Policy - 90 47 4

Post reply on HN