Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Issues with upstream DNS provider

status.heroku.com

Post reply on HN
Print view View original

61–70 of 159 posts

Re: Issues with upstream DNS provider

#61

Post by zoomzoom »

rajamaka wrote: post #36
bobsingor wrote: post #30

Earlier quoted context omitted.

Yes, i'm also done with Heroku. What alternative do you recommend?

Moving to ECS on AWS, only reason is that I'm a DevOps engineer with familiarity in that space. Honestly wouldn't know where to send developers without the time to figure the nuts and bolts of convenience that Heroku provides.

Please check out withcoherence.com (I'm a cofounder). We manage the nuts and bolts of apps using ECS/fargate on AWS and Cloud Run/GKE on GCP. We also integrate managed cloud IDEs and managed CI/CD pipelines using the same simple configuration, and put a dashboard on top of it!

Re: Issues with upstream DNS provider

#63

Post by iancarroll »

cyral wrote: post #56
rajamaka wrote: post #53

Earlier quoted context omitted.

No historical DNS records on that site for us unfortunately, any other possible sites to check?

Try this command: nslookup DOMAIN 212.230.255.1 That is a random public DNS server I found on a DNS checker site that works (as it seems to be down from all the common ones like 8.8.8.8, 1.1.1.1, etc). DOMAIN can be your actual domain or the long heroku alias.

Thanks, for some reason this worked!

Re: Issues with upstream DNS provider

#64

Post by ab-dm »

rsweeney21 wrote: post #35

We just deployed a workaround that brought our site back up. 1. Find your DNS Target in heroku. It should end with .herokudns.com 2. Lookup the historical DNS record to get the IP addresses. You can find historical DNS records here: https://securitytrails.com/dns-trails 3. Replace your CNAME record in your DNS provider with A records that point to the IP addresses you just found. Your site should come back up shortly…

Oh my god, thank you so much. You're a lifesaver.

For those on Cloudflare, this will still work. I just deleted the existing CNAME, added the 4 IP addresses as seperate A records and it came back up instantly.

Re: Issues with upstream DNS provider

#66

Post by ab-dm »

exitnode_ttl0 wrote: post #58
rsweeney21 wrote: post #35

We just deployed a workaround that brought our site back up. 1. Find your DNS Target in heroku. It should end with .herokudns.com 2. Lookup the historical DNS record to get the IP addresses. You can find historical DNS records here: https://securitytrails.com/dns-trails 3. Replace your CNAME record in your DNS provider with A records that point to the IP addresses you just found. Your site should come back up shortly…

Anybody know what the risks are for doing this? We have not one but 13 heroku domains running where we might do this. What are the chances that their current work on the DNS issues might replace the underlying IP-addresses and hence we are making it harder on ourselves?

I thought about this, and it can't really get any worse than 'total outage', so I thought it was worth taking a shot. It can't go any more down.

It worked for me, so once Heroku has sorted their s*t out and their DNS system is back up it's just a matter of deleting the A records and replacing it with the CName.

Worst case scenario is the underlying Amazon IP Addresses change and it goes back down.

Re: Issues with upstream DNS provider

#67

Post by ladybro »

rsweeney21 wrote: post #35

We just deployed a workaround that brought our site back up. 1. Find your DNS Target in heroku. It should end with .herokudns.com 2. Lookup the historical DNS record to get the IP addresses. You can find historical DNS records here: https://securitytrails.com/dns-trails 3. Replace your CNAME record in your DNS provider with A records that point to the IP addresses you just found. Your site should come back up shortly…

Worked for me as well! Thank you!

Re: Issues with upstream DNS provider

#68

Post by ladybro »

theunixbeard wrote: post #45
rsweeney21 wrote: post #35

We just deployed a workaround that brought our site back up. 1. Find your DNS Target in heroku. It should end with .herokudns.com 2. Lookup the historical DNS record to get the IP addresses. You can find historical DNS records here: https://securitytrails.com/dns-trails 3. Replace your CNAME record in your DNS provider with A records that point to the IP addresses you just found. Your site should come back up shortly…

Presumably if your site is behind Cloudflare then this strategy won't work, right? Since the IP addresses that Security Trails sees are just of Cloudflare rather than your actual Heroku IPs...? Or is it possible in the Cloudflare dashboard there is somewhere to see your Heroku server's IP address?

Worked for us on Cloudflare

Re: Issues with upstream DNS provider

#69

Post by rsweeney21 »

theunixbeard wrote: post #45
rsweeney21 wrote: post #35

We just deployed a workaround that brought our site back up. 1. Find your DNS Target in heroku. It should end with .herokudns.com 2. Lookup the historical DNS record to get the IP addresses. You can find historical DNS records here: https://securitytrails.com/dns-trails 3. Replace your CNAME record in your DNS provider with A records that point to the IP addresses you just found. Your site should come back up shortly…

Presumably if your site is behind Cloudflare then this strategy won't work, right? Since the IP addresses that Security Trails sees are just of Cloudflare rather than your actual Heroku IPs...? Or is it possible in the Cloudflare dashboard there is somewhere to see your Heroku server's IP address?

We use cloudflare. If you look up the IP address for your public domain name, you will get the cloudflare IP, yes. If you lookup the IP address of the CNAME target, you will get the heroku IP.

Re: Issues with upstream DNS provider

#70

Post by frameworkzach »

rsweeney21 wrote: post #35

We just deployed a workaround that brought our site back up. 1. Find your DNS Target in heroku. It should end with .herokudns.com 2. Lookup the historical DNS record to get the IP addresses. You can find historical DNS records here: https://securitytrails.com/dns-trails 3. Replace your CNAME record in your DNS provider with A records that point to the IP addresses you just found. Your site should come back up shortly…

This worked for us too - thank you!

Edit: It also works for subdomains.

Post reply on HN