Live data from Hacker News

Cloudflare silently deleted my DNS records

txti.es

361–370 of 433 posts

Re: Cloudflare silently deleted my DNS records

#361
post #189

Occasions are rare where I get to say "hey, I built a thing that might help here!" - so forgive me as I take this opportunity with both hands. Whether this was a bug or a rare protective mechanisme, there will be times when your DNS provider makes a mistake and removes records. You mentioned in your post your DNS isn't hard to reproduce, but how certain are you that _all_ records are restored? How long do you have to…

Nice! A few years ago I whipped up something simple using dig, diff, cron and some bash scripts. It's handy to get alerted when something changes, and I've definetly caught a few unintentional changes.

Re: Cloudflare silently deleted my DNS records

#362

Earlier quoted context omitted.

We used ns1 at my last job, they were indeed great to us. We moved from self-hosting DNS because the DNS servers would randomly become unresponsive and would start returning fake records. After switching to ns1 and getting our first bill, we realized that a lot of our network equipment apparently did a DNS lookup for every log line. This resulted in an exceedingly large bill, which ns1 happily reversed (we did fix ou…

I thought self-hosting DNS would be easy, since it is 40 year old tech now. Even now, like you, I get the occasional (ie. once per year) named segfaults or it randomly stops responding over TCP. Few SaaS products are less effort than one reboot per year, but still worth it IMO.

Since then I've started using CoreDNS, which does seem easier to monitor. (I don't know if it's faster or more reliable, but it has a lot of ways to figure out what's going wrong. As it turns out, DNS causes people a lot of trouble these days when they use it for service discovery, and their services discover each other hundreds of times a second. So that's why DNS servers grew APIs and observability features.)

Re: Cloudflare silently deleted my DNS records

#363
post #286

Earlier quoted context omitted.

I looked into self-hosting DNS and it doesn't seem like that big of a deal as long as you can ensure uptime to be honest. If you set up the two first on different hosts and possibly have #3/4 being cloud providers I think you're pretty good. Does anyone here have experience with running their own DNS servers for their domains?

You don't even need multiple servers (especially if both your website and mail run on the same server), it's a misconception debunked by the author of djbdns: http://cr.yp.to/djbdns/third-party.html

That talks about whether you have servers on multiple networks. If it debunks the idea that you should have at least two servers I can't find where.

Re: Cloudflare silently deleted my DNS records

#364
post #189

Occasions are rare where I get to say "hey, I built a thing that might help here!" - so forgive me as I take this opportunity with both hands. Whether this was a bug or a rare protective mechanisme, there will be times when your DNS provider makes a mistake and removes records. You mentioned in your post your DNS isn't hard to reproduce, but how certain are you that _all_ records are restored? How long do you have to…

Just wanted to point out that the menu doesn't work properly on mobile in case you want to fix it.

https://imgur.com/a/TcibAKs

Re: Cloudflare silently deleted my DNS records

#365

Earlier quoted context omitted.

Then I'd like to know the CEO's opinion then.

It's hard to understand criticism when it is not put into words. From my perspective, a developer from the company the article is about arrives in the comment section. I ask an honest question about the communication process internal to the company. I am told this developer is the CTO and is supervised by the CEO. Then I state the fact that I would appreciate if we also knew what the CEO thinks about this situation.…

This reply might be what you're looking for: https://news.ycombinator.com/item?id=22407399

Cloudflare is a solid company with a good reputation and has done a lot for general welfare of the internet. The staff there generally care about customers and it seems like they're trying to figure out what went wrong here, and of course preventing negative PR is always a bonus.

There's really not much of a process when you're a top officer at a company, you have the freedom and responsibility to exercise your judgement and do whatever it takes for the good of the organization.

Re: Cloudflare silently deleted my DNS records

#366

This is being looked into internally and I am involved. Likely won’t post an update here as it pertains to a customer account (unless customer agrees). BTW If you, dear reader, ever find yourself so frustrated with Cloudflare that you feel like your only recourse is a blog post... my email is jgc@cloudflare.com and I’m happy to hear from people.

I've put this idea forward a number of times here on HN in regards to other big tech companies. Technology companies need an "ombudsman" - a contact that customers can go to when the normal tech support processes have failed. The Ombudsman must not be part of the technology companies ordinary support processes, it must be entirely separate, and have highest level authority to demand action within the company. To avoi…

Didnt we just have a 500+ comment rant about how shit HackerOne was?

Re: Cloudflare silently deleted my DNS records

#367
post #359

Earlier quoted context omitted.

> if there's no cost to the company to ignore the customer Simplifying, that's "if (false)", since ignoring the customers in the limit gets you a failed business and bankruptcy.

For the non-enterprise user, isn't this the scenario established today, with no failed business?

No, I don't think so. All the major internet companies listen to their non-enterprise users. They may not provide enterprise quality support to each individual user, but user voices in aggregate are considered critically important.

Re: Cloudflare silently deleted my DNS records

#369
post #358

Earlier quoted context omitted.

> if there's no cost to the company to ignore the customer Simplifying, that's "if (false)", since ignoring the customers in the limit gets you a failed business and bankruptcy.

not if you have a monopoly. Which is why customer support for most ISPs in the US is so abysmal. And even if it is just a sufficiently large company, if the customer is not a large payer and the issue is non-trivial to solve, the the cost of losing that customer could very well be less than the cost of fixing the issue.

What I was addressing was specifically the notion that companies pay zero cost for ignoring a customer, or the idea that they believe the same. That's false. Now there's no question there are some customers these large companies are better off without, and there are large and small examples of companies firing customers, or ignoring them til they go away. Nothing wrong with that. All the time on this site we talk about firing unprofitable customers. If you do this judiciously it's just good business. Doing that in particular cases is not the same as doing that generally.

Re: Cloudflare silently deleted my DNS records

#370
post #207

Earlier quoted context omitted.

I do wonder; when a domain is suspended, why don't you send out a courtesy notification e-mail?

what if their email address is @ the domain

The OP made it clear that he used a gmail account for contact so there was no excuse for him to have not received a message regarding the deletion of his domain records.
Post reply on HN