Cloudflare silently deleted my DNS records • Hacker News Board

Re: Cloudflare silently deleted my DNS records

#311

Post by donmcronald » Mon, Feb 24, 2020, 11:45 PM UTC

flurdy wrote: post #78

whatthesmack wrote: post #2

This is frightening. I just started the process of moving all ~60 of my domains from Amazon Registrar + Google Cloud DNS to Cloudflare, and will definitely wait until somebody from Cloudflare chimes in here to clarify what's going on.

Don't put all your eggs in one basket, ie. don't just use one provider. Also for your core domains, do not let the registrar and dns provider be the same entity. Also, don't decide on not migrating just because of one bad experience. None of them are perfect, though vigilance is wise. (I know am probably preaching to the choir :) )

This is what I do. Domains are at Porkbun and domain registration is the ONLY thing I use them for. DNS and caching are at Cloudflare. Hosting is wherever makes sense.

Consolidating domain registration, DNS hosting, and site hosting under the same account is a terrible idea and a big risk IMO. Always ask yourself “what if this account gets banned?” All of the big tech companies have automated systems that could turn you into an outlier with no support.

Re: Cloudflare silently deleted my DNS records

#312

Post by [deleted] » Mon, Feb 24, 2020, 11:49 PM UTC

jlawer wrote: post #200

jfkebwjsbx wrote: post #171

Earlier quoted context omitted.

$20 won't stop anyone with using it. If it is refundable, make it $300. That is high enough that only people and businesses with a showstopper situation will use it.

$300 becomes a pretty high barrier for a lot of people. There a places with decent connectivity where that is a good weeks wages

[deleted]

Re: Cloudflare silently deleted my DNS records

#313

Post by nickjj » Tue, Feb 25, 2020, 12:01 AM UTC

TylerE wrote: post #277

nickjj wrote: post #233

Earlier quoted context omitted.

> To avoid the Ombudsman being overused, you could give it a price of say $20, which is always refunded when the case is resolved. What if the cost was put onto the business instead of the consumer and the business just hired support people who are all Ombudsmans by default? Instead of focusing on copy / pasting boilerplate scripts and answering as many tickets as possible, they should focus on the problem the custom…

Because that isn't a useful allocation of resources. Support that powerful are going to basically be devs. With dev salary expectations.

I think it's useful and it's very much worth it to give support a salary where they are enthusiastic to be employed.

If someone uses your product or service and has an issue, the first contact they have with your company is through support.

A crappy support experience could easily be the difference between having a life time customer valued at thousands of dollars, or your customer feeling frustrated and going to another company, netting you $0. That has a butterfly effect too because having 100 happy customers who are praising your support could lead to many new sales from organic recommendations. Having a bunch of customers who felt neglected by your support could yield a situation where your company is now on the front page of HN for having bad support or worse.

For whatever reasons, bigger companies focus more on measurable metrics like "tickets closed per hour", where the emphasis is on things that aren't important because measuring customer happiness is pretty subjective and doesn't translate well to employee evaluation scores.

I know if I ever got the point where I would need to delegate support assistance to someone, you can be sure I would pay them amazingly well, at least equal to a developer's salary because I never want anyone to ever feel like they get ignored or have a low quality experience with my products.

Re: Cloudflare silently deleted my DNS records

#314

Post by tedivm » Tue, Feb 25, 2020, 12:02 AM UTC

SahAssar wrote: post #289

mark_and_sweep wrote: post #228

Earlier quoted context omitted.

Well, I haven't analysed the exact technical reason for why submitting the order failed. But I'm pretty certain that submitting a HTML form is a solved problem in web development.. Or at least it should be. I haven't tried submitting a form with an async fetch from a web worker that communicates with a redux store implemented in WebAssembly yet (or whatever that web app is doing..).

If the order site is just submitting an HTML form in the old way with credentials stored in a cookie (also the old way) that would probably be open to trivial CSRF attacks. If it is somehow checking for support for SameSite, Secure, CSP or any of the other mechanisms that have been implemented in the last years then it might fail. Or they might be using mechanisms that work around the problem that those three are sup…

CSRF doesn't require any javascript to defend against. Having a nonce inside of the HTML resolves that completely. SameSite and Secure cookies also don't require any javascript, just some extra HTTP headers. I don't think security justifies this.

Re: Cloudflare silently deleted my DNS records

#315

Post by iudqnolq » Tue, Feb 25, 2020, 12:04 AM UTC

DangerousPie wrote: post #307

jiggawatts wrote: post #279

Earlier quoted context omitted.

Please explain something to me. For me and my of my customers, having your "entire cloud deleted" is like... the #1 nightmare scenario. So why does this capability/function even exist for active accounts at CloudFlare? It sounds like the OP fell victim to what is essentially a regular process. Or to put it another way: No amount of explanation or assurance is ever going to make me feel comfortable with my doctor havi…

It doesn't sound like it actually deleted anything other than the DNS entries. It's not like any actual data got lost.

(OP here)

It appeared to me as though they deleted my domain registration from their system. That's mostly why I freaked out. I wasn't sure if I'd even be able to transfer my registration easily.

However, on the advice of someone from HN, I found that if I went through the flow of linking an external domain to my cloudflare account the registration details came back.

Re: Cloudflare silently deleted my DNS records

#316

Post by londons_explore » Tue, Feb 25, 2020, 12:06 AM UTC

iudqnolq wrote: post #208

dkersten wrote: post #198

Earlier quoted context omitted.

Imagine how you would feel though if you were in OP’s shoes, where your DNS entries disappeared and client emails lost, through no fault of your own, but support won’t even listen to you until you pay money.

I think I'd be fine with that? I would trust Cloudflare to pay me back, and I'm already putting something on the line. If it really was my fault this is going to be very embarrassing. Edit: In case it isn't clear from context I'm the OP so I'm pretty sure I know what I feel.

Except you can bet that a lot of cases will end up with a battle of back and forth emails arguing over whose fault it was and who should pay the $100.

The vast majority of support tickets are the customers 'fault'. They'll be things like setting up DNS records incorrectly, believing they don't need to pay a renewal with their old registrar just because they moved their DNS over to cloudflare, frustrations over cached resources, etc.

Re: Cloudflare silently deleted my DNS records

#317

Post by thefifthsetpin » Tue, Feb 25, 2020, 12:13 AM UTC

wu_187 wrote: post #207

nodesocket wrote: post #108

Earlier quoted context omitted.

I do wonder; when a domain is suspended, why don't you send out a courtesy notification e-mail?

what if their email address is @ the domain

Even if cloudflare stopped publishing the correct MX records for a domain, wouldn't Cloudflare still be able to send email to the MX server that had been specified? Seems like email should at worst be down for everyone but Cloudflare.

Re: Cloudflare silently deleted my DNS records

#318

Post by to11mtm » Tue, Feb 25, 2020, 12:15 AM UTC

basch wrote: post #238

herbstein wrote: post #232

Earlier quoted context omitted.

> But you get it back You're missing the part where some people, that actually would need this support, literally wouldn't be able to find that money because of the difference in purchasing power of the local currency.

I was thinking price for business not personal accounts. Maybe youre thinking those are the same thing in freelance.

No, He's thinking about various developing nations that are trying to have a tech boom.

Phillipines is a good example; Average yearly salary is somewhere around 12k USD, Median is 8k USD.

That means if someone is doing a tech startup there, 300$ is somewhere between a quarter and a half of some employee's pay for a -month-. And given the potential for tight margin/cash flow of startups even in the US... it would still price a lot of smaller players out of the market.

Re: Cloudflare silently deleted my DNS records

#319

Post by londons_explore » Tue, Feb 25, 2020, 12:17 AM UTC

jrockway wrote: post #241

samcrawford wrote: post #139

Earlier quoted context omitted.

I've used ns1 for a few years, they've been great!

We used ns1 at my last job, they were indeed great to us. We moved from self-hosting DNS because the DNS servers would randomly become unresponsive and would start returning fake records. After switching to ns1 and getting our first bill, we realized that a lot of our network equipment apparently did a DNS lookup for every log line. This resulted in an exceedingly large bill, which ns1 happily reversed (we did fix ou…

I thought self-hosting DNS would be easy, since it is 40 year old tech now.

Even now, like you, I get the occasional (ie. once per year) named segfaults or it randomly stops responding over TCP.

Few SaaS products are less effort than one reboot per year, but still worth it IMO.

Re: Cloudflare silently deleted my DNS records

#320

Post by p1necone » Tue, Feb 25, 2020, 12:22 AM UTC

andrewstuart wrote: post #125

jgrahamc wrote: post #83

This is being looked into internally and I am involved. Likely won’t post an update here as it pertains to a customer account (unless customer agrees). BTW If you, dear reader, ever find yourself so frustrated with Cloudflare that you feel like your only recourse is a blog post... my email is jgc@cloudflare.com and I’m happy to hear from people.

I've put this idea forward a number of times here on HN in regards to other big tech companies. Technology companies need an "ombudsman" - a contact that customers can go to when the normal tech support processes have failed. The Ombudsman must not be part of the technology companies ordinary support processes, it must be entirely separate, and have highest level authority to demand action within the company. To avoi…

The problem is that big companies don't care about giving quality support for their products, and for the most part they get away with it. From their perspective there's no problem to solve.

Your solution basically boils down to "companies are failing at escalating support issues well, so they should escalate support issues well."