Live data from Hacker News

InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

community.influxdata.com

131–140 of 278 posts

Re: InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

#131
post #77

Unfortunately we’ve been bitten by influx operation issues a few times before. We adopted influxDB long time ago and always had to deal with breaking changes for each upgrade and every time we had an issue their answer would be: upgrade the latest version and see if it continues. Then recently they made a change to Telegraf that broke all our data collection because they changed the environment variable replacer and…

We self host influxdb, never had this problem.

Which problem? Of the massive breaking changes between 0.8 and later, and then between 1.x and 2.x? Not to mention InfluxQL to Flux?

Also, they did remove clustering in the open source version which was a very poor move from a PR perspective. And in my view, they have never recovered from it - years ago it was Prometheus vs InfluxDB for (non-SaaS) observability metrics, nowadays the only question is which backend for Prometheus to choose.

Re: InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

#132

It seems this probably happened due to some regulation or other. The sunset date for the service should have been a month prior so that influx could have kept the data legally until the 30th in case of this situation happening. They wanted to have the euros flowing in right until the last minute. 1. Flash messages on all user facing consoles. 2. No new resource able to be created for 6 mo this. 3. Emails. 4. Service…

Perhaps service shutdown is also the only valid case where it can be okay to intermittently fail API requests?

Re: InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

#133

I've seen better communications around company-internal services that have been deprecated and for which a replacement exists that we need to migrate to. Heck, I've seen this a couple of times. We had even tried out Influx a few different times. It was always ok, but never quite good enough. Now with this, I think, this seals the deal on me ever considering Influx either as a product or as a service.

>I've seen better communications around company-internal services

Our team maintains an internal CRM. When we plan to delete data or deprecate features, what we usually do (beyond sending emails):

- hide features/data from the UI without actually deleting them - if no one complains, after a few weeks, proceed with the removal

- for critical data, make sure there are backups, store them for about a month - if no one requests them, delete them

Re: InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

#134

Earlier quoted context omitted.

Email only is not even close to best effort. I know it’s standard to only do email for tech companies, but all other types of companies usually do physical mail and phone calls on top of emails for important notifications. I am not a customer, but it’s really annoying me how tech companies repeatedly think sending emails is somehow anything but the absolute minimum, most lazy option.

Why do customers believe that they don’t need to read their emails?

Are you trolling or never used email?

Re: InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

#135
post #34
post #29

Hi, cofounder and CTO here. We notified everyone via email on February 23, April 6 and May 15th. We also offered to help migrate all users. I realize that it's not ideal that we've shut down this system, but we made our best efforts to notify affected users and give them options to move over to other regions. If you've been impacted by this, please email me personally and I will do my best to help out: paul at influx…

Paul: I'm surprised you didn't do a scream test. Not everyone is going to see those emails and even those that do may not understand what they are reading. Internally at my company we always do scream tests as part of our EOL process because we know we can't reach everyone, even our own employees. https://www.microsoft.com/insidetrack/blog/microsoft-uses-a-... Fun story: my mortgage got sold last year. Not the first…

> Not everyone is going to see those emails and even those that do may not understand what they are reading.

If that's the case, these companies/people have no business using cloud services. Fair enough that you might not understand the ramification, in that case you contact support. If you don't see those email... that's on you. We operate out of a number of datacenters, they all communicate via email, giving us one month to three notice regarding service windows. If we fail to plan for an outage because we didn't see an email, that's our problem. I don't know why anyone would expect more from a SaaS company.

For really large customers, I would assume that they have a customer service representative and yes that person should have called. If you're just a small customer (even if you might be big in your own mind) and just have an account that get billed to a company credit card each month, it's a little naive to think you'd get anything more than an email.

We've already seen a number SaaS company just shutting of customers for little to no reason, even AWS has done this. Running things in the cloud is a risk, and it's you job as the operations team to stay on top of things, have backup plan, because you cannot expect cloud vendors to care about some random customer who just signed up using a credit card and a nondescript email. They should, but they don't.

A good rule is: Don't expect a SaaS/cloud company to put in more effort contacting you than you did signing up.

Re: InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

#136

Earlier quoted context omitted.

Lmao are you serious? What about emails buried in spam? What if contact x left the company and the emails are black holes? There are a million valid reasons for emails to go poof. "But we emailed you" is weak.

You actually need to read your spam to check and if a company didn’t bother transition an employee out properly (i.e figure out what their email address was attached to), why is that on the supplier? Why do they need to move mountains so that you can avoid any seriousness about your own operations?

Because I’m the customer.

> Why do they need to move mountains so that you can avoid any seriousness about your own operations?

You won’t stay in business for long with that attitude.

Re: InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

#137

It seems this probably happened due to some regulation or other. The sunset date for the service should have been a month prior so that influx could have kept the data legally until the 30th in case of this situation happening. They wanted to have the euros flowing in right until the last minute. 1. Flash messages on all user facing consoles. 2. No new resource able to be created for 6 mo this. 3. Emails. 4. Service…

Given they shut down two DCs half a world apart, it's not regulations. It's cost.

But its a paid service right? Is it a pricing issue? If it is, isn't it better to increase the price?

Re: InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

#138
post #29

Hi, cofounder and CTO here. We notified everyone via email on February 23, April 6 and May 15th. We also offered to help migrate all users. I realize that it's not ideal that we've shut down this system, but we made our best efforts to notify affected users and give them options to move over to other regions. If you've been impacted by this, please email me personally and I will do my best to help out: paul at influx…

Multiple comments in the linked issue report not receiving an email.

Did you use the same email you use for spam/"marketing" for this notification?

The correct course of action is to shutdown the service and give people time to fetch data, not to erase the data as the first indication of shutdown.

A few emails are not sufficient if the end result is dataloss, a comment in documentation or release notes is not sufficient (the only reference at least one person in the referenced issue found).

truly mind blowing behavior.

Re: InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

#139

Earlier quoted context omitted.

> I was like: you guys should've FedEx'd me this in the first place. For all they knew, I wasn't getting their emails or letters in the first place because nothing had been sent signature required. I love the scream test, but the analogy you bring up actually - this seems unfair. The cost of Fedex'ing everyone is astounding (for many businesses). But I like the concept. Definitely a sort of "shut off the server for l…

When you buy a large, registered debt such as a mortgage the cost of Fedex'ing everyone should be factored into the sale. If that's too much money you shouldn't be buying such assets. Notifying those that are affected properly would seem to be the least you can do in such situations.

At the end of the day you are the one who pays for the fedex though.

Re: InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

#140

Earlier quoted context omitted.

When you buy a large, registered debt such as a mortgage the cost of Fedex'ing everyone should be factored into the sale. If that's too much money you shouldn't be buying such assets. Notifying those that are affected properly would seem to be the least you can do in such situations.

Exactly and they’re find sending junk mail all the time and FedEx isn’t that expensive for large businesses like that.

At least here in Germany junk mail senders actually get reduced rates.
Post reply on HN