Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

community.influxdata.com

Post reply on HN
Print view View original

21–30 of 278 posts

Re: InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

#21

Post by teeray »

tredre3 wrote: post #4

> According to the support, the notification emails to the users were sent on Feb 23, Apr 6 and May 15th. However, we did not receive those at all. If true, this is concerning. One message getting lost in spam understandable. But three over 6 months would imply they're being blacklisted and/or their mail sender is simply broken. Do serious companies not have canaries or other checks in place to ensure their notificat…

> Do serious companies not have canaries or other checks in place to ensure their notifications are correctly delivered to customers?

Even with a canary, you have no guarantee that a particular customer will receive your email. In this case, maximum CYA is a certified letter sent return receipt requested. There are services out there to mass mail those.

Re: InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

#22

Post by fbdab103 »

tredre3 wrote: post #4

> According to the support, the notification emails to the users were sent on Feb 23, Apr 6 and May 15th. However, we did not receive those at all. If true, this is concerning. One message getting lost in spam understandable. But three over 6 months would imply they're being blacklisted and/or their mail sender is simply broken. Do serious companies not have canaries or other checks in place to ensure their notificat…

Even if all three emails were properly delivered, that is not sufficient notice for a storage service. Why is there not also a reminder on the dashboard?

A financial service I use was recently purchased by another. The company has been aggressive in keeping me on the loop about what is upcoming. Maybe six months before the actual event a heads up. Again at two months. Then at one month. Then every week, along with countdowns to the deadline. "Are you ready? This is really happening. Here are relevant docs on how to ensure your transition goes smoothly."

Re: InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

#23

Post by ilyt »

teraflop wrote: post #20
lopkeny12ko wrote: post #14

Wow, the incredibly callous 3-word explanation of the issue by pointing to a docs link with no other context. Really gives off "it's your fault for not reading the wiki." Is this how InfluxDB treats their customers? Incidentally at work we've been evaluating a new hosted observability provider, looks like we can rule out Influx as an option.

> Really gives off "it's your fault for not reading the wiki." Is this how InfluxDB treats their customers? I don't see any indication that the person who posted that is associated with InfluxDB. In fact, it doesn't seem like any staff member has posted in that forum in the past week. Up to you if you consider that better or worse.

It's weekend and I assume people designated to answer questions on forums do not work 24/7.

It could be them not sending notification about it early enough or at all, it could be that note being stuck in spam or something, it could be person complaining not reading the e-mails.

I wouldn't jump to conclusions here

Re: InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

#24

Post by scosman »

remram wrote: post #16
SSLy wrote: post #9

Earlier quoted context omitted.

you start with reliability brownouts. first fail 0.1% requests, then after a week 1%, then after a month 5%.

That seems like the worst of both worlds, during the brown-out you have to keep paying for the compute while your customers don't get a reliable service, even if they have a plan to migrate. Also you probably can't keep charging customers for that period since you offer a crippled service on purpose.

If you are shutting it down, you can pay for the grace period. Period.

Re: InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

#25

Post by Rapzid »

tredre3 wrote: post #4

> According to the support, the notification emails to the users were sent on Feb 23, Apr 6 and May 15th. However, we did not receive those at all. If true, this is concerning. One message getting lost in spam understandable. But three over 6 months would imply they're being blacklisted and/or their mail sender is simply broken. Do serious companies not have canaries or other checks in place to ensure their notificat…

Mature ops use brownouts for this exact reason. Shut the service off for a few hours and you have the attention of anyone still using the service.

Re: InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

#26

Post by adrr »

arp242 wrote: post #13
tredre3 wrote: post #4

> According to the support, the notification emails to the users were sent on Feb 23, Apr 6 and May 15th. However, we did not receive those at all. If true, this is concerning. One message getting lost in spam understandable. But three over 6 months would imply they're being blacklisted and/or their mail sender is simply broken. Do serious companies not have canaries or other checks in place to ensure their notificat…

If a spam service erroneously marks one email as spam, chances are it will also marks other very similar emails as spam. So it's not too surprising all three were marked. For these kind of automated emails getting all emails consistently being delivered to everyone is really hard, almost impossible. The problem here isn't really that emails weren't being delivered, it's that they seem to have tried only one method to…

It could end up in the promotions tab or update tab that no one checks.

Re: InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

#27

Post by remram »

scosman wrote: post #24
remram wrote: post #16

Earlier quoted context omitted.

That seems like the worst of both worlds, during the brown-out you have to keep paying for the compute while your customers don't get a reliable service, even if they have a plan to migrate. Also you probably can't keep charging customers for that period since you offer a crippled service on purpose.

If you are shutting it down, you can pay for the grace period. Period.

Just shut it down for real (after proper early warnings), so you save on compute and no one is confused about the state, and offer data retrieval for the grace period.

Brownouts are great for API changes, but not very useful before a full shutdown.

Re: InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

#28

Post by linsomniac »

I like Influx, and have been using it for all our TSDB needs for ~7 years. It sounds like there have been a lot of shake-ups internally, all their hopes for Flux (the InfluxQL language) have been aborted. I'm wondering about how they're doing.

This situation doesn't seem like it is specifically them doing anything wrong, if indeed they did send out multiple notifications over 6 months. It sounds like it's caught many people by surprise though, which makes one wonder if there was a problem with their announcements of this change. Definitely seems like it could have been handled better than "deletion with no recovery" though, some sort of "shutdown, wait a week or month, delete" would have been better.

Re: InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

#29

Post by pauldix »

Hi, cofounder and CTO here. We notified everyone via email on February 23, April 6 and May 15th. We also offered to help migrate all users. I realize that it's not ideal that we've shut down this system, but we made our best efforts to notify affected users and give them options to move over to other regions. If you've been impacted by this, please email me personally and I will do my best to help out: paul at influxdata.com.

Re: InfluxDB Cloud shuts down in Belgium; some weren't notified before data deletion

#30

Post by chillfox »

pauldix wrote: post #29

Hi, cofounder and CTO here. We notified everyone via email on February 23, April 6 and May 15th. We also offered to help migrate all users. I realize that it's not ideal that we've shut down this system, but we made our best efforts to notify affected users and give them options to move over to other regions. If you've been impacted by this, please email me personally and I will do my best to help out: paul at influx…

Email only is not even close to best effort. I know it’s standard to only do email for tech companies, but all other types of companies usually do physical mail and phone calls on top of emails for important notifications.

I am not a customer, but it’s really annoying me how tech companies repeatedly think sending emails is somehow anything but the absolute minimum, most lazy option.

Post reply on HN