Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

BunnyCDN has been silently losing our production files for 15 months

old.reddit.com

Post reply on HN
Print view View original

11–20 of 51 posts

Re: BunnyCDN has been silently losing our production files for 15 months

#11

Post by simoncion »

kjs3 wrote: post #8

Wait...this has been going on for 15 months without resolution or recompense and you haven't pulled up stakes and moved to almost anyone else? I get "it's a lot of work to move" and maybe "we hope they'll figure this out so we don't have to move", but in my world that excuse runs out waaaaaay before 15 months. The people I'm accountable to would have hauled me out back and put me out of their misery after, maybe, 6 m…

> ...but in my world that excuse runs out waaaaaay before 15 months.

I expect the combination of

  Yes, we should've migrated away sooner, we never had the capacity to do so and hoped Bunny would just get their shit together.
and

  The loss rate isn't enormous in percentage terms, but it's consistent and ongoing.
means that detecting and dealing with the loss is substantially less work than moving away. [0] I expect that Management is fully aware of what's up and is making the call here.

[0] "Just" add a retry if the post-upload verification step fails! Sure, it's slower, but it works, right??? mournful sob

Re: BunnyCDN has been silently losing our production files for 15 months

#12

Post by getcrunk »

I’ve used bunny for a few years … happily. I wonder if this is a bug due to some meta data of the files like the names or something. Very weird. Good thing you had metrics to catch it.

I upload all object storage stuff to bunny for live but also to backblace for backup.

I’ve always wanted to implement fail over client side for any asset over to bacblaze but seems like a lot of overhead

Re: BunnyCDN has been silently losing our production files for 15 months

#13

Post by sergiotapia »

Havoc wrote: post #10
s09dfhks wrote: post #2

Interesting timing given the post yesterday about someone switching to BunnyCDN from cloudflare

To be fair the Reddit user account is 8 years old. Not conclusive but does suggest it may be organic

you can purchase reddit accounts with age/content.

Re: BunnyCDN has been silently losing our production files for 15 months

#15

Post by dgl »

I tried using their Magic Containers product and there were issues that showed a lack of attention to detail as well.

It's supposed to scale globally (magically!) but I found multiple cases where particular nodes were problematic and the health checks didn't detect them (in fact to start with the health checks didn't even work properly if you had multiple containers, they did fix that). The support was quite slow too, after finding multiple product issues they'd escalate to developers and then come back a month later and ask to retest, but some of this took multiple round trips. I was only using this on a side project, but definitely wouldn't consider them for anything critical, even if they are quite cheap.

Re: BunnyCDN has been silently losing our production files for 15 months

#17

Post by 12907835202 »

reddalo wrote: post #4

That's scary. I'm in the process of moving all of my services to Europe, and I had considered BunnyCDN, but after this I'm not so sure anymore. I also tried Hetzner Object Storage. I love Hetzner, they're great, except for their Object Storage service, which is completely unreliable (errors, slowness, etc.). I'm surprised that Hetzner still hasn't retired that product until it's properly fixed. My last chance is with…

I guess you may have alot of files, but to me object storage is so cheap I keep copies on Aws S3, wasabi, r2 and a 16 TB HDD on my hetzner server.

Admittedly 4 might be too many. But at some point I switched to r2 for the free public egress and deleting one of Aws or wasabi has never been a priority and I don't want to do it without putting in the time to quadruple check I'm not deleting anything important.

But at the very least id have 3. I'd hate to discover that my S3 was blown up in a war and then copying everything my HDD was the last straw that pushed an aging drive over the edge.

Re: BunnyCDN has been silently losing our production files for 15 months

#20

Post by ipaddr »

DANmode wrote: post #7
m3nu wrote: post #6

Also a user of their CDN, storage and container service for about 4 years. My experience was more positive. One time I had a minor issue with their storage where I couldn't replace a file or something. This was fairly early after the product launched. They fixed it and gave me free credit for reporting it.

If the story is true, remaining a user seems…terrifying?

Because they fixed a minor issue many years ago?
Post reply on HN