Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloudflare Sippy: Incrementally Migrate Data from AWS S3 to Reduce Egress Fees

blog.cloudflare.com

Post reply on HN
Print view View original

11–20 of 168 posts

Re: Cloudflare Sippy: Incrementally Migrate Data from AWS S3 to Reduce Egress Fees

#11

Post by supriyo-biswas »

figassis wrote: post #3

This is amazing. I’m thinking of building the same thing for self hosted S3 (ie. Minio). I love AWS, they're very reliable and i feel like depending on them is not risky. But being 100% dependent on a single provider scares me a lot, and I often feel like S3 is the one thing that would cost me a lot if I tried to migrate. I have not accounted for all the data I have there, but it's lots of small files (all the user u…

Deletions do not encounter costs, but obtaining a list of objects in order to delete them certainly does.

Re: Cloudflare Sippy: Incrementally Migrate Data from AWS S3 to Reduce Egress Fees

#12

Post by lhoff »

monlockandkey wrote: post #7

Wow this is really clever. Would be cool to have an open source data migrator with the same concept

That thought was my first impression as well. However with a second thought i'm not sure if i'm convinced by the long time scenario that you would end up using this tool. After all it only makes sense to use this tool if you never intent to migrate the data completely. Which means, the data will till the end be scatered between AWS and Cloudflare with no apparent system other than the last access. Maybe I am overlooking something.

Re: Cloudflare Sippy: Incrementally Migrate Data from AWS S3 to Reduce Egress Fees

#13

Post by anonzzzies »

figassis wrote: post #6
pzmarzly wrote: post #4

It's a seemingly simple and obvious way to lazily migrate your data, but if using Sippy means one less thing for the application code to worry about, and (I assume) is a free add-on, then it provides a ton of value. I have to admit that Cloudflare has been killing it recently with DevX / OpsX. If I wasn't against that company's role in modern internet (as a user of Tor, their firewall is annoying to no end), I would…

This, CF is the only service that I find amazing, and that I do not use for anything. Compared to AWS, I think I prefer the "we're your unopinionated infra provider, if you want a WAF we have that too", vs the CF "block the world, especially the developing world, give zero craps about it". I fundamentally would be unhappy as their customer even if their service were stellar because I do not want my apps to be associa…

The problem is; as a tiny SaaS owner, we still get massive DDOS attacks a few times per year. With other services I would pay a fortune; with CF it’s free. Downtime has people walk away and CF saved my bacon many times. If there are friendlier alternatives that work as effective then I have not found. Mind you, I don’t mind paying a few 100$/mo for such a service but on aws etc one attack burns that for more than a year. The one over Christmas last year would’ve been enough to just close the doors if I had mitigated that via aws.

Re: Cloudflare Sippy: Incrementally Migrate Data from AWS S3 to Reduce Egress Fees

#17

Post by NicoJuicy »

pzmarzly wrote: post #4

It's a seemingly simple and obvious way to lazily migrate your data, but if using Sippy means one less thing for the application code to worry about, and (I assume) is a free add-on, then it provides a ton of value. I have to admit that Cloudflare has been killing it recently with DevX / OpsX. If I wasn't against that company's role in modern internet (as a user of Tor, their firewall is annoying to no end), I would…

Cloudflare doesn't block TOR specifically.

It's due to their users and associated behavior that tor Exit nodes have an elevated bad reputation.

> https://developers.cloudflare.com/support/firewall/learn-mor...

> Due to the behavior of some individuals using the Tor network (spammers, distributors of malware, attackers, etc.), the IP addresses of Tor exit nodes may earn a bad reputation, elevating their Cloudflare threat score.

Customers of cloudflare have an option to improve experience for Tor users

> Beyond applying firewall filters to Tor traffic, Cloudflare users can improve the Tor user experience by enabling Onion Routing. Onion Routing allows Cloudflare to serve your website’s content directly through the Tor network, without requiring exit nodes.

Email the sites where you have issues and ask them to enable Tor routing.

Re: Cloudflare Sippy: Incrementally Migrate Data from AWS S3 to Reduce Egress Fees

#18

Post by NicoJuicy »

lhoff wrote: post #12
monlockandkey wrote: post #7

Wow this is really clever. Would be cool to have an open source data migrator with the same concept

That thought was my first impression as well. However with a second thought i'm not sure if i'm convinced by the long time scenario that you would end up using this tool. After all it only makes sense to use this tool if you never intent to migrate the data completely. Which means, the data will till the end be scatered between AWS and Cloudflare with no apparent system other than the last access. Maybe I am overlook…

Nah. This is literally to have a migration path and to only occur once the Amazon eggres fees.

At the end, you can still decide to migrate all data or to abandon the not-used-till-now data.

Re: Cloudflare Sippy: Incrementally Migrate Data from AWS S3 to Reduce Egress Fees

#19

Post by andrewstuart »

9 cents a gigabyte downloaded versus 0 cents a gigabyte downloaded is a pretty good deal.

There’s not much AWS can do about it because they must make untold billions from those sweet, sweet S3 egress fees.

I’d be willing to bet S3 egress fees make up about 60% of all AWS revenue.

Post reply on HN