If these kind of entities actually want to preserve resources, they shouldn't be generating a petabyte of bandwidth charges. Contact soundcloud and come to an agreement that will be responsible.
Archiveteam exists mostly because your suggested course of action rarely works.
Archiveteam are backing up SoundCloud
201–210 of 230 posts
Re: Archiveteam are backing up SoundCloud
#202If these kind of entities actually want to preserve resources, they shouldn't be generating a petabyte of bandwidth charges. Contact soundcloud and come to an agreement that will be responsible.
That would be legally iffy, since they don't own the copyright, and so it's unlikely that they're allowed to bless an wholesale copy of their users' content to some third-party.
Re: Archiveteam are backing up SoundCloud
#203Earlier quoted context omitted.
> I'm sure SC serves up a lot of content per day, but how do you think they will react by suddenly having someone download all of their 900 TB or whatever it is in one day? How much will Archiveteam be contributing to SC's downfall by suddenly causing them a huge unexpected bill? Obviously, there's more than just the bandwidth cost, but assuming they pay $0.02/GB for CDN traffic, we're talking about $18k. It's not no…
That's making the incorrect assumption that all 900TB is hosted by the CDN... you typically only have a fraction of your content, known as your working set, in the CDN's working cache. So you should factor in the bandwidth cost for CDN traffic AND origin pulls, which if you're serving from AWS (and not using CloudFront), is $0.08/GB. The size of your working set also influences your overall CDN bill. Storage isn't fr…
Why stay with a host that bends you over a barrel for transit? Even Netflix doesn't use Amazon to deliver their data heavy content, as their pricing is pie in the sky high.
Re: Archiveteam are backing up SoundCloud
#204Earlier quoted context omitted.
And this is how Google Drive Unlimited becomes Google Drive Limited... Like every other Unlimited service, because people like this abuse the shit out of it then when it inevitably dies they run around exclaiming "dont blame me brah they said it was unlimited...., shouldn't have called it unlimited"
It's okay, for everyone one like this, there's 1000's like me that pay for the service and use a fraction of it. Think of it as we are all contributing to keeping this data safe and available.
Re: Archiveteam are backing up SoundCloud
#205Earlier quoted context omitted.
Except they're not an asshole. Companies should be responsible for what they put in their marketing. If someone calling the bluff on the "unlimited" plan makes them "downsize" the plan to actual ~10TB + $x per Y additional TB, then so be it. It probably doesn't change the reality in any way, and at least the company is no longer lying about its service.
The problem is they dont downsize to a "reasonable" 10TB, they over correct in the opposite directly like Amazon did, and people that used the service as intended get fucked.
Re: Archiveteam are backing up SoundCloud
#206Earlier quoted context omitted.
That's making the incorrect assumption that all 900TB is hosted by the CDN... you typically only have a fraction of your content, known as your working set, in the CDN's working cache. So you should factor in the bandwidth cost for CDN traffic AND origin pulls, which if you're serving from AWS (and not using CloudFront), is $0.08/GB. The size of your working set also influences your overall CDN bill. Storage isn't fr…
Is it just me, or is charging pennies per gigabyte usurious? If I paid even a penny per gig on my home internet, I'd have nearly a $150 bill just in gig charges, and I know I'm one of the lower bandwidth customers on my block. Why stay with a host that bends you over a barrel for transit? Even Netflix doesn't use Amazon to deliver their data heavy content, as their pricing is pie in the sky high.
Re: Archiveteam are backing up SoundCloud
#207Earlier quoted context omitted.
This seems a bit ridiculous. 900TB costs $22000 in hard drives (assuming $100/4TB HDD), without any redundancy. I wonder what their storage solution is like.
Consider what Soundcloud, which ist hosted on AWS, payed for this. The cheapest public offer from AWS is 0.05$/GB for OUTBOUND traffic (for at least 350TB/month). If you consume more, you negotiate directly with AWS. Lets assume they pay 0.005$/GB, which results in a 4500$ bill(@ 0.005/GB) or 45000$ bill(@ 0.05$) for Soundcloud.
Actually it's $0.02/GB if your traffic is in the US and you do 5PB/month: https://aws.amazon.com/cloudfront/pricing/
But yes, they likely have a non-public deal.
Re: Archiveteam are backing up SoundCloud
#208It's a node tool built a few years ago to download the playlists of users through your command line. Might be helpful for a situation where you'd like to back up your own playlists.
You'll need to get an API key - no sure how feasible that is at this moment.
Re: Archiveteam are backing up SoundCloud
#209Earlier quoted context omitted.
Is it just me, or is charging pennies per gigabyte usurious? If I paid even a penny per gig on my home internet, I'd have nearly a $150 bill just in gig charges, and I know I'm one of the lower bandwidth customers on my block. Why stay with a host that bends you over a barrel for transit? Even Netflix doesn't use Amazon to deliver their data heavy content, as their pricing is pie in the sky high.
What block do you live on that 15TB is on the low end?
Re: Archiveteam are backing up SoundCloud
#210How can tracks be downloaded when many of them are pay-to-access? or stream-only?
Generally, if you "stream" (send) me bits, they're mine to do what I want with (practically, maybe not legally). Although, if anyone could tell me how to do that with AES-SAMPLE HLS video, I'd be very happy.
I'm always confused when people say they've backed up some website when there are these practical and legal barriers.