Aren’t all the authors of that paper from Apple?
Apple downloads ~45 TB of models per day from our S3 bucket
31–40 of 237 posts
Re: Apple downloads ~45 TB of models per day from our S3 bucket
#32I don't have high hopes for his business prospects if this is how he handles one of the richest companies in the world clearly having a high need for something his company offers. Maybe spend less time on Twitter and more on your business model?
They're basically bragging they have something Apple really wants. Now they have a bunch of people at least interested in what they got. I'll say that's not a bad PR.
Also a great way to throw out massive red flags to any enterprise user that cares about privacy and non-disclosure.
IP address data is pretty sensitive information, and throwing it out there like this, even in aggregate, is not OK because of what it shows.
No matter how much PR this gets, this goes both ways.
Re: Apple downloads ~45 TB of models per day from our S3 bucket
#33What’s the backstory on this? (Is it something I should already know)
Re: Apple downloads ~45 TB of models per day from our S3 bucket
#34What does hugging face do? Do they implement models from papers and make them available for free?
Yes. I don’t know if that’s all that they do. They often port new Tensorflow models to PyTorch as well. They provide straightforward APIs, nice documentation, and clear tutorials. I use their stuff pretty regularly.
Re: Apple downloads ~45 TB of models per day from our S3 bucket
#35In a few weeks he can just point Apple's IP range to a shared iCloud folder.
Re: Apple downloads ~45 TB of models per day from our S3 bucket
#36Earlier quoted context omitted.
Backblaze B2 + Cloudflare would be a perfect combination for hosting a static site. Unfortunately there's no way to map a Backblaze bucket to a domain, so even if you use cloudflare to point www.mydomain.com to it your files still show up at www.mydomain.com/path/to/bucket. But it certainly works if you need to CDN a bunch of large files.
You can use Cloudflare Workers to rewrite the path.
Those get fairly expensive if you have a lot of requests.
Page rules are free if you only need a few, though.
Re: Apple downloads ~45 TB of models per day from our S3 bucket
#37This looks kind of interesting:
https://github.com/huggingface/pytorch-pretrained-BigGAN/blo...
When you look further down you find:
https://github.com/huggingface/pytorch-pretrained-BigGAN/blo...
And that's just a quick search for s3 in the repo. It would not surprise me in the least to discover a `from_pretrained` that points at one of the s3 resources being pulled. There's probably other stuff like that as well in the code that could be causing equally nasty heartache .. especially if non-persistent containers are involved....
(This is a WAG aka Wild A Guess)
EDIT: Dug a little more and found:
https://github.com/search?q=org%3Ahuggingface+s3&type=Code
Unless I'm mistaken here, there's a crap ton of code that could be downloading models at runtime ... Which seems significantly less than ideal ...
Re: Apple downloads ~45 TB of models per day from our S3 bucket
#38Earlier quoted context omitted.
They're basically bragging they have something Apple really wants. Now they have a bunch of people at least interested in what they got. I'll say that's not a bad PR.
Also a great way to throw out massive red flags to any enterprise user that cares about privacy and non-disclosure. IP address data is pretty sensitive information, and throwing it out there like this, even in aggregate, is not OK because of what it shows. No matter how much PR this gets, this goes both ways.
big companies are also notorious for reaping whatever they can take from smaller companies...and when its time for the smaller company to monetize..."whoops we don't have budget for that."
Re: Apple downloads ~45 TB of models per day from our S3 bucket
#39This looks kind of interesting: https://github.com/huggingface/pytorch-pretrained-BigGAN/blo... When you look further down you find: https://github.com/huggingface/pytorch-pretrained-BigGAN/blo... And that's just a quick search for s3 in the repo. It would not surprise me in the least to discover a `from_pretrained` that points at one of the s3 resources being pulled. There's probably other stuff like that as well in…
Based on this, they might not even realize they are downloading this stuff ...
Re: Apple downloads ~45 TB of models per day from our S3 bucket
#40"Almost everyone" working on NLP uses one of hugginface's pretrained models at one point or another, sooner or later: https://github.com/huggingface/pytorch-transformers
It's so damn convenient, and so nicely done.
And they keep doing neat things like this one: https://github.com/huggingface/swift-coreml-transformers
Kudos to Julien Chaumond et al for their work!