Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple downloads ~45 TB of models per day from our S3 bucket

twitter.com

Post reply on HN
Print view View original

91–100 of 237 posts

Re: Apple downloads ~45 TB of models per day from our S3 bucket

#91

Post by lvh »

the8472 wrote: post #47
joatmon-snoo wrote: post #32

Earlier quoted context omitted.

Also a great way to throw out massive red flags to any enterprise user that cares about privacy and non-disclosure. IP address data is pretty sensitive information, and throwing it out there like this, even in aggregate, is not OK because of what it shows. No matter how much PR this gets, this goes both ways.

They're not showing IP addresses, they're showing /8 CIDR blocks, that's very coarse granularity.

The real information is that Apple is downloading them. Why would they care about granularity?

Re: Apple downloads ~45 TB of models per day from our S3 bucket

#94

Post by vonseel »

flotwig wrote: post #74
brodie78382 wrote: post #67

Earlier quoted context omitted.

I used to work for HP and someone explained to me a select few companies got /8's when the internet was still young. HP got one, Compaq had one which HP now also owns. I was basically told if you had a /8 you didn't give it up because of how valuable and rare they now are (this was around 2010, too). GE, Kodak, Apple, and Microsoft were a few other names that came up in that discussion as well.

GE actually sold their entire 3.0.0.0/8 block off to AWS a few years ago. It's a little awkward since a lot of internal software is still configured to whitelist all access from that space since it was a constant for so long.

I wonder what became of the Kodak block? Seems GE would find more use/investment for IP space than a failed film company (IIRC)...

Re: Apple downloads ~45 TB of models per day from our S3 bucket

#95

Post by vonseel »

ethbro wrote: post #79
brodie78382 wrote: post #67

Earlier quoted context omitted.

I used to work for HP and someone explained to me a select few companies got /8's when the internet was still young. HP got one, Compaq had one which HP now also owns. I was basically told if you had a /8 you didn't give it up because of how valuable and rare they now are (this was around 2010, too). GE, Kodak, Apple, and Microsoft were a few other names that came up in that discussion as well.

Map: https://www.caida.org/research/id-consumption/census-map/ima... (a bit dated, but shows the historic allocations) Look up "CIDR", history thereof, for reasons why it looks this way.

Wow, this is very cool! Thank you.

Re: Apple downloads ~45 TB of models per day from our S3 bucket

#96

Post by heyoni »

qes wrote: post #80
paxys wrote: post #51

That's about $4000/month in bandwidth costs, assuming retail pricing. FYI he is bragging, not complaining. There are a dozen ways to reduce or eliminate this problem.

> That's about $4000/month in bandwidth costs You're an order of magnitude off. 45 TB per day is 1,350 TB in a month, or 1,350,000 GB. Show me somewhere you can get a petabyte of egress inside a calendar month for 4 figures USD... Let's suppose you even used the cheaper egress from Cloudfront rather than serving from S3 (lol @ your wallet if you serve 1 PB doing that). https://aws.amazon.com/blogs/aws/aws-data-transf…

How the hell are they paying that bill then??

Re: Apple downloads ~45 TB of models per day from our S3 bucket

#97

Post by baroffoos »

CobrastanJorji wrote: post #56

If you host large, publicly available data in a cloud blob service, but you don't have a budget for it, one option is to use the "Requester Pays" feature that Amazon and Google provide. This makes the data available to anyone to download, but they need to pay the download cost themselves. This is at the tradeoff of making your data significantly more irritating to access, as it's no longer just plugging in a URL into…

Or just post a magnet link.

Re: Apple downloads ~45 TB of models per day from our S3 bucket

#98

Post by semicolon_storm »

lvh wrote: post #91
the8472 wrote: post #47

Earlier quoted context omitted.

They're not showing IP addresses, they're showing /8 CIDR blocks, that's very coarse granularity.

The real information is that Apple is downloading them. Why would they care about granularity?

If Apple didn't want that information to be public, they probably shouldn't have downloaded 45TB of data a day from a service they have no paid agreement with.

Re: Apple downloads ~45 TB of models per day from our S3 bucket

#99

Post by vxNsr »

qes wrote: post #78
nathantotten wrote: post #58

Earlier quoted context omitted.

Not to mention that if Cloudflare CDN was in front of it this traffic would be free.

Not with this amount of traffic it won't be free. 45 TB per day lol Cloudflare will be disabling your account and in contact for payment in a hurry. Go ahead and try, see how far their "free" tier really goes.

Just waiting for the Cloudflare CEO who lurks around to pop in here and offer it for free.

Re: Apple downloads ~45 TB of models per day from our S3 bucket

#100

Post by lvh »

semicolon_storm wrote: post #98
lvh wrote: post #91

Earlier quoted context omitted.

The real information is that Apple is downloading them. Why would they care about granularity?

If Apple didn't want that information to be public, they probably shouldn't have downloaded 45TB of data a day from a service they have no paid agreement with.

I don’t disagree, but that’s not responsive to my (or GPs) point. GP implied the aggregate is what made it OK: if anything that made it more sensitive (not everyone would bother to aggregate and look up who owns the IPs).
Post reply on HN