Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

iOS 6.0 Causes CDN Overages

labs.prx.org

Post reply on HN
Print view View original

61–70 of 96 posts

Re: iOS 6.0 Causes CDN Overages

#61

Post by donavanm »

blinkingled wrote: post #37
chc wrote: post #29

Earlier quoted context omitted.

That doesn't seem practical. How would they know if it's the same client? Just watching for multiple requests from the same IP will result in a lot false positives (e.g. most college students will be SOL). And what would they do if they could detect it but the client didn't realize it already had the file? Just drop the request?

Load balancers can insert cookies to identify clients uniquely. (That solves the proxy/single-ip-multiple-clients problem - multiple clients from one proxy IP can still be uniquely tracked this way.) The newer load balancers (F5/iRules for e.g.) are also capable of doing per resource, per client throttling using the inserted cookies or session table.

Server side state per client is insane. Pushing it client side would be what, a kb? Wih an average object size of 50kb that's 2%. I doubt CDN customers would be willing to take a 2% cost hit for this feature.

Now, assume we solve all that. The state is pushed on the client. The performance hit is so small as to be free. And the CDN eats the dev and opex. So what does the CDN do when "bad" range requests are detected? Throw a 400? What customer is willing to break all IOS 6 clients?

Also, iRules. Serious LOLs.

Re: iOS 6.0 Causes CDN Overages

#62

Post by danso »

scrumper wrote: post #53
danso wrote: post #22

Earlier quoted context omitted.

I dunno, how crummy the App Store has become is a perfect example of this. The App Store is one prominent way they make loads of money and build loyalty to the ecosystem, and yet it is a shitshow on IOS6...the responsiveness is as bad as I've seen on non-iOS software, but for me, the two really inexplicable failures are: 1. the use of modals to show individual apps on the iPad. So if you get to an app page and accide…

Your Sony: is it an NEX-7? There's a firmware update just out which fixes the video button problem: http://blog.sony.com/alphafirmwareupdates

You have made my day...that blasted button is so clearly in the wrong spot that every time I bump it I secretly hope the Sony engineer responsible has been fired

Re: iOS 6.0 Causes CDN Overages

#63

Post by blinkingled »

donavanm wrote: post #61
blinkingled wrote: post #37

Earlier quoted context omitted.

Load balancers can insert cookies to identify clients uniquely. (That solves the proxy/single-ip-multiple-clients problem - multiple clients from one proxy IP can still be uniquely tracked this way.) The newer load balancers (F5/iRules for e.g.) are also capable of doing per resource, per client throttling using the inserted cookies or session table.

Server side state per client is insane. Pushing it client side would be what, a kb? Wih an average object size of 50kb that's 2%. I doubt CDN customers would be willing to take a 2% cost hit for this feature. Now, assume we solve all that. The state is pushed on the client. The performance hit is so small as to be free. And the CDN eats the dev and opex. So what does the CDN do when "bad" range requests are detected?…

For starters, average podcast download is 40Mb or so. More than 50Kb in any case. Next maintaining state per client isn't anything new - it might not be ideal for CDNs but many LBs keep state for things such as maintaining session persistence etc.

If there is a potential for clients intentionally or unintentionally jacking up your CDN bill - I am sure somebody has a solution to prevent it - especially since money is at stake.

You laughed at iRules - I got that part, but what about iRules doing HTTP Request throttling? Far as I know it can do that based on source ip and port, uri etc. which should work for this kind of scenario with some modifications.

Re: iOS 6.0 Causes CDN Overages

#64

Post by untog »

michael_miller wrote: post #58
untog wrote: post #31

This is where Apple's closed nature is very annoying. Assuming the article is correct, it was bug introduced in 6.0, and fixed in 6.0.1. So Apple knew about it, investigated it and published a fix. Yet they didn't tell anyone about it. Or the crippling effect this bug could have on their data allowances. Why not? Are they really that terrified of admitting to making a mistake?

To play devil's advocate, why would it matter? Is there any fix app developers could implement, short of blocking requests from iOS 6 users?

It would at least communicate to users that the upgrade to 6.0.1 is very important if they use the Podcasts app. Many may have put off installing the update because they thought their phone was working fine. A bandwidth eating bug has to be along the least visible.. in the short term.

Re: iOS 6.0 Causes CDN Overages

#65

Post by pretoriusB »

robomartin wrote: post #35

I have never considered Apple to be a solid software company. They are a marketing company in the tech sector. If they made washing machines they'd be sexy as hell. In my years of exposure to Apple tech I have always been in awe of their industrial design, mechanical design and manufacturing prowess. Their software, however, often leaves much to be desired. I say this both as a developer and a user. From OSX to iOS t…

>I have never considered Apple to be a solid software company.

What does that even mean? That they make bad software? For one, they make the most popular certified UNIX OS in the world, and one of the two most popular mobile OSs. Their professional offerings are great too: Logic Pro, Aperture, Final Cut Pro, etc. In fact, FCPX aside, they are professional staples, with few competitors.

And huge number of desktop and the majority of mobile users use a browser they developed to the best game in town starting from humble open source origins (khtml -> webkit).

And that Clang thing, that FreeBSD recently adopted? Their work too, along with other LLVM infrastructure.

>Their software, however, often leaves much to be desired. I say this both as a developer and a user.

Care to mention any substantially better mobile OS than iOS?

(One might argue that Android is better. But substantially better, no way in hell).

Re: iOS 6.0 Causes CDN Overages

#66

Post by pretoriusB »

zizee wrote: post #57
robomartin wrote: post #35

I have never considered Apple to be a solid software company. They are a marketing company in the tech sector. If they made washing machines they'd be sexy as hell. In my years of exposure to Apple tech I have always been in awe of their industrial design, mechanical design and manufacturing prowess. Their software, however, often leaves much to be desired. I say this both as a developer and a user. From OSX to iOS t…

addlepate wrote 37 minutes ago | link [dead] OK. I bet you use a WebKit browser. addlepate: you were hell-banned 100 days ago (unfairly in my mind).

Here's an easy clue: if your comments never get above or below 1 point for days, that means you are hell-banned.

Re: iOS 6.0 Causes CDN Overages

#67

Post by pretoriusB »

dageshi wrote: post #9

Is it just me or are apple making more dumb mistakes lately? There's this and that ajax caching issue from a month or two ago. Perhaps I'm just having selective memory?

>Is it just me or are apple making more dumb mistakes lately? There's this and that ajax caching issue from a month or two ago. Perhaps I'm just having selective memory?

No, it's not just you. It's the whole tech pop culture sector that has selective memory.

Apple did as much mistakes on the old times. It's just that it's not the underdog anymore, and all the hip kids are looking to fry Apple for those, instead of understanding that, heck, mistakes happen in every software/hardware offering.

The iPhone 4 antenna issue? Not even a blip compared to the iBook G3 logic board fiasco. But everybody forgets those things and adds it to some all-encompassing theory of decline...

Same with the notion of the "non-innovative update" for iPhone 5, etc.

The forgotten that less than a decade ago Apple did even far more minor incremental updates to the iPod line (a new click wheel, some more storage, a little thinner, now with a color screen, now it plays video, etc) and everybody was overwhelmed everytime a new device came out.

Suddenly they expect miracles out of every Keynote.

Re: iOS 6.0 Causes CDN Overages

#68

Post by devcpp »

pretoriusB wrote: post #65
robomartin wrote: post #35

I have never considered Apple to be a solid software company. They are a marketing company in the tech sector. If they made washing machines they'd be sexy as hell. In my years of exposure to Apple tech I have always been in awe of their industrial design, mechanical design and manufacturing prowess. Their software, however, often leaves much to be desired. I say this both as a developer and a user. From OSX to iOS t…

> I have never considered Apple to be a solid software company. What does that even mean? That they make bad software? For one, they make the most popular certified UNIX OS in the world, and one of the two most popular mobile OSs. Their professional offerings are great too: Logic Pro, Aperture, Final Cut Pro, etc. In fact, FCPX aside, they are professional staples, with few competitors. And huge number of desktop and…

Popularity means good software? Windows must be the best OS by very far then!

Regarding mobile OS, we all know this now depends on the number of "apps" available. Many may have made a superior OS, but adoption has been too low because of the lack of apps.

Re: iOS 6.0 Causes CDN Overages

#69

Post by wildranter »

devcpp wrote: post #68
pretoriusB wrote: post #65

Earlier quoted context omitted.

> I have never considered Apple to be a solid software company. What does that even mean? That they make bad software? For one, they make the most popular certified UNIX OS in the world, and one of the two most popular mobile OSs. Their professional offerings are great too: Logic Pro, Aperture, Final Cut Pro, etc. In fact, FCPX aside, they are professional staples, with few competitors. And huge number of desktop and…

Popularity means good software? Windows must be the best OS by very far then! Regarding mobile OS, we all know this now depends on the number of "apps" available. Many may have made a superior OS, but adoption has been too low because of the lack of apps.

No, it doesn't. I think what you really meant popularity of a mobile OS depends on the number of apps available. In reality, a good OS really depends on the quality of its API. Code you can rely on to build great apps.

And, what do you mean by many have made superior OS? You meant Blackberry OS 10? Nah, it's vaporware. Or what you really meant WebOS? Nah, it's fartware. Nokiat attempts with weird funny names? Windows Mobile 8? Have you heard of DLL hell or whatever flavor of bad API Redmond throws out the windows these days?

We really have two mobile OS, Android and iOS. They both have good APIs, and also some dirty little bugs inherent of any new code base. And we have to deal with it.

Re: iOS 6.0 Causes CDN Overages

#70

Post by donavanm »

blinkingled wrote: post #63
donavanm wrote: post #61

Earlier quoted context omitted.

Server side state per client is insane. Pushing it client side would be what, a kb? Wih an average object size of 50kb that's 2%. I doubt CDN customers would be willing to take a 2% cost hit for this feature. Now, assume we solve all that. The state is pushed on the client. The performance hit is so small as to be free. And the CDN eats the dev and opex. So what does the CDN do when "bad" range requests are detected?…

For starters, average podcast download is 40Mb or so. More than 50Kb in any case. Next maintaining state per client isn't anything new - it might not be ideal for CDNs but many LBs keep state for things such as maintaining session persistence etc. If there is a potential for clients intentionally or unintentionally jacking up your CDN bill - I am sure somebody has a solution to prevent it - especially since money is…

Your average podcast source material may be 40mb. Clients will most assuredly not be fetching that all in single request. Your state transfer overhead is per request. The 50kb estimate is an educated guess at the average size of an http response from a CDN. You have to keep state on every request, otherwise how can you detect your "bad request" patterns. Now, you could only stick your state on to Range'd requests. But a lot (most?) of the customers affected by this are doing a significant portion of requests as ranges already.

IRules, specifically, don't work for any reasonable packet rate. Every request must now come off the nic, across the bus, hit CPU, hit a few times memory, and back. Tracking state, in general, kills packet rates. There's no guarantee that your flow is going to be hitting the same interface, interrupt, processor, or even host. At every one of those levels shared state dramatically increases complexity and reduces your max possible packet rate. Silicon really can't flip bits that quickly.

Which gets us back to the business question. So you've found a "bad" client. What aceptable action can you take? Throttle all iOS 6 users? Throw 400s?

To be honest here this excess cost is going to be absorbed in three places. 1) end users will suck it up in data charges because they have no alternative 2) sites will eat the bandwidth charges. They can't passit on if they have no directly associated revenue. Or they don't want to lose customers. 3) CDN/providers will take a relatively small hit issuing credits to keep their customers happy.

Notice who won't lose a cent here? Apple and other broken client providers.

Post reply on HN