Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CDN vs S3

jdorfman.posthaven.com

Post reply on HN
Print view View original

61–70 of 87 posts

Re: CDN vs S3

#61

Post by [deleted] »

jcampbell1 wrote: post #58
jefe78 wrote: post #39

Earlier quoted context omitted.

You're doing something horribly wrong. I work for a live streaming company and we make extensive use of Varnish. It can probably solve the problem you're describing.

I'm not doing a damn thing wrong other than using Cloudfront. The problem is on their end, not mine. Thinking Varnish could solve this problem is utterly confused. Do you know what CDN does? CDNs have servers located around the world so files are loaded quickly and with low latency. Furthermore, your profile suggest you work for a pump and dump penny stock company (basically a scam). If your employer is paying you in…

[deleted]

Re: CDN vs S3

#62

Post by ballard »

jcampbell1 wrote: post #58
jefe78 wrote: post #39

Earlier quoted context omitted.

You're doing something horribly wrong. I work for a live streaming company and we make extensive use of Varnish. It can probably solve the problem you're describing.

I'm not doing a damn thing wrong other than using Cloudfront. The problem is on their end, not mine. Thinking Varnish could solve this problem is utterly confused. Do you know what CDN does? CDNs have servers located around the world so files are loaded quickly and with low latency. Furthermore, your profile suggest you work for a pump and dump penny stock company (basically a scam). If your employer is paying you in…

Sounds like troll bait, recall the recent article about PG's modding algorithms. Life's too short.

There's lots of video CDN "solutions", and it's almost always cheapest (even after labor support) to DIY with bare metal at very large scale. If it were me, I would eval video CDN shops using tsung test cases wired up as nagios checks. Gotta make sure their stuff stays working.

A payment gateway once mistakenly deployed API changes to production without notice. Trust no one.

Anyone evaluated? http://live.bittorrent.com

Re: CDN vs S3

#63

Post by tzury »

jere wrote: post #53

For everyone saying this is obvious: I'm one of those idiots that didn't quite get it and I appreciate this post a lot. For the past week, I've been working on writing a little static site generator and putting it on S3 and I thought I was pretty damn clever for finally getting around to doing that... except now it turns out I'm clueless yet again. I'm looking at Cloudfront now, but I'm still not sure if has all the…

You should simply put CloudFront in front of S3.

In that case, you get the benefit of both worlds.

Re: CDN vs S3

#64

Post by jcampbell1 »

[deleted] wrote: post #61
jcampbell1 wrote: post #58

Earlier quoted context omitted.

I'm not doing a damn thing wrong other than using Cloudfront. The problem is on their end, not mine. Thinking Varnish could solve this problem is utterly confused. Do you know what CDN does? CDNs have servers located around the world so files are loaded quickly and with low latency. Furthermore, your profile suggest you work for a pump and dump penny stock company (basically a scam). If your employer is paying you in…

[deleted]

I am sincerely not being spiteful. I am trying to warn you. Your company's unaudited financials state that there is only $7,000 in the bank.

http://www.sec.gov/Archives/edgar/data/1499274/0001096350130...

I sign my own paycheck, so the notion of whether he is overpaying is a bit confusing.

Re: CDN vs S3

#65

Post by tzury »

asb wrote: post #9
davidandgoliath wrote: post #2

I thought this was obvious? Amazon's Cloudfront on the other hand is a CDN and works great :)

If only Route53 allowed you to point the apex domain at Cloudfront (as I understand it, it's currently S3 or ELB).

You may not want to do so as:

  1) POST (and so are PUT, DELETE, OPTIONS and CONNECT) 
     are not yet supported on CloudFront.
  2) HTTPS/SSL for your own domain is not yet supported 
     on CloudFront.

Re: CDN vs S3

#66

Post by jefe78 »

jcampbell1 wrote: post #64
[deleted] wrote: post #61

Earlier quoted context omitted.

[deleted]

I am sincerely not being spiteful. I am trying to warn you. Your company's unaudited financials state that there is only $7,000 in the bank. http://www.sec.gov/Archives/edgar/data/1499274/0001096350130... I sign my own paycheck, so the notion of whether he is overpaying is a bit confusing.

I'm entirely aware of the financial situation. How is calling my employer a pump and dump scam NOT spiteful when I've worked on this project from the beginning?

Edit: Also, I don't appreciate you posting that. It's completely off-topic. Keep it classy.

Re: CDN vs S3

#67

Post by photorized »

ballard wrote: post #62
jcampbell1 wrote: post #58

Earlier quoted context omitted.

I'm not doing a damn thing wrong other than using Cloudfront. The problem is on their end, not mine. Thinking Varnish could solve this problem is utterly confused. Do you know what CDN does? CDNs have servers located around the world so files are loaded quickly and with low latency. Furthermore, your profile suggest you work for a pump and dump penny stock company (basically a scam). If your employer is paying you in…

Sounds like troll bait, recall the recent article about PG's modding algorithms. Life's too short. There's lots of video CDN "solutions", and it's almost always cheapest (even after labor support) to DIY with bare metal at very large scale. If it were me, I would eval video CDN shops using tsung test cases wired up as nagios checks. Gotta make sure their stuff stays working . A payment gateway once mistakenly deploye…

and it's almost always cheapest (even after labor support) to DIY with bare metal at very large scale

If you simply need to deliver files or live streams, without needing to provide complex functionality at the edge (various kinds of protection, geo blocking, or pay-per-minute), and your traffic patterns are predictable - it's often cheaper to build your own solution. Once you start thinking about backbone and colo redundancy, deploy in different countries with contract commits - things get expensive very quickly.

The beauty of using a massive third party delivery service isn't performance, it's elasticity. Just like with the web apps (frequently hosted on DIY systems) that go down as soon as the link goes up on HN - being able to absorb traffic spikes without failing (and without forcing you to commit to a higher tier for a year) can be very valuable.

Re: CDN vs S3

#68

Post by photorized »

ballard wrote: post #59
jefe78 wrote: post #39

Earlier quoted context omitted.

You're doing something horribly wrong. I work for a live streaming company and we make extensive use of Varnish. It can probably solve the problem you're describing.

Have you (or anyone else) had a chance to A/B test caches? That is, setup network API requests to be duplicated/filtered from production and sent to test environment(s).. credit to netflix. Setup enough identical boxes with each of Squid, Nginx, Varnish, trafficserver, etc. and evaluate each with basically the same traffic and however much tweaking.

Can also be done with logs - play back same request logs against several different setups, compare performance.

Re: CDN vs S3

#69

Post by aritraghosh007 »

jere wrote: post #53

For everyone saying this is obvious: I'm one of those idiots that didn't quite get it and I appreciate this post a lot. For the past week, I've been working on writing a little static site generator and putting it on S3 and I thought I was pretty damn clever for finally getting around to doing that... except now it turns out I'm clueless yet again. I'm looking at Cloudfront now, but I'm still not sure if has all the…

You should really be using CloudFront and "NOT" S3 if you are looking for a CDN solution. There is a reason why CloudFront exists after all. S3 is a merely a drop-in storage option, consider it as a big file system residing within big fat piped data centers.

Re: CDN vs S3

#70

Post by photorized »

ballard wrote: post #59
jefe78 wrote: post #39

Earlier quoted context omitted.

You're doing something horribly wrong. I work for a live streaming company and we make extensive use of Varnish. It can probably solve the problem you're describing.

Have you (or anyone else) had a chance to A/B test caches? That is, setup network API requests to be duplicated/filtered from production and sent to test environment(s).. credit to netflix. Setup enough identical boxes with each of Squid, Nginx, Varnish, trafficserver, etc. and evaluate each with basically the same traffic and however much tweaking.

Let's keep in mind that individual box performance will not directly translate into your cluster performance or global network performance. A box can be fine-tuned to serve a file at lightning speed, but once you connect a bunch of them together, and start delivering lots of different files to millions of people - different factors come into play. Distributing files, replacing files when updated, content churn, etc etc

Simple caching works for images, but doesn't work for large video files, for example (look at latest financials from public CDNs - they are all bleeding cash).

It's really not that simple as testing a box to see which setup works best.

Post reply on HN