My word. I'm sort of gob smacked this article exists. I know there are nuances in the article, but my first impression was it's saying "we went back to basics and stopped using needless expensive AWS stuff that caused us to completely over architect our application and the results were much better". Which is good lesson, and a good story, but there's a kind of irony it's come from an internal Amazon team. As another…
Scaling up the Prime Video audio/video monitoring service and reducing costs
421–430 of 526 posts
Re: Scaling up the Prime Video audio/video monitoring service and reducing costs
#422AWS has a great business model of people over "optimizing" their architecture using new toys from amazon and being charged through the nose for it. It's amazing how clients that are doing a few requests per second will want a fully distributed, serverless, microservice + dynamodb + s3 + athena + etc + etc, in order to serve a semi-static web app and print some reports off throughout the day and pay 10-50k a month whe…
Re: Scaling up the Prime Video audio/video monitoring service and reducing costs
#423This is not a discussion of monolith vs serverless. This is some terrible engineering all over that was "fixed". Some excerpts: > This eliminated the need for the S3 bucket as the intermediate storage for video frames because our data transfer now happened in the memory. My candid reaction: Seriously? WTF? I am honestly surprised that someone thought it was a good idea to shuffle video frames over the wire to S3 and…
I've seen some staggering cost savings realized because someone happened to notice that an inefficient implementation that wasn't a problem two years ago at the scale it was running at back then did not age well to the 10x volume it was handling two years later. The reason it hadn't fallen over was that horizontal scaling features built into the cloud products were able to keep it running with minimal attention from the SRE's.
Re: Scaling up the Prime Video audio/video monitoring service and reducing costs
#424Earlier quoted context omitted.
Seems a bit of a dark pattern to have shortcut to onboard and not offer the same shortcut to offboard. Just like easy subscribe-online publications that will have you call during 2 hours with a rep pushing you discounts or whatever to cancel such subscription. Just not cool
I'd characterize this asymmetry (convenient shortcut IN, standard export OUT) as a predictable, transparent, minor annoyance -- not a "dark pattern" representing deceptive or unethical practices.
Just I'd try to stay away from amazon if I could
Re: Scaling up the Prime Video audio/video monitoring service and reducing costs
#425My word. I'm sort of gob smacked this article exists. I know there are nuances in the article, but my first impression was it's saying "we went back to basics and stopped using needless expensive AWS stuff that caused us to completely over architect our application and the results were much better". Which is good lesson, and a good story, but there's a kind of irony it's come from an internal Amazon team. As another…
Re: Scaling up the Prime Video audio/video monitoring service and reducing costs
#426Earlier quoted context omitted.
I'm intentionally ignoring any of the sarcasm in your comment. The time needed for a db dump is always dependent upon the amount of data. This is true regardless of the db software or where it's running.
I can't think of any project I've worked on where the main data base could be backed up and restored to a different database in "2 minutes" The sarcasm was warranted.
Re: Scaling up the Prime Video audio/video monitoring service and reducing costs
#427Earlier quoted context omitted.
Doesn't Docker use cached image layers to solve that? Your PDF rendering could be in one layer that never changes, and the rest goes on top.
The problem is when you have more than one such service. Now when one of them changes, all of them need to be rebuilt. You can solve this with multi-stage builds, but those only work if your build result can be easily copied.
Non-copyable build outputs sound a bit wild - you're thinking of builds that encode absolute paths into the output binaries?
Re: Scaling up the Prime Video audio/video monitoring service and reducing costs
#428AWS has a great business model of people over "optimizing" their architecture using new toys from amazon and being charged through the nose for it. It's amazing how clients that are doing a few requests per second will want a fully distributed, serverless, microservice + dynamodb + s3 + athena + etc + etc, in order to serve a semi-static web app and print some reports off throughout the day and pay 10-50k a month whe…
Re: Scaling up the Prime Video audio/video monitoring service and reducing costs
#429Breaking things into tiny functions and putting them on many different servers incurs tradeoff costs in both complexity and compute. There is a complexity cost in having to deal with the setup, security, and orchestration of those functions, and a compute cost because if the overall system is running constantly it will be less efficient and therefore more expensive than running on one box.
I agree on the tradeoffs you have to make. The main cost driver here was storage and traffic, though.