Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Prime Video service dumps microservices, cuts AWS bill 90%

thestack.technology

Post reply on HN
Print view View original

31–40 of 42 posts

Re: Prime Video service dumps microservices, cuts AWS bill 90%

#31

Post by janosdebugs »

marcus_holmes wrote: post #30
janosdebugs wrote: post #28

Earlier quoted context omitted.

Funnily enough, both things can be true depending on the context. Microservices are easier to debug: sure, if you compare a single monolithic binary with no logs and trace points to a microservice system that runs on top of a fancy trace the heck out of everything service mesh. Logging is still hard. Deployment time: the same reason why people wouldn't compile dependencies statically. It's easier to replace a .dll or…

I think part of the problem is that people who are wildly enthusiastic about microservices haven't experienced DLL Hell and don't understand the problems that can happen.

No argument here, I personally don't think microservies are the solution to code organization problems. However, many ecosystems don't have a good structure that lets less experienced coders work efficiently on large-scale systems (NodeJS, Go, etc.)

Re: Prime Video service dumps microservices, cuts AWS bill 90%

#32

Post by that_guy_iain »

janosdebugs wrote: post #28
that_guy_iain wrote: post #23

I ended up having a weird debate on Reddit over this. For some reason people seem to think that there is a single better choice and that if you ever go from a monolith to microservices that the monolith failed or whatever. One person even made the claim that microservices are easier to maintain and debug even tho a single code base that you can use tools on and easily keep in sync seems miles easier to maintain than…

Funnily enough, both things can be true depending on the context. Microservices are easier to debug: sure, if you compare a single monolithic binary with no logs and trace points to a microservice system that runs on top of a fancy trace the heck out of everything service mesh. Logging is still hard. Deployment time: the same reason why people wouldn't compile dependencies statically. It's easier to replace a .dll or…

> Deployment time: the same reason why people wouldn't compile dependencies statically. It's easier to replace a .dll or .so than the entire application, less bandwith. However, mess up the versions and you end up having a really bad day. Same with microservices: sure, a tiny service is easy to deploy, but heaven forbid you deploy incompatible versions.

That argument I could understand. But their argument turned out to be that they thought a monolith meant everyone has to spend weeks between deployments.

So it wasn't how long it took do the deployment but how often they could do deployments. Which is entirely a process matter. With microservices in my experience it can turn in to a mess of we need to sync our deployments because they built a distributed monolith.

To be honest, most of their arguments seemed to be they had tons of technical debt and blamed that it was a monolith for the technical debt existing than the fact it's technical debt.

> Microservices give people who don't have an architectural view/experience in building complex systems from scratch a framework to put their stuff in without thinking much about it. Good luck getting the same level of organization in a monolithic NodeJS application. It's hard, even seasoned veterans often get the abstraction layers wrong.

If you don't have the discipline to keep things organised in a monolith how are you going to manage with microservices? Especially, when there are literally tools to help you do this in monoliths. This is how people end up with distributed monoliths.

Is building a high quality monolith hard? Yes. Is it harder than building a high quality microservices system, I would say no.

Re: Prime Video service dumps microservices, cuts AWS bill 90%

#33

Post by tomjen3 »

xiwenc wrote: post #20
dharmab wrote: post #2

Article's headline and text mention this was achieved by switching from Lambda to ECS, which makes a lot of sense. Lambda is expensive for frequently used services.

If i recall one of the biggest selling point of lambda (serverless) was the “infinite” scale. Sounds like the “cash cow” for cloud service providers when used in efficiently/inappropriately. Besides the lower maintenance efforts.

Thats true. I think the example from AWS is some once in a quarter data ingestion being much, much cheaper on lambda, and of course if you have your own build machines then you are limited to how often you can build without having people waiting.

But if you want low maintenance efforts, then you can get managed Kubernetes.

Re: Prime Video service dumps microservices, cuts AWS bill 90%

#34

Post by williamcotton »

acd wrote: post #7

Monoliths have local CPU L1-Cache speeds, distributed microservices have JSON desearlization,searlization, network round trips. Distributed stateless monoliths can be faster than microservice. Carbon foot print, cloud bill of too many microservices is probably high.

Carbon foot print vs “engineering time costs more than computing time”

Re: Prime Video service dumps microservices, cuts AWS bill 90%

#35

Post by traspler »

emodendroket wrote: post #25
zamnos wrote: post #22

Earlier quoted context omitted.

Just serve and not transcode? Because it sounds like the Amazon Prime microservice was doing some heavy processing of uploaded video - breaking it into frames and applying some ML algorithms to them, and also the same with the audio. Sounds like even if you can fit in under the 10 min lamda execution timeout (assuming internal teams are limited to the same), that AWS Lambda/Step Functions isn't a generic job queue ha…

Writing every frame to S3 and reading them back in other steps was probably a major factor.

There is not much of an option with step functions, outputs/inputs can be max 256KB so especially if you go into or out of a map/parallel state you can‘t do much more than pass references to the data and not the data itself.

Re: Prime Video service dumps microservices, cuts AWS bill 90%

#36

Post by AbbeSomething »

emodendroket wrote: post #26
AbbeSomething wrote: post #14

CPU’s are mind boggling fast compared to data transfer. There are exceptions but for the vast majority of workloads reducing data transfers is the easiest optimization. But scale is the obvious key component, having CPU’s allocated to doing mostly nothing is such a waste.

Well, except that's not necessarily an "easy" optimization. If you go all-out with the highly managed stuff it's kind of like plugging together very simple Legos.

Agree, east was a poor choice of word. Perhaps powerful would be better because frequently it can be difficult to attain. But it is difficult to pick good nuances for broad generalizations. There are many other attractive aspects of micro services and many cases for which they are perfectly well suited.

Re: Prime Video service dumps microservices, cuts AWS bill 90%

#37

Post by bdcravens »

ChicagoDave wrote: post #16

As I recall, https://icloudguru.com/ used serverless to serve their video lessons and said they were paying pennies. I wonder what they're doing differently. I tried searching for the original article, but it's too ambiguous with their general offerings, so no luck.

Serverless isn't "cheap". It's rightsized. If you aren't serving 24/7, it makes sense to move to an on-demand model. However if you compare the unit of compute pricing, Lambda is comparable to other services, and likely more expensive than buying compute wholesale. You can just buy it in smaller portions.

Re: Prime Video service dumps microservices, cuts AWS bill 90%

#38

Post by otabdeveloper4 »

williamcotton wrote: post #34
acd wrote: post #7

Monoliths have local CPU L1-Cache speeds, distributed microservices have JSON desearlization,searlization, network round trips. Distributed stateless monoliths can be faster than microservice. Carbon foot print, cloud bill of too many microservices is probably high.

Carbon foot print vs “engineering time costs more than computing time”

Microservices never decrease engineering time costs.

Project manager overhead - maybe, but ideally that should be zero anyways.

Re: Prime Video service dumps microservices, cuts AWS bill 90%

#39

Post by nasmorn »

tomjen3 wrote: post #33
xiwenc wrote: post #20

Earlier quoted context omitted.

If i recall one of the biggest selling point of lambda (serverless) was the “infinite” scale. Sounds like the “cash cow” for cloud service providers when used in efficiently/inappropriately. Besides the lower maintenance efforts.

Thats true. I think the example from AWS is some once in a quarter data ingestion being much, much cheaper on lambda, and of course if you have your own build machines then you are limited to how often you can build without having people waiting. But if you want low maintenance efforts, then you can get managed Kubernetes.

Sorry but managed Kubernetes is not low maintenance like Faas. I had a kubernetes upgrade fail and it was not fun to stand up a new cluster install newer versions of the ingress that could not longer be upgraded, migrate all the workloads. The k8s cluster is still very much your problem

Re: Prime Video service dumps microservices, cuts AWS bill 90%

#40

Post by emodendroket »

traspler wrote: post #35
emodendroket wrote: post #25

Earlier quoted context omitted.

Writing every frame to S3 and reading them back in other steps was probably a major factor.

There is not much of an option with step functions, outputs/inputs can be max 256KB so especially if you go into or out of a map/parallel state you can‘t do much more than pass references to the data and not the data itself.

Sure but I guess I mean the nature of that task probably made them the wrong architectural choice.
Post reply on HN