Some of this would have been really easy to predict (eg. hitting account limits) if they simply took the time to calculate how many workflow transitions they'd need to execute for the load.
Scaling up the Prime Video audio/video monitoring service and reducing costs
51–60 of 526 posts
Re: Scaling up the Prime Video audio/video monitoring service and reducing costs
#52I'm pretty convinced that microservices are one of those things that make sense 5% of the time and the other 95% is cargo culting.
I haven't yet seen a project/product which would need microservice architecture for technical reasons. If you need to scale, you can just scale monoliths (perhaps serving in different roles).
The use case for microservice architecture is IMHO an organizational / high level architecture driven. I've worked in a big company (20K employees) which was completely redesigning its back-office IT solution which ended up as a mesh of various microservices serving various needs (typically consumed by purpose built frontends), worked on by different teams. There monolith didn't make sense, because there was no single purpose, no single product.
But if I'm building a product, I will choose monolith every time. Maaaaybe in some very special cases, I will build some auxiliary services serving the monolith, but there needs to be a very good reason to do so.
Re: Scaling up the Prime Video audio/video monitoring service and reducing costs
#53Storing individual frames in S3??? Insanity! Their initial distributed architecture is unbelievable.
Re: Scaling up the Prime Video audio/video monitoring service and reducing costs
#54I wouldn't call it a monolith as the number of instances could be scaled up. Mono implies single instance. They just combined multiple microservices into a larger one.
Re: Scaling up the Prime Video audio/video monitoring service and reducing costs
#55I'm pretty convinced that microservices are one of those things that make sense 5% of the time and the other 95% is cargo culting.
As for me,I have been trying to discover that 5% that cannot be done without microservices.
If I could trivially package them up and deploy them locally with intrinsically less effort and wall-time then the old way, that'd be amazing.
If I could somehow get the horizontal scaling promises and redundancy as some kind of built-in, like I can with say, memcache, that's be cool.
If I could do these kinds of "hard" things with them more trivially, that'd be really nice.
There's a lot of things I want them to do but it's a god-damn bull-riding rodeo every time I try to get there.
And before you reply, I know you're an expert and can do all these things trivially. That's amazing. The vast majority of the industry creates a giant fragile spaghetti knot with them and I am not a full time k8s admin nor do I want this to be a career trajectory. It should be like you know, wine, ffmpeg, imagemagick, virtualbox, lua, qemu, redis, gnuplot, lvm2, gdb, ssh, sqlite; tools like that. It's pretty easy to get them to do really nice things. Those things deliver on their promises and potential pretty nicely.
It's nice that nobody feels a need to hype curl or squid. They just work. Isn't that nice? I mean look at gdb's website: https://www.sourceware.org/gdb/ it doesn't even have CSS animations --- in fact, it doesn't even have CSS.
Re: Scaling up the Prime Video audio/video monitoring service and reducing costs
#56Next they will transition to on premises hardware from the cloud to save another 90%.... oh wait...
Re: Scaling up the Prime Video audio/video monitoring service and reducing costs
#57They changed a single service, the Prime Video audio/video monitoring service, from a few Lambda and Step Function components into a 'monolith'. This monolith is still one of presumably many services within Prime Video.
Re: Scaling up the Prime Video audio/video monitoring service and reducing costs
#58Everything old is new again.
Re: Scaling up the Prime Video audio/video monitoring service and reducing costs
#59Re: Scaling up the Prime Video audio/video monitoring service and reducing costs
#60Earlier quoted context omitted.
As for me,I have been trying to discover that 5% that cannot be done without microservices.
Like any tool, there is nothing that cannot be done without microservices. However that doesn't mean they never make sense. Microservices have certain costs and certain benefits. I can believe there are certain situations where the benefits outweigh the costs. Its just not most situations. But that doesn't mean it never exists. I could believe it makes sense in extremely large apps with huge number of different group…