Live data from Hacker News

Scaling up the Prime Video audio/video monitoring service and reducing costs

primevideotech.com

51–60 of 526 posts

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#51
I don't want to come off too harsh on this, but it sounds like the service didn't meet the initial design requirements?

Some of this would have been really easy to predict (eg. hitting account limits) if they simply took the time to calculate how many workflow transitions they'd need to execute for the load.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#52
post #10

I'm pretty convinced that microservices are one of those things that make sense 5% of the time and the other 95% is cargo culting.

I agree, my intuition would put it to 1% vs. 99% (difficult to quantify of course).

I haven't yet seen a project/product which would need microservice architecture for technical reasons. If you need to scale, you can just scale monoliths (perhaps serving in different roles).

The use case for microservice architecture is IMHO an organizational / high level architecture driven. I've worked in a big company (20K employees) which was completely redesigning its back-office IT solution which ended up as a mesh of various microservices serving various needs (typically consumed by purpose built frontends), worked on by different teams. There monolith didn't make sense, because there was no single purpose, no single product.

But if I'm building a product, I will choose monolith every time. Maaaaybe in some very special cases, I will build some auxiliary services serving the monolith, but there needs to be a very good reason to do so.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#55
post #14
post #10

I'm pretty convinced that microservices are one of those things that make sense 5% of the time and the other 95% is cargo culting.

As for me,I have been trying to discover that 5% that cannot be done without microservices.

There's a bunch of things I'd like to have them do. If they could span across machines like clusters that would be amazing.

If I could trivially package them up and deploy them locally with intrinsically less effort and wall-time then the old way, that'd be amazing.

If I could somehow get the horizontal scaling promises and redundancy as some kind of built-in, like I can with say, memcache, that's be cool.

If I could do these kinds of "hard" things with them more trivially, that'd be really nice.

There's a lot of things I want them to do but it's a god-damn bull-riding rodeo every time I try to get there.

And before you reply, I know you're an expert and can do all these things trivially. That's amazing. The vast majority of the industry creates a giant fragile spaghetti knot with them and I am not a full time k8s admin nor do I want this to be a career trajectory. It should be like you know, wine, ffmpeg, imagemagick, virtualbox, lua, qemu, redis, gnuplot, lvm2, gdb, ssh, sqlite; tools like that. It's pretty easy to get them to do really nice things. Those things deliver on their promises and potential pretty nicely.

It's nice that nobody feels a need to hype curl or squid. They just work. Isn't that nice? I mean look at gdb's website: https://www.sourceware.org/gdb/ it doesn't even have CSS animations --- in fact, it doesn't even have CSS.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#57
The title is editorialised to be clickbait. The original title is "Scaling up the Prime Video audio/video monitoring service and reducing costs by 90%".

They changed a single service, the Prime Video audio/video monitoring service, from a few Lambda and Step Function components into a 'monolith'. This monolith is still one of presumably many services within Prime Video.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#59
post #26

I wouldn't call it a monolith as the number of instances could be scaled up. Mono implies single instance. They just combined multiple microservices into a larger one.

I am not sure if you’re joking.

We could call it a polylith.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#60
post #22
post #14

Earlier quoted context omitted.

As for me,I have been trying to discover that 5% that cannot be done without microservices.

Like any tool, there is nothing that cannot be done without microservices. However that doesn't mean they never make sense. Microservices have certain costs and certain benefits. I can believe there are certain situations where the benefits outweigh the costs. Its just not most situations. But that doesn't mean it never exists. I could believe it makes sense in extremely large apps with huge number of different group…

That's very hypothetical. Building a distributed system just might always be way more resource intensive than comparable monlith regardless of number of developers involved.
Post reply on HN