Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling up the Prime Video audio/video monitoring service and reducing costs

primevideotech.com

Post reply on HN
Print view View original

381–390 of 526 posts

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#381

Post by t_sawyer »

Hamuko wrote: post #160
Aeolun wrote: post #131

Earlier quoted context omitted.

I feel like it’s an object lesson in using the right solution for a problem. Step functions do not appear to me to be something that you’d use for things that need to be executed multiple times per second.

Yeah, this is my takeaway too. I'm pretty happy with the monolith that we run at our business and this seems to validate our decision to stick to that monolith, but I'm also pretty confident that where we use AWS Lambda, serverless is absolutely the right way to go. For example, I've written a Lambda application to reply to webhook calls and send API calls whenever those come in. It costs maybe $2 per month to run in…

But background tasks are a thing. You could add a webhook endpoint to your monolith that writes a background job. Then your background worker (running on the same ec2 because it’s hardware requirements look pretty low at $2 a month in lambda) runs the job. $2 a month is now $0 since it’s running next to your monolith on the VM you’re already paying for.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#382

Post by haspok »

ImPleadThe5th wrote: post #250
bawolff wrote: post #10

I'm pretty convinced that microservices are one of those things that make sense 5% of the time and the other 95% is cargo culting.

I really feel like microservices primarily solve people/team organizing problems more than it solves any computing problems.

It's called Conway's law.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#383

Post by re-thc »

Ensorceled wrote: post #339
manfre wrote: post #292

Earlier quoted context omitted.

I'm intentionally ignoring any of the sarcasm in your comment. The time needed for a db dump is always dependent upon the amount of data. This is true regardless of the db software or where it's running.

I can't think of any project I've worked on where the main data base could be backed up and restored to a different database in "2 minutes" The sarcasm was warranted.

With SSO multi-account and 2FA, it'd probably take longer than 2 minutes just to authenticate and configure AWS correctly to get started.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#384

Post by pjc50 »

henry_bone wrote: post #86
The_Colonel wrote: post #52

Earlier quoted context omitted.

I agree, my intuition would put it to 1% vs. 99% (difficult to quantify of course). I haven't yet seen a project/product which would need microservice architecture for technical reasons. If you need to scale, you can just scale monoliths (perhaps serving in different roles). The use case for microservice architecture is IMHO an organizational / high level architecture driven. I've worked in a big company (20K employe…

Seems like Conway's law applies:- "Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure." -- Melvin E. Conway https://en.wikipedia.org/wiki/Conway%27s_law

Came here to say this - and it applies in the other direction. Microservices allow you to split work between teams without having to coordinate deployment and iteration cadence quite so tightly.

If you have a single team, you shouldn't be doing microservices.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#385

Post by steveBK123 »

Hamuko wrote: post #160
Aeolun wrote: post #131

Earlier quoted context omitted.

I feel like it’s an object lesson in using the right solution for a problem. Step functions do not appear to me to be something that you’d use for things that need to be executed multiple times per second.

Yeah, this is my takeaway too. I'm pretty happy with the monolith that we run at our business and this seems to validate our decision to stick to that monolith, but I'm also pretty confident that where we use AWS Lambda, serverless is absolutely the right way to go. For example, I've written a Lambda application to reply to webhook calls and send API calls whenever those come in. It costs maybe $2 per month to run in…

It really depends on the quantity & scale though right? If in my entire estate I have a single shell script that I run - wow lambdas / serverless are amazing.

When I have 200 things that cost $5/mo each to run but fit nicely on a single 8core/32gb ram server.. then this lambda stuff starts to seem crazy expensive right?

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#386

Post by adql »

ikiris wrote: post #6

Sending video frames between services is expensive, also doing per state transition hosting on things doing state transitions multiple times per second in a single stream is also expensive... Like, did they even think about cost when designing this the first time?

>Like, did they even think about cost when designing this the first time?

Obviously no, only after managers complained.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#387

Post by cmrdporcupine »

cglong wrote: post #135
bawolff wrote: post #10

I'm pretty convinced that microservices are one of those things that make sense 5% of the time and the other 95% is cargo culting.

My team owns an API monolith that hosts several completely unrelated endpoints. I keep thinking this would be a good candidate for breaking into microservices, but I do wonder if I'm buying into the hype.

When you say unrelated, are you sure? Do they share -- or should they share -- a common underlying relational data model?

My biggest grief with microservices is the fact that it's effectively become a war on having a coherent logical normalized relational data model inside an organization.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#388

Post by awaythrow98765 »

jabradoodle wrote: post #359
awaythrow98765 wrote: post #289

Earlier quoted context omitted.

> If your business invests in physical servers anticipating strong growth next year then later finds out actually we're going into a recession and those servers are no longer needed, then that's a sunk cost. Cloud vendors also mostly sell minimum use packages for discounts in the range of 20 to 80% (called e.g. "committed use discount" or "compute savings plan"). Lots of businesses use those, because two-digit discou…

It's not really the same spot in that your paying monthly rather than upfront. Devs tend to think about total $, the business/accountants do care about Opex vs Capex. Also it's going to be simpler to provision your base (commited use) on the cloud and then handle bursts on the cloud, than it is to have your base on prem and burst to the cloud.

That is not really a difference, you may as well lease your server farm in the basement, practically the same cost as buying it, just as a monthly payment with the supposed "advantages" the business people might care about.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#389

Post by codedokode »

kypro wrote: post #282
BbzzbB wrote: post #194

Earlier quoted context omitted.

There was an article not long ago from AWS saying they'll be focussing on cutting cost for customers. Maybe the next step of that process will be pushing their clients off of AWS and telling them to just host on prem.

I know you're joking around, but no, as they also explained a benefit of cloud (and therefore using AWS) is that it can scale flexibly with their customers' businesses. If your business invests in physical servers anticipating strong growth next year then later finds out actually we're going into a recession and those servers are no longer needed, then that's a sunk cost. With cloud if demand drops you can scale up a…

You can sell the servers and partially return the money.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#390

Post by pnt12 »

boxed wrote: post #112
The_Colonel wrote: post #52

Earlier quoted context omitted.

I agree, my intuition would put it to 1% vs. 99% (difficult to quantify of course). I haven't yet seen a project/product which would need microservice architecture for technical reasons. If you need to scale, you can just scale monoliths (perhaps serving in different roles). The use case for microservice architecture is IMHO an organizational / high level architecture driven. I've worked in a big company (20K employe…

I built a little microservice on the side of my monolith for PDF creation. It used headless chrome and ghostscript to render html to a nice PDF. The problem I had with having that code inside the monolith was that it increased my docker image creation for deploys by a lot . And that code pretty much never changed anyway. I did feel a bit embarrassed having to make a microservice after having argued against them so mu…

I feel you - I worked on a project with pdf generation and the tool to generate it was quite heavy.
Post reply on HN