Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling up the Prime Video audio/video monitoring service and reducing costs

primevideotech.com

Post reply on HN
Print view View original

301–310 of 526 posts

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#301

Post by [deleted] »

kerkeslager wrote: post #242
Paul-Craft wrote: post #214

Earlier quoted context omitted.

Nope. AWS makes it dead simple to move from RDS to Aurora by clicking a button. There's no way to move data from Aurora to RDS short of doing a SQL dump and reloading everything that way. I found this out when my previous employer was looking at moving from RDS to Aurora.

> > Aurora is I think pretty simple to move away from, since it's just fully compatible Postgres or Mysql. We even use a local postgres for development purposes against an Aurora solution. > Nope. AWS makes it dead simple to move from RDS to Aurora by clicking a button. There's no way to move data from Aurora to RDS short of doing a SQL dump and reloading everything that way. I found this out when my previous employe…

[deleted]

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#302

Post by dmw_ng »

lastangryman wrote: post #68

My word. I'm sort of gob smacked this article exists. I know there are nuances in the article, but my first impression was it's saying "we went back to basics and stopped using needless expensive AWS stuff that caused us to completely over architect our application and the results were much better". Which is good lesson, and a good story, but there's a kind of irony it's come from an internal Amazon team. As another…

The smoking gun is probably the box that was previously labelled "Media Conversion Service" (Elemental MediaConvert - easily 5-6 figures/mo. for a small amount of snappy on-demand capacity, or crippled slow-as-molasses reserved queues) now labelled "Media Converter" running on ECS. For example, vt1 instances are At some point the industry will wake up to the fact the AWS pricing pages are the real API docs, meanwhile dumb shit like this will keep happening over and over again, and AWS absolutely are not to blame for it, any more than e.g. a vendor of cabling is guilty of burning down the house of someone who plugged 10 electric heaters into a chain of double-gang power extension cords

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#304

Post by adql »

nomilk wrote: post #195
YetAnotherNick wrote: post #114

Earlier quoted context omitted.

Microservices works better if you don't trust other team. While having trust seem like a basic thing, this is absolutely not the case for a lot of companies. With microservices, it is easy to see services which are down or have high error rate or latency, have clear API contract and call out the team for breaking API contract, and assign cost for which the teams have incentive to reduce, or at least not increase it.

I haven't worked on a microservice architecture yet, but this is a very interesting idea that I hadn't heard before. That micro services can potentially give greater visibility of each team's performance, improving accountability.

It's way too easy to just add a lot of friction if you go too far.

I also think it would be way healthier if teams acted as "maintainers" rather than "sole developer" of a service.

For example if team A wants feature from service team B manages they should be free (after communicating that so there is no confict/work duplication) to just make that feature and submit pull request to the team B.

Then team B can make sure it's up to their standard but that's shorter work than getting the whole machine of "submit a ticket for team B to add feature, find manpower to do it, and schedule work" running.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#305

Post by adql »

datadeft wrote: post #287
bawolff wrote: post #10

I'm pretty convinced that microservices are one of those things that make sense 5% of the time and the other 95% is cargo culting.

Microservices make sense for a lot more than 5%. If fact I think it is much closer to the 80/20, 80% working on serverless, 20% not. Video streaming obviously not going to work on AWS Lambda to begin with.

They really don't unless you routinely put a bunch of tiny self-contained apps.

The "pain" of figuring out how to deploy your "normal app" quickly amortizes over just how much easier and more reliable code is.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#307

Post by yakshaving_jgt »

kreco wrote: post #197

At this point isn't the lesson to use serverless stack for fast iterative processes then use a custom solution once you know exactly what you want? I have 0 experience with serverless/cloud. Just a thought.

I think the lesson ought to be that you should start by writing one computer program and running it on one computer.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#308

Post by herculity275 »

kstenerud wrote: post #94
jjevanoorschot wrote: post #57

The title is editorialised to be clickbait. The original title is "Scaling up the Prime Video audio/video monitoring service and reducing costs by 90%". They changed a single service, the Prime Video audio/video monitoring service, from a few Lambda and Step Function components into a 'monolith'. This monolith is still one of presumably many services within Prime Video.

The subtitle is "The move from a distributed microservices architecture to a monolith application helped achieve higher scale, resilience, and reduce costs." And the article itself mentions the 90% cost reduction. So the title seems pretty much in-line with the original intent.

Prime Video has hundreds of teams, VQA is a tiny team that owns a very specific QA service. Omitting that distinction from the title absolutely is clickbait.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#309

Post by abluecloud »

boredumb wrote: post #283

AWS has a great business model of people over "optimizing" their architecture using new toys from amazon and being charged through the nose for it. It's amazing how clients that are doing a few requests per second will want a fully distributed, serverless, microservice + dynamodb + s3 + athena + etc + etc, in order to serve a semi-static web app and print some reports off throughout the day and pay 10-50k a month whe…

It's honestly like a cult and a desire to want to "do it right" on AWS. The last few projects I've spent so much time setting up code deploy, load balancers, certificates, SES, route 53... This newest project, I've gone to heroku with everything being basically a few clicks to get setup.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#310

Post by belorn »

kypro wrote: post #282
BbzzbB wrote: post #194

Earlier quoted context omitted.

There was an article not long ago from AWS saying they'll be focussing on cutting cost for customers. Maybe the next step of that process will be pushing their clients off of AWS and telling them to just host on prem.

I know you're joking around, but no, as they also explained a benefit of cloud (and therefore using AWS) is that it can scale flexibly with their customers' businesses. If your business invests in physical servers anticipating strong growth next year then later finds out actually we're going into a recession and those servers are no longer needed, then that's a sunk cost. With cloud if demand drops you can scale up a…

Using insurance to cover unexpected costs is always a gamble, one way or an other. A business that invest into physical servers could sell of those servers if they later find out that there is a recession, which might cost more or less compared to a cloud solution.

If a business invest into a cloud infrastructure and create a binding contract for 5 years, only to find out that they actually want to abandon that project a year later, that is also a sunk cost. Long term contracts tend to be cheaper, so its a trade off between saving money vs risk.

It all depend on the risk analysis, how risk averse one want to be, and the economics/liquidity needs.

Post reply on HN