Live data from Hacker News

Scaling up the Prime Video audio/video monitoring service and reducing costs

primevideotech.com

491–500 of 526 posts

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#491

Earlier quoted context omitted.

Well, the elapsed time is how long I need production to be offline so there are a few more people "just waiting"

Why would you take production offline to swap databases? Elapsed time is how long you need to run the new database alongside the old one.

This whole thread is about incompatible databases. If you know a 2 minute solution to a live transfer in that situation I’m all ears.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#492

Earlier quoted context omitted.

Then why are you defending the ridiculous "2 minute" thing? If the CEO asked how long a migration will take would you respond "2 minutes of engineering time"?

In that situation I would guess the one-click tool doesn’t really handle everything you’d need either so I don’t get what the point of the comparison is.

I’m not the person would said it was two minutes.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#493

Earlier quoted context omitted.

What if some team makes the their part 10 times slower. This is not a theoretical scenario, but one I saw happening many times. While technically you could partition and monitor each part of the monolith separately, but then you are just reinventing microservice architecture.

You just run a profiler. It's better and arguably easier than microservice profiling since it doesn't only measure at the interface, can measure fine grained memory usage, etc. Google "continuous profiling". I'm not sure why you would think that that reinvents microservices. Microservices is just taking a monolith and moving the components into separate processes that communicate via RPC.

In my experience, profiling is hard and a lot of times doesn't show the issue. e.g. for unnamed goroutines it is hard to see which goroutine profiler is referring to. Or say if some code change increased CPU usage without increase in time/memory, it will affect the entire monolith performance. Yes a good maintainer could pinpoint the issue, but remember my premise was that it is low trust environment, and saying I think your code change increased CPU usage involves talking to managers and being in 2 meetings. And in microservice they would have to deal with their alerts to not miss the SLA.

> Microservices is just taking a monolith and moving the components into separate processes that communicate via RPC.

Microservice architecture divides the responsibility much more than that. They have separate redis cache, local cache, tests, and even likely has different DB etc.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#494
Rarely will I defend Amazon in anything, but I'll make an exception.

In my experience, AWS/Amazon people do not force you or even direct you to a particular architectural choice. They are relatively indifferent about it.

Instead, trend-driven architectures seem to come from the tech community themselves. It's the customers often making the wrong choice.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#495

Earlier quoted context omitted.

>They converted every 8-byte payload buffer into a canonical binary representation, i.e. ascii strings of 1's and 0's. Honestly sounds like something I'd do but I've never programmed anything more dangerous than a toaster let alone a car.

Horribly inefficient code is a wonderful thing at a small scale. The faster you solve your problem, the sooner you can solve the next problem. I once threw together a mylar balloon helium blimp in the shape of a Dragon space capsule. My goal was to fly it over the cafeteria crowd at SpaceX during the C2 launch. For control, I used the PCB of a travel wifi router. I soldered three small DC motors to its LED outputs. T…

> For control, I used the PCB of a travel wifi router. I soldered three small DC motors to its LED outputs. The embedded software consisted of something like:

> nc -l -u -p 10000 | bash

That's a neat idea. Did you have to flash it with a custom firmware or do they typically come with netcat etc installed?

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#496
post #458

Earlier quoted context omitted.

This. And I think people tend not to understand how little actual hardware they are paying for when using AWS et al. A really cheap server leasing deal will cost you yearly about as much as the purchase price of the server. With opaque AWS services it is probably more like a month of subscription to pay for the hardware that you are indirectly using.

I worked for a global company that maintained it's own "cloud" of VMs that we'd use for development purposes. They were entirely unusable. Opening a relatively small file in notepad could take multiple minutes. OS click and typing response times were measured in seconds. Despite wasting thousands of developer hours each year, they refused to upgrade their data center. Probably because doing so would have been a major…

Notepad? Clicking? I think I see the problem.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#497
post #109

Earlier quoted context omitted.

There's a lot of room for "chinese-room" automation: Write a verbal description of your cloud function and let the LLM simulate the execution. Very cheap to develop. Very expensive to execute.

Prompt: Is the following number even or odd...

> Prompt: Is Avogadro's number even?

> ChatGPT: Yes, Avogadro's number is even. The value of Avogadro's number is approximately 6.022 x 10^23, and since it ends with the digit 2, it is an even number.

Right answer, wrong reasoning.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#498
post #112

Earlier quoted context omitted.

I agree, my intuition would put it to 1% vs. 99% (difficult to quantify of course). I haven't yet seen a project/product which would need microservice architecture for technical reasons. If you need to scale, you can just scale monoliths (perhaps serving in different roles). The use case for microservice architecture is IMHO an organizational / high level architecture driven. I've worked in a big company (20K employe…

I built a little microservice on the side of my monolith for PDF creation. It used headless chrome and ghostscript to render html to a nice PDF. The problem I had with having that code inside the monolith was that it increased my docker image creation for deploys by a lot . And that code pretty much never changed anyway. I did feel a bit embarrassed having to make a microservice after having argued against them so mu…

https://m.signalvnoise.com/the-majestic-monolith-can-become-...

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#499

Earlier quoted context omitted.

Horribly inefficient code is a wonderful thing at a small scale. The faster you solve your problem, the sooner you can solve the next problem. I once threw together a mylar balloon helium blimp in the shape of a Dragon space capsule. My goal was to fly it over the cafeteria crowd at SpaceX during the C2 launch. For control, I used the PCB of a travel wifi router. I soldered three small DC motors to its LED outputs. T…

> For control, I used the PCB of a travel wifi router. I soldered three small DC motors to its LED outputs. The embedded software consisted of something like: > nc -l -u -p 10000 | bash That's a neat idea. Did you have to flash it with a custom firmware or do they typically come with netcat etc installed?

The router had openWRT support. Once I flashed that on, it had minimal versions of all the usual cli tools through BusyBox.

Re: Scaling up the Prime Video audio/video monitoring service and reducing costs

#500

Earlier quoted context omitted.

In that situation I would guess the one-click tool doesn’t really handle everything you’d need either so I don’t get what the point of the comparison is.

I’m not the person would said it was two minutes.

Nor am I but if we go back up the claim was that they were making it much easier to get in than get out, but unless you're telling me that the one-click tool somehow solves all the issues of migrating a large production database that's in use the difference between the two is a minuscule amount of active work.
Post reply on HN