Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We replaced H.264 streaming with JPEG screenshots (and it worked better)

blog.helix.ml

Post reply on HN
Print view View original

241–250 of 332 posts

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#241

Post by TiredOfLife »

Dylan16807 wrote: post #46
HelloUsername wrote: post #40

Earlier quoted context omitted.

> Try 1Mbps and iterate from there. From the article: “Just lower the bitrate,” you say. Great idea. Now it’s 10Mbps of blocky garbage that’s still 30 seconds behind.

Rejecting it out of hand isn't actually trying it. 10Mbps is still way too high of a minimum. It's more than YouTube uses for full motion 4k. And it would not be blocky garbage, it would still look a lot better than JPEG.

Youtube 4k uses VP9 and AV1 codecs that are multiple generations ahead of H.264

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#242

Post by josephernest »

Related: for some hardware project, I have a backend server (either C++ or python) receiving frames from an industrial camera, uncompressed.

And I need these frames displayed in a web browser client but on the same computer (instead of network trip like in this article).

How would you do this ?

I eventually did more or less like OP with uncompressed frames.

My goal is to minimize CPU usage on the computer. Would h264 compression be a good thing here given source and destination are the same machine?

Other ideas?

NB: this camera cannot be directly accessed by the browser.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#244

Post by antisol »

ozim wrote: post #183
qbow883 wrote: post #154

Setting aside the various formatting problems and the LLM writing style, this just seems all kinds of wrong throughout. > “Just lower the bitrate,” you say. Great idea. Now it’s 10Mbps of blocky garbage that’s still 30 seconds behind. 10Mbps should be way more than enough for a mostly static image with some scrolling text. (And 40Mbps are ridiculous.) This is very likely to be caused by bad encoding settings and/or a…

* Why are we badly reinventing MJPEG instead of configuring the tools we already have?* Getting to know and understand existing tools costs time/money. If it less expensive or more expensive than reinventing something badly is very complicated to judge and depends on loads of factors. Might be that reinventing something badly - but good enough for the case is best use of resources.

From TFA:

    Implementation complexity: 
     h264 Stream: 3 months of rust
     JPEG Spam: fetch() in a loop
I don't see how it could have taken 3 months to read up on existing technologies. And that "3 month" number is before we start factoring in time spent on:

* Writing code for JPEG Spam / "fetch() in a loop" method

* Mechanisms to switch between h264 / jpeg modes

* Debugging implementation of 2 modes

* Debugging switching back and forth between the 2 modes

* Maintenance of 2 modes into the future

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#245

Post by kalleboo »

kccqzy wrote: post #84

There are so many things that I would have done differently. > We added a keyframes_only flag. We modified the video decoder to check FrameType::Idr. We set GOP to 60 (one keyframe per second at 60fps). We tested. Why muck around with P-frames and keyframes? Just make your video 1fps. > Now it’s 10Mbps of blocky garbage that’s still 30 seconds behind. 10 Mbps is way too much. I occasionally watch YouTube videos where…

> I occasionally watch YouTube videos

My experience is that at the same bitrate, real-time hardware encoding is way worse quality than offline CPU encoding (what YouTube does when you upload a video) so you can't compare them directly.

10 Mbps is still crazy high, and the target should still be around 1 Mbps.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#246

Post by afiori »

mikepavone wrote: post #102

> When the network is bad, you get... fewer JPEGs. That’s it. The ones that arrive are perfect. This would make sense... if they were using UDP, but they are using TCP. All the JPEGs they send will get there eventually (unless the connection drops). JPEG does not fix your buffering and congestion control problems. What presumably happened here is the way they implemented their JPEG screenshots, they have some mechani…

I am almost sure that the most perfect solution would involve using a video codec protocol but the issue is implementation complexity and having to implement a production encoder yourself if your usecase is unusual.

This is exactly the point of the article they tried keyframes only but their library had a bug that broke it

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#248

Post by antisol »

stefan_ wrote: post #186
colechristensen wrote: post #171

Earlier quoted context omitted.

This is where LLMs shine, where you need to dip your toes into really complex systems but basically just to do one thing with pretty straightforward requirements.

The peak of irony, because you know how these people arrived at their 40 Mbit bitrate H264 and their ineffective tinkering with the same in the first place is guaranteed to be some LLMs expert suggestions. As is often the case, because they had no understanding of the really complex system subject matter whatsoever, they were unable to guide the LLM and ended up with .. slop. Which then turned into a slop blog post.…

  > God knows what process led them to do video streaming for showing their AI agent work in the first place.
This was my first thought, too.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#249

Post by antisol »

josephernest wrote: post #242

Related: for some hardware project, I have a backend server (either C++ or python) receiving frames from an industrial camera, uncompressed. And I need these frames displayed in a web browser client but on the same computer (instead of network trip like in this article). How would you do this ? I eventually did more or less like OP with uncompressed frames. My goal is to minimize CPU usage on the computer. Would h264…

> How would you do this ?

It depends. I have many questions.

> My goal is to minimize CPU usage on the computer. Would h264 compression be a good thing here given source and destination are the same machine?

No.

> Other ideas?

1. Why does it need to be displayed in a web browser (as opposed to more appropriate / better performing software specifically built for video)?

2. via what interface/library is the camera connected to the machine? What format/codec is the uncompressed stream you're getting from the camera?

3. I am available at very reasonable consulting rates

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#250

Post by tsimionescu »

londons_explore wrote: post #179
Sesse__ wrote: post #176

Earlier quoted context omitted.

> I couldn't find a single video codec which could do that without extensive internal surgery to save all internal state after the 15th frame. fork()? :-) But most software, video codec or not, simply isn't written to serialize its state at arbitrary points. Why would it?

A word processor can save it's state at an arbitrary point... That's what the save button is for, and it's functional at any point in the document writing process! In fact, nearly everything in computing is serializable - or if it isn't, there is some other project with a similar purpose which is. However this is not the case with video codecs - but this is just one of many examples of where the video codec landscape…

> A word processor can save it's state at an arbitrary point... That's what the save button is for, and it's functional at any point in the document writing process!

No, they generally can't save their whole internal state to be resumed later, and definitely not in the document you were editing. For example, when you save a document in vim it doesn't store the mode you were in, or the keyboard macro step that was executing, or the search buffer, or anything like that.

> In fact, nearly everything in computing is serializable - or if it isn't, there is some other project with a similar purpose which is.

Serializable in principle, maybe. Actually serializable in the sense that the code contains a way to dump to a file and back, absolutely not. It's extremely rare for programs to expose a way to save and restore from a mid-state in the algorithm they're implementing.

> Another for example is that on the internet lots of videos have a 'poster frame' - often the first frame of the video. That frame for nearly all usecases ends up downloaded twice - once as a jpeg, and again inside the video content.

Actually, it's extremely common for a video thumbnail to contain extra edits such as overlayed text and other graphics that don't end up in the video itself. It's also very common for the thumbnail to not be the first frame in the video.

Post reply on HN