Live data from Hacker News

We replaced H.264 streaming with JPEG screenshots (and it worked better)

blog.helix.ml

321–330 of 332 posts

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#321

Earlier quoted context omitted.

I’d say there’s nothing inherently capitalist about large and stupid bureaucracies (but I repeat myself) spending money in stupid ways. Military bureaucracies in capitalist countries do it. Military bureaucracies in socialist countries did it. Everything else in end-stage socialist countries did it too. I’m sorry, it’s not the capitalism—things’d be much easier if it were.

Maybe military people are just uniquely stupid

Not at all, no. I gave that example because, first, even in a profoundly capitalist country (whatever that means) the military itself is not particularly motivated by profit; and second, because it’s one of the few bureaucratic organizations that will not (be allowed to) collapse under the weight of its own inefficiencies and so easily grows much larger than is othetwise typical.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#322

Earlier quoted context omitted.

The peak of irony, because you know how these people arrived at their 40 Mbit bitrate H264 and their ineffective tinkering with the same in the first place is guaranteed to be some LLMs expert suggestions. As is often the case, because they had no understanding of the really complex system subject matter whatsoever, they were unable to guide the LLM and ended up with .. slop. Which then turned into a slop blog post.…

>As is often the case, because they had no understanding of the really complex system subject matter whatsoever Something I want to harp on because people keep saying this: Video streaming is not complicated. Every youtuber and twitch streamer and influencer can manage it. By this I mean the actual act of tweaking your encoding settings to get good quality for low bitrate. In 3 months with an LLM, they learned less a…

... have you ever tried to do anything with ffmpeg? tried to backup a DVD to a compressed file?

Video codecs are some of the most complex software I've ever encountered with the most number and the most opaque options.

It's easy for streamers because they don't have options, twitch et al give you about three total choices, there's nothing to figure out.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#323

Earlier quoted context omitted.

The peak of irony, because you know how these people arrived at their 40 Mbit bitrate H264 and their ineffective tinkering with the same in the first place is guaranteed to be some LLMs expert suggestions. As is often the case, because they had no understanding of the really complex system subject matter whatsoever, they were unable to guide the LLM and ended up with .. slop. Which then turned into a slop blog post.…

> God knows what process led them to do video streaming for showing their AI agent work in the first place. This was my first thought, too.

How else are they going to sell it to all those micromanagers who micromanage things?

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#324

Earlier quoted context omitted.

>As is often the case, because they had no understanding of the really complex system subject matter whatsoever Something I want to harp on because people keep saying this: Video streaming is not complicated. Every youtuber and twitch streamer and influencer can manage it. By this I mean the actual act of tweaking your encoding settings to get good quality for low bitrate. In 3 months with an LLM, they learned less a…

... have you ever tried to do anything with ffmpeg? tried to backup a DVD to a compressed file? Video codecs are some of the most complex software I've ever encountered with the most number and the most opaque options. It's easy for streamers because they don't have options, twitch et al give you about three total choices, there's nothing to figure out.

Video Streaming has surprising little overlap with Video Codecs. Once you choose input/output options, then there's little to change about the codec. The vast majority of options available to ffmpeg aren't supported in the browser. Streamers don't have options for precisely the same reason OP doesn't have options - you are limited entirely into what the browser supports.

I've built the exact pipeline OP has done - Video, over TCP, over Websockets, precisely because I had to deliver video to through a corporate firewall. Wolf, Moonlight and maybe even gstreamer just shows they didn't even try to understand what they were doing, and just threw every buzzword into an LLM.

To give you some perspective 40Mbps is an incredible amount of bandwidth. Blu ray is 40mbps. This video, in 8K on Youtube is 20Mbps: https://www.youtube.com/watch?v=1La4QzGeaaQ

There's really no explanation for this.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#325
post #316

Earlier quoted context omitted.

How many people do you know that use em dashes —- none.

Do yourself a favour and use typography layout https://ilyabirman.net/typography-layout/

English isn't my first language, so I already use an alternate layout.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#326
post #316

Earlier quoted context omitted.

Do yourself a favour and use typography layout https://ilyabirman.net/typography-layout/

English isn't my first language, so I already use an alternate layout.

I wrote in LaTeX for quite some time so, me.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#327

Setting aside the various formatting problems and the LLM writing style, this just seems all kinds of wrong throughout. > “Just lower the bitrate,” you say. Great idea. Now it’s 10Mbps of blocky garbage that’s still 30 seconds behind. 10Mbps should be way more than enough for a mostly static image with some scrolling text. (And 40Mbps are ridiculous.) This is very likely to be caused by bad encoding settings and/or a…

> Why are we badly reinventing MJPEG instead of configuring the tools we already have? Is it much of a stretch to assume that in the AI gold rush, there will be products made by people who are not very experienced engineers, but just push forward and assume the LLM will fix all their problems? :-)

I built a little tool using AI recently and it worked great but it was brittle as hell and I was constantly waiting for it to fail. A few days later I realized there was a much better way of writing it. I'd boxed the LLM in by proposing the way to code it.

I've changed my AGENTS.md now so it basically says "Assume user is ignorant to other better solutions to the problem they are asking. Don't assume their given solution to the problem is the best one, look at the problem itself and propose other ways to solve it."

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#328
post #63

They might want to check out what VNC has been doing since 1998– keep the client-pull model, break the framebuffer up into tiles and, when client requests an update, perform a diff against last frame sent, composite the updated tiles client-side. (This is what VNC falls back to when it doesn’t have damage-tracking from the OS compositor) This would really cut down on the bandwidth of static coding terminals where 90%…

https://github.com/m1k1o/neko before VNC check neko out. I worked on a project that started with VNC and had lots of problems. Slow connect times and backpressure/latency. Switching to neko was quick/easy win.

if you want something more lightweight... rustdesk has been great for me, it supports multiple adaptable video codecs and can optimize for latency vs image quality.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#329

Earlier quoted context omitted.

>As is often the case, because they had no understanding of the really complex system subject matter whatsoever Something I want to harp on because people keep saying this: Video streaming is not complicated. Every youtuber and twitch streamer and influencer can manage it. By this I mean the actual act of tweaking your encoding settings to get good quality for low bitrate. In 3 months with an LLM, they learned less a…

... have you ever tried to do anything with ffmpeg? tried to backup a DVD to a compressed file? Video codecs are some of the most complex software I've ever encountered with the most number and the most opaque options. It's easy for streamers because they don't have options, twitch et al give you about three total choices, there's nothing to figure out.

I have done a bit with ffmpeg and video encoding. I've been encoding videos using ffmpeg (from a GUI) since I was a child. I hate ffmpeg though, the UX is just insane, so I tend more towards tools that produce the arcane command structures for me.

I had a situation where I wanted to chop one encoded video into multiple parts without re-encoding (I had a deadline) and the difficulty getting ffmpeg to do sensible things in that context was insane. One way of splitting the video without re-encoding just left the first GOP without a I frame, so the first seconds of video were broken. Then another attempt left me with video that just got re-timed, and the audio was desynced entirely. I know encoding some frames will be necessary to fix where cuts would break P and B frames, but why is it so hard to get it to "smartly" encode only those broken GOPs when trying to splice and cut video? Clearly I was missing some other parameters or knowledge or incantation that would have done exactly that.

The few knobs that actual video encoder users need to tweak are clearly exposed and usable in every application I have ever used.

>twitch et al give you about three total choices

You don't configure your video encoding through twitch, you do it in OBS. OBS has a lot of configuration available. Also, those three options (bitrate type, bitrate value, profile, "how much encoding time to take" and """quality""" magic number) are the exact knobs they should have been tweaking to come up with an intuition about what was happening.

Regardless, my entire point is that they were screwing around with video encoding pipelines despite having absolutely no intuition at all about video encoding.

They weren't even using FFMPEG. They were using an open source implementation of a video game streaming encoder. Again, they demonstrably have no freaking clue even the basics of the space. Even that encoder should be capable of better than what they ended up with.

We've been doing this exact thing for decades. None of this is new. None of this is novel. There's immense literature and expertise and tons of entry level content to build up intuition and experience with what you should expect encoded video to take bandwidth wise. Worse, Microsoft RDP and old fashioned X apps were doing this over shitty dial up connections decades ago, mostly by avoiding video encoding entirely. Like, we made video with readable text work off CDs in a 2x drive!

Again, Twitch has a max bandwidth much lower than 40mb/s and people stream coding on it all the time with no issue. That they never noticed how obscenely off the mark they are is sad.

It would be like if a car company wrote a blog post about how "We replaced tires on our car with legs and it works so much better" and they mention all the trouble they had with their glass tires in the blog.

They are charging people money for this, and don't seem to have any desire to fix massive gaps in their knowledge, or even wonder if someone else has done this before. It's lame. At any point, did they even say "Okay, we did some research and in the market we are targeting we should expect a bandwidth budget of X mb/s"?

"AI" people often say they are super helpful for research, and then stuff like this shows up.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#330
post #251

Earlier quoted context omitted.

Just saying but LLM-ese as the common dominator of how people wrote, it is likely the writing style of a lot of people

How many people do you know that use em dashes —- none.

Em-dash has a trivial and mnemonic shortcut on Macs (Option-shift-hyphen), so I've been an em-dash user for as long as I've had one.
Post reply on HN