Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We replaced H.264 streaming with JPEG screenshots (and it worked better)

blog.helix.ml

Post reply on HN
Print view View original

221–230 of 332 posts

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#221

Post by lewq »

Hi, author of the post here. Just fixed up some formatting issues from when we copied it into substack, sorry about that. Yeah, I used Opus 4.5 to help me write it (and it actually made me laugh!). But the struggle was real. Something I didn't make clear enough in the post is that jpeg works because each screenshot is taken exactly when it's requested. Whereas streaming video is pushing a certain frame rate. The client driving the frame rate is exactly what makes it not queue frames. Yes, I wish we could UDP in enterprise networks too, but we can't. The problem actually isn't opening the UDP port, it's hosting UDP on their Kubernetes cluster. "You want to what?? We have ingress. For HTTPS"

Join our discord for private beta in January! https://discord.gg/VJftd844GE

(This post written by human)

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#222

Post by nwallin »

taberiand wrote: post #118
kccqzy wrote: post #84

There are so many things that I would have done differently. > We added a keyframes_only flag. We modified the video decoder to check FrameType::Idr. We set GOP to 60 (one keyframe per second at 60fps). We tested. Why muck around with P-frames and keyframes? Just make your video 1fps. > Now it’s 10Mbps of blocky garbage that’s still 30 seconds behind. 10 Mbps is way too much. I occasionally watch YouTube videos where…

This blog post smells of LLM, both in the language style and the muddled explanations / bad technical justifications. I wouldn't be surprised if their code is also vibe coded slop.

> I wouldn't be surprised if their code is also vibe coded slop.

That's my takeaway from this too. I think they tried the first thing the LLM suggested, it didn't work, they asked the LLM to fix it, and ended up with this crap. They never tried to really understand the problems they were facing.

Video is really fiddly. You have all sorts of parameters to fiddle with. If you don't dig into that and figure out what tradeoffs you need to make, you'll easily end up in the position where checks notes you think you need 40Mbps for 1080p video and 10Mbps is just too shitty.

There's various points in the article where they talk about having 30 seconds of latency. Whatever's causing this, this is a solved problem. We all have experience dealing with video teleconferencing, this isn't anything new, it's nothing special, they're just doing it wrong. They say it doesn't work because of corporate network policy, but we all use Teams or Slack.

I think you're right. They just did a bunch of LLM slop and decided to just send it. At no point did they understand any of their problems any deeper than the LLM tried to understand the problem.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#223

Post by darthwalsh »

michaelt wrote: post #76
egorfine wrote: post #5

> The constraint that ruined everything: It has to work on enterprise networks. > You know what enterprise networks love? HTTP. HTTPS. Port 443. That’s it. That’s the list. That's not enough. Corporate networks also love to MITM their own workstations and reinterpret http traffic. So, no WebSockets and no Server-Side Events either, because their corporate firewall is a piece of software no one in the world wants and…

> And you have to work around that, because IT dept of the corporation will never lift restrictions. Unless the corporation is 100% in-office, I’d wager they do in fact make exceptions - otherwise they wouldn’t have a working videoconferencing system. The challenge is getting corporate insiders to like your product enough to get it through the exception process (a total hassle) when the firewall’s restrictions mean y…

I think our corporate VPN doesn't send zoom video traffic through the VPN. As you enabled the VPN, you didn't see any dropped frames.

Split tunnelling means the UDP packets just go through the normal internet.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#224

Post by tcherasaro »

Reminds me when I was working on the video system for a mast on a sub-marine 20 years ago.

Customer had impossible set of latency, resolution, processing and storage requirements for their video. They also insisted we use this new H.264 standard that just came out though not a requirement.

We quickly found MJPEG was superior for meeting their requirements in every way. It took a lot of convincing though. H.264 was and would still be a complete non-starter for them.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#225

Post by Scaevolus »

martinald wrote: post #204
ryukoposting wrote: post #142

Earlier quoted context omitted.

Of all the suggestions in the comments here, this seems like the best one to start with. Also... I get that the dumb solution to "ugly text at low bitrates" is "make the bitrate higher." But still, nobody looked at a 40M minimum and wondered if they might be looking at this problem from the wrong angle entirely?

In fairness VNC-style approaches are bloody awful even over my 2.5gbit/sec lan on very fast hardware. It just cannot do 4K well (not sure if they need 4k or not). I spent some time compiling the "new" xrdp with x264 and it is incredibly good, basically cannot really tell that I'm remote desktoping. The bandwidth was extremely low as well. You are correct on that part, 40mbit/sec is nuts for high quality. I suspect if…

Moonlight is mostly designed to stream your gaming desktop to a portable device or your TV at minimal latency and maximum quality within a LAN. For that, 40Mbps is quite reasonable. It's obviously absurd for mundane VNC/productivity workloads.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#227

Post by Dylan16807 »

mdavid626 wrote: post #100
kccqzy wrote: post #84

There are so many things that I would have done differently. > We added a keyframes_only flag. We modified the video decoder to check FrameType::Idr. We set GOP to 60 (one keyframe per second at 60fps). We tested. Why muck around with P-frames and keyframes? Just make your video 1fps. > Now it’s 10Mbps of blocky garbage that’s still 30 seconds behind. 10 Mbps is way too much. I occasionally watch YouTube videos where…

Setting to 1 FPS might not be enough. GOP or P frame setting needs to be adjusted to make every frame keyframe.

Why would you do that?

Nearly-static content is where you want even fewer keyframes than usual. In a situation like this you need them when the connection is interrupted and you reset things, and not much of anywhere else.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#228

Post by Terretta »

Helix is a commercial multi-protocol streaming server:

https://en.wikipedia.org/wiki/Helix_Universal_Server

HTTP Live Streaming is already a thing:

https://en.wikipedia.org/wiki/HTTP_Live_Streaming

See also DASH, M-JPEG, progressive download, etc.

> "Who knew?"

Everyone in the streaming industry, and not so long ago that it's been forgotten.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#229

Post by dinobones »

You spent 3 months on this hacked together garbage when you probably could’ve just configured a pre-existing solution off the shelf with like 10 minutes of reading and understanding documentation.

This blog post reeks of “you can just do things” type of engineering. This is the quality of engineering I would expect from “TPOT” (that part of Twitter) where people talk about working 12 hour days. It’s cause they’re working 12 hours on bullshit like this.

Building some sweet custom codec or binary transportation algorithm was barely cute in like 1989. It definitely ain’t cute now.

How many of these AI and “agentic” companies are just misled engineers thinking they are cracked and writing needlessly complex solutions to problems that dont even exist?

Just burn it all down. Let it pop already.

Post reply on HN