Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We replaced H.264 streaming with JPEG screenshots (and it worked better)

blog.helix.ml

Post reply on HN
Print view View original

111–120 of 332 posts

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#111

Post by cogman10 »

esseph wrote: post #103
michaelt wrote: post #90

Earlier quoted context omitted.

Believe me, the average Fortune 500 CEO does not know or care what “SSL MITM” is, or whether passwords should contain symbols and be changed monthly, or what the difference is between ‘VPN’ and ‘Zero Trust’. They delegate that stuff. To the corporate IT department.

But they also say "Here, this is Sarah your auditor. Answer these questions and resolve the findings." - every year It's all CyberSecurity insurance compliance that in many cases deviates from security best practices.

This is where the problems come from. Auditors are definitely what ultimately causes IT departments to make dumb decisions.

For example, we got dinged on an audit because instead of using RSA4096, we used ed25519. I kid you not, their main complaint was there wasn't enough bits which meant it wasn't secure.

Auditors are snake oil salesman.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#112

Post by eterm »

dotancohen wrote: post #51

They're just streaming a video feed of an LLC running in a terminal? Why not stream the actual text? Or fetch it piecemeal over AJAX requests? They complain that corporate networks support only HTTPS and nothing else? Do they not understand what the first T stands for?

Indeed, live text streaming is well over 100 years old:

https://en.wikipedia.org/wiki/Teleprinter

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#113

Post by nico »

Super interesting. Some time ago I wrote some code that breaks down a jpeg image into smaller frames of itself, then creates an h.264 video with the frames, outputting a smaller file than the original image

You can then extract the frames from the video and reconstruct the original jpeg

Additionally, instead of converting to video, you can use the smaller images of the original, to progressively load the bigger image, ie. when you get the first frame, you have a lower quality version of the whole image, then as you get more frames, the code progressively adds detail with the extra pixels contained in each frame

It was a fun project, but the extra compression doesn’t work for all images, and I also discovered how amazing jpeg is - you can get amazing compression just by changing the quality/size ratio parameter when creating a file

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#115

Post by TacticalCoder »

MBCook wrote: post #65

So it’s video of an AI typing text? Why not just send text? Why do you need video at all?

You apparently need video for the 45 seconds window you then get before preventing catastrophic things to happen. From TFA:

> You’re watching the AI type code from 45 seconds ago > > By the time you see a bug, the AI has already committed it to main > > Everything is terrible forever

Is this satire? I mean: if the solution for things to not be terrible forever consists in catching what an AI is doing in 45 seconds (!) before the AI commits to trunk, I'm sorry but you should seriously re-evaluate your life plans.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#116

Post by kevlened »

Aurornis wrote: post #99
egorfine wrote: post #5

> The constraint that ruined everything: It has to work on enterprise networks. > You know what enterprise networks love? HTTP. HTTPS. Port 443. That’s it. That’s the list. That's not enough. Corporate networks also love to MITM their own workstations and reinterpret http traffic. So, no WebSockets and no Server-Side Events either, because their corporate firewall is a piece of software no one in the world wants and…

> So, no WebSockets The corporate firewall debate came up when we considered websockets at a previous company. Everyone has parroted the same information for so long that it was just assumed that websockets and corporate firewalls were going to cause us huge problems. We went with websockets anyway and it was fine. Almost no traffic to the no-websockets fallback path, and the traffic that did arrive appeared to be fr…

I've had to switch from SSE to WebSockets to navigate a corporate network (the entire SSE would have to close before the user received any of the response).

Then we ran into a network where WebSockets were blocked, so we switched to streaming http.

No trouble with streaming http using a standard content-type yet.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#117

Post by poly2it »

Why is video streaming so difficult? We've been doing it for decades, why is there seemingly no FOSS library which let's me encode an arbitrary dynamic frame rate image stream in Rust and get HD data with delta encoding in a browser receiver? This is insanity.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#118

Post by taberiand »

kccqzy wrote: post #84

There are so many things that I would have done differently. > We added a keyframes_only flag. We modified the video decoder to check FrameType::Idr. We set GOP to 60 (one keyframe per second at 60fps). We tested. Why muck around with P-frames and keyframes? Just make your video 1fps. > Now it’s 10Mbps of blocky garbage that’s still 30 seconds behind. 10 Mbps is way too much. I occasionally watch YouTube videos where…

This blog post smells of LLM, both in the language style and the muddled explanations / bad technical justifications. I wouldn't be surprised if their code is also vibe coded slop.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#119

Post by collingreen »

unethical_ban wrote: post #68
kbelder wrote: post #58

Earlier quoted context omitted.

Oh, they'll do that anyway, once they find the workaround (Oh... you can paste a credit card if you put periods instead of dashes! Oh... I have to save the file and do it from my phone! Oh... I'll upload it as a .txt file and change the extension on the server!) It's purely illusory security, that doesn't protect anything but does levy a constant performance tax on nearly every task.

What's the term for the ideology that "laws are silly because people sometimes break them"?

I don't think that's a good read if the post you're implying this at. I think a more charitable read would be something like "people break rules for convenience so if your security relies on nobody breaking rules then you don't have thorough security".

You and op can be right at the same time. You imply the rules probably help a lot even while imperfect. They imply that pretending rules alone are enough to be perfect is incomplete.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#120

Post by worksonmine »

I'm confused, do people actually watch their agents code like it was a screen share? Why does the AI even mess with that, just send a diff over text? Is it getting a keyboard next?

This is the definition of over-engineering. I don't usually criticize ideas but this is so stupid my head hurts.

Post reply on HN