Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We replaced H.264 streaming with JPEG screenshots (and it worked better)

blog.helix.ml

Post reply on HN
Print view View original

141–150 of 332 posts

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#141

Post by dimatura »

sevensor wrote: post #78

No mention of PNGs? I don’t usually go to jpegs first for screenshots of text. Did png have worse compression? Burn more cpu? I’m sure there are good reasons, but it seems like they’ve glossed over the obvious choice here. edit: Thanks for the answers! The consensus is that PNG en/de -coding is too expensive compared to jpeg.

PNGs of screenshots would probably compress well, and the quality to size ratio would definitely be better than JPG, but the size would likely still be larger than a heavily compressed JPG. And PNG encoding/decoding is relatively slow compared to JPG.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#142

Post by ryukoposting »

adamjs wrote: post #63

They might want to check out what VNC has been doing since 1998– keep the client-pull model, break the framebuffer up into tiles and, when client requests an update, perform a diff against last frame sent, composite the updated tiles client-side. (This is what VNC falls back to when it doesn’t have damage-tracking from the OS compositor) This would really cut down on the bandwidth of static coding terminals where 90%…

Of all the suggestions in the comments here, this seems like the best one to start with.

Also... I get that the dumb solution to "ugly text at low bitrates" is "make the bitrate higher." But still, nobody looked at a 40M minimum and wondered if they might be looking at this problem from the wrong angle entirely?

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#143

Post by ronyfadel »

STELLANOVA wrote: post #132

We did something similar +12 years ago with `streaming` AWS running app inside the browser. Basically you can run 3d studio max on chromebook. App is actually running on AWS instance and it just sending jpegs to browser to `stream` it. We did a lot of QoS logic and other stuff but it was actually working pretty nice. Adobe used it for some time to allow user to run Photoshop in the browser. Good old days..

Who’s “we” in this case? Amazon (AWS)?

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#144

Post by K0nserv »

You can do TURN using TLS/TCP over port 443. This can fool some firewalls, but will still fail for instances when an intercepting HTTP proxy is used.

The neat thing about ICE is that you get automatic fallbacks and best path selection. So best case IPv6 UDP, worst case TCP/TLS

One of the nice things about HTTP3 and QUIC will be that UDP port 443 will be more likely to be open in the future.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#146

Post by jcalvinowens »

Dylan16807 wrote: post #55
vscode-rest wrote: post #53

Earlier quoted context omitted.

1Mbps for video is rule of thumb I use. Of course that will depend on customer expectations. 500K can work, but it won’t be pretty.

For normal video I think that's a good rule of thumb. For mostly-static content at 4fps you can cut a bunch more bitrate corners before it looks bad. (And 2-3 JPEGs per second won't even look good at 1Mbps.)

>> 10Mbps is still way too high of a minimum. It's more than YouTube uses for full motion 4k.

> And 2-3 JPEGs per second won't even look good at 1Mbps.

Unqualified claims like these are utterly meaningless. It depends too much on exactly what you're doing, some sorts of images will compress much better than others.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#148

Post by mikepavone »

01HNNWZ0MV43FF wrote: post #126
eichin wrote: post #123

Earlier quoted context omitted.

Probably either (1) they don't request another jpeg until they have the previous one on-screen (so everything is completely serialized and there are no frames "in-flight" ever) (2) they're doing a fresh GET for each and getting a new connection anyway (unless that kind of thing is pipelined these days? in which case it still falls back to (1) above.)

You can still get this backpressure properly even if you're doing it push-style. The TCP socket will eventually fill up its buffer and start blocking your writes. When that happens, you stop encoding new frames until the socket is able to send again. The trick is to not buffer frames on the sender.

You probably won't get acceptable latency this way since you have no control over buffer sizes on all the boxes between you and the receiver. Buffer bloat is a real problem. That said, yeah if you're getting 30-45 seconds behind at 40 Mbps you've probably got a fair bit of sender-side buffering happening.

Re: We replaced H.264 streaming with JPEG screenshots (and it worked better)

#150

Post by Sean-Der »

adamjs wrote: post #63

They might want to check out what VNC has been doing since 1998– keep the client-pull model, break the framebuffer up into tiles and, when client requests an update, perform a diff against last frame sent, composite the updated tiles client-side. (This is what VNC falls back to when it doesn’t have damage-tracking from the OS compositor) This would really cut down on the bandwidth of static coding terminals where 90%…

https://github.com/m1k1o/neko before VNC check neko out.

I worked on a project that started with VNC and had lots of problems. Slow connect times and backpressure/latency. Switching to neko was quick/easy win.

Post reply on HN