Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Video Works

howvideo.works

Post reply on HN
Print view View original

21–26 of 26 posts

Re: How Video Works

#21

Post by rememberlenny »

Thank you to the Mux team for putting this together! I've been using Mux as the basis of milk.video's transcoding and video serving from the start, and it's been an absolute pleasure to work with.

This site is an incredible, concise, and comprehensive resource for people trying to better understand how video works.

I attended the demuxed conference this year, and was exposed for the first time into the nitty gritty of how video works behind the scenes.

Huge props for the content not being a sales pitch, and truly being educational and informative.

Re: How Video Works

#22

Post by knaik94 »

Jackypot wrote: post #19

Perhaps someone here can answer a question I've had for a while: if you stream a video which is actually a static image (think a song on YouTube, the 'video' is just the album cover) is there any way to optimise that? Or must the server stream that same image constantly as though it was a regular video?

That's optimized based on the video codec itself where you keep frames with major changes but surrounding frames can be optimized by encoding where certain pixels move. That's essentially what the bitrate tells us, how much of the bits are new and how many are copied from a previous frame. This is an oversimplification, but with static images, you would need a low bitrate after the initial few frames because it doesn't change. And bitrate is a proxy to how much data is transmitted while streaming, it's bits per second.

So a more concise answer, the codecs used for video by YouTube have the optimization you're thinking about built in. It would send a "pixel didn't change from last frame" and not need to send all the color information for all the pixels.

YouTube does not really do variable frame rate, and it's messy for editing, but it's another optimization that is possible and could be useful for the type of video you're describing.

The article describes DASH which you would need to send the full frame initially every time segment but per segment the previously described concept still applies. I don't believe YouTube uses DASH for anything outside live streams.

Re: How Video Works

#23

Post by rocgf »

Jackypot wrote: post #19

Perhaps someone here can answer a question I've had for a while: if you stream a video which is actually a static image (think a song on YouTube, the 'video' is just the album cover) is there any way to optimise that? Or must the server stream that same image constantly as though it was a regular video?

I do not know the answer, but I would absolutely shocked if it was streamed as a regular video. To the best of my knowledge, there are many, many optimizations which go into video.

This is a pretty good high-level video I came across - https://www.youtube.com/watch?v=r6Rp-uo6HmI

Re: How Video Works

#24

Post by galad87 »

j1elo wrote: post #8
AnotherGoodName wrote: post #4

One thing i'd love to see added here is a blurb about the importance of the location of the video metadata in the file. Specifically that you need to have the metadata at the start of the file rather than the end of the file (which is the default) for low-latency playback on web. Explained: Freshly recorded MPEG (and almost all other container types) typically saves the header at the end of the file. This is the logi…

This is what the MP4 format calls the "moov atom". Writing metadata at the end, instead of periodically interleaving it inside the file, is not only useless for web playback, but also for any creation process (i.e., live recording) which intends to call itself robust . Imagine recording a 1-hour long video, only to have some unrelated issue abruptly stopping it all (e.g. the application crashes, or the video camera s…

And that's the reason fragmented mp4 were invented.

Re: How Video Works

#25

Post by zinekeller »

Jackypot wrote: post #19

Perhaps someone here can answer a question I've had for a while: if you stream a video which is actually a static image (think a song on YouTube, the 'video' is just the album cover) is there any way to optimise that? Or must the server stream that same image constantly as though it was a regular video?

The previous answers were speculative, but I have tried to download a random audio/video that was uploaded through YouTube's media programme (directly by media companies), and the answer is that they do re-send the video frame every 5 seconds for both H.264 and VP9, and it is encoded both at a rate of 25 frames per second and the width and height are the same. Here are the file details for both formats:

H.264/AAC4:

  Video
  ID                                       : 1
  Format                                   : AVC
  Format/Info                              : Advanced Video Codec
  Format profile                           : High@L3.2
  Format settings                          : CABAC / 2 Ref Frames
  Format settings, CABAC                   : Yes
  Format settings, Reference frames        : 2 frames
  Codec ID                                 : avc1
  Codec ID/Info                            : Advanced Video Coding
  Duration                                 : 3 min 29 s
  Bit rate                                 : 375 kb/s
  Width                                    : 1 080 pixels
  Height                                   : 1 080 pixels
  Display aspect ratio                     : 1.000
  Frame rate mode                          : Constant
  Frame rate                               : 25.000 FPS
  Color space                              : YUV
  Chroma subsampling                       : 4:2:0
  Bit depth                                : 8 bits
  Scan type                                : Progressive
  Bits/(Pixel*Frame)                       : 0.013
  Stream size                              : 9.35 MiB (74%)
  Writing library                          : x264 core 155 r2901 7d0ff22
  Codec configuration box                  : avcC

  Audio
  ID                                       : 2
  Format                                   : AAC LC
  Format/Info                              : Advanced Audio Codec Low Complexity
  Codec ID                                 : mp4a-40-2
  Duration                                 : 3 min 29 s
  Bit rate mode                            : Constant
  Bit rate                                 : 128 kb/s
  Channel(s)                               : 2 channels
  Channel layout                           : L R
  Sampling rate                            : 44.1 kHz
  Frame rate                               : 43.066 FPS (1024 SPF)
  Compression mode                         : Lossy
  Stream size                              : 3.19 MiB (25%)
  Default                                  : Yes
  Alternate group                          : 1
VP9/Opus:

  Video
  ID                                       : 1
  Format                                   : VP9
  Codec ID                                 : V_VP9
  Duration                                 : 3 min 29 s
  Width                                    : 1 080 pixels
  Height                                   : 1 080 pixels
  Display aspect ratio                     : 1.000
  Frame rate mode                          : Constant
  Frame rate                               : 25.000 FPS
  Language                                 : English
  Default                                  : Yes
  Forced                                   : No
  Color range                              : Limited

  Audio
  ID                                       : 2
  Format                                   : Opus
  Codec ID                                 : A_OPUS
  Duration                                 : 3 min 29 s
  Channel(s)                               : 2 channels
  Channel layout                           : L R
  Sampling rate                            : 48.0 kHz
  Bit depth                                : 32 bits
  Compression mode                         : Lossy
  Language                                 : English
  Default                                  : Yes
  Forced                                   : No

Re: How Video Works

#26

Post by LocalH »

rocgf wrote: post #23
Jackypot wrote: post #19

Perhaps someone here can answer a question I've had for a while: if you stream a video which is actually a static image (think a song on YouTube, the 'video' is just the album cover) is there any way to optimise that? Or must the server stream that same image constantly as though it was a regular video?

I do not know the answer, but I would absolutely shocked if it was streamed as a regular video. To the best of my knowledge, there are many, many optimizations which go into video. This is a pretty good high-level video I came across - https://www.youtube.com/watch?v=r6Rp-uo6HmI

I mean, technically, it would be "streamed as a regular video", it's just that "regular video encoding" is pretty efficient at encoding a single unchanging image nowadays.
Post reply on HN