Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Twitter's GIF hack

blog.embed.ly

Post reply on HN
Print view View original

151–160 of 160 posts

Re: Twitter's GIF hack

#151

Post by 4ad »

pornel wrote: post #119
ars wrote: post #12

This is not true: "A GIF is literally a sequence of independent images squeezed into the same file. An mp4 video can take advantage of all kinds of fancy compression techniques like keyframes and forward-predictive frames." That's not why the mp4 is smaller. A GIF is not [just] independent images, you can have each frame reuse pixels from previous frames. The mp4 is smaller because lossy jpeg compression is used on e…

I was able to halve size of GIFs with lossy LZW compression: https://pornel.net/lossygif With a good palette (which most software is terrible at) and proper delta frames (without artificial differences created by blindly applied diffusion) GIFs can be 1/4th of the size you get from an average GIF maker. Still, it's a terrible video codec. I think trauma caused by awful plugins like RealPlayer and QuickTime made peopl…

> needs more memory or CPU (to enable seeking, used when you switch tabs)

Gif is pretty bad, yes, but somehow I can browse pages with thousands of gifs. I can't browse pages with thousand of embedded videos.

Re: Twitter's GIF hack

#152

Post by pornel »

4ad wrote: post #151
pornel wrote: post #119

Earlier quoted context omitted.

I was able to halve size of GIFs with lossy LZW compression: https://pornel.net/lossygif With a good palette (which most software is terrible at) and proper delta frames (without artificial differences created by blindly applied diffusion) GIFs can be 1/4th of the size you get from an average GIF maker. Still, it's a terrible video codec. I think trauma caused by awful plugins like RealPlayer and QuickTime made peopl…

> needs more memory or CPU (to enable seeking, used when you switch tabs) Gif is pretty bad, yes, but somehow I can browse pages with thousands of gifs. I can't browse pages with thousand of embedded videos.

> Gif is pretty bad, yes, but somehow I can browse pages with thousands of gifs. I can't browse pages with thousand of embedded videos.

That's true, but it's not limited by compression, only by assumptions/optimizations in the browsers.

For example browsers try very hard top stop decoding GIFs that are not currently visible on the screen. This could be done even better with a properly seekable video codec, but it's just not implemented.

Re: Twitter's GIF hack

#153

Post by Kayou »

ferongr wrote: post #105
Kayou wrote: post #95

Earlier quoted context omitted.

Thank you! I was gonna post the exact same thing. Nice website, though. I encoded the gif in 420 (subsampling), 422 and 444, having a nice quality with 444 with VLC, I embed the three different videos in a web page and when playing in my browser, they are all as bad as the twitter video! I thought that Firefox was using a standard library to play MP4?

Firefox uses the Windows-bundled decoder on Windows Vista or later. Chroma upsampling happens after decoding so it's the renderer's responsibility (Firefox, Chrome, VLC, madVR etc). Depending on the webpage you uploaded them, the videos could be re-encoded (or VLC's options could be no-ops and the artifacts hidden when played back in VLC due to better chroma upsampling) hence the problem.

Okay, thanks for the explanation on the chroma upsampling! I use Linux but it should work all the same.

When I said that I embedded the videos in a web page, I wrote a small page from scratch to display them in my browser, locally, without any server. So no re-encoding. I think that VLC just has a better chroma upsampling.

Re: Twitter's GIF hack

#154

Post by 4ad »

pornel wrote: post #152
4ad wrote: post #151

Earlier quoted context omitted.

> needs more memory or CPU (to enable seeking, used when you switch tabs) Gif is pretty bad, yes, but somehow I can browse pages with thousands of gifs. I can't browse pages with thousand of embedded videos.

> Gif is pretty bad, yes, but somehow I can browse pages with thousands of gifs. I can't browse pages with thousand of embedded videos. That's true, but it's not limited by compression, only by assumptions/optimizations in the browsers. For example browsers try very hard top stop decoding GIFs that are not currently visible on the screen. This could be done even better with a properly seekable video codec, but it's j…

No argument there. I was just stating a reason why gif-heavy site operators are reluctant to switch to video. It's chicken and the egg. They won't switch because video is not as optimised yet, the browser vendor has no reason to optimise video further because sites usually don't have that many embedded videos.

Re: Twitter's GIF hack

#155

Post by feld »

eridius wrote: post #112
ars wrote: post #12

This is not true: "A GIF is literally a sequence of independent images squeezed into the same file. An mp4 video can take advantage of all kinds of fancy compression techniques like keyframes and forward-predictive frames." That's not why the mp4 is smaller. A GIF is not [just] independent images, you can have each frame reuse pixels from previous frames. The mp4 is smaller because lossy jpeg compression is used on e…

It's not just lossless/lossy. GIF is also rather primitive; you can store a delta from frame to frame, but the delta has to include every pixel that changed. A lossless animated format invented today wouldn't have that restriction, it would support things like pixel motion (which e.g. H.264 has). Some years ago there were multiple efforts to produce animated PNGs. There was MNG (Multiple-image Network Graphics, which…

APNG is used heavily in the gaming (slot machine) industry.

Re: Twitter's GIF hack

#156

Post by eridius »

feld wrote: post #155
eridius wrote: post #112

Earlier quoted context omitted.

It's not just lossless/lossy. GIF is also rather primitive; you can store a delta from frame to frame, but the delta has to include every pixel that changed. A lossless animated format invented today wouldn't have that restriction, it would support things like pixel motion (which e.g. H.264 has). Some years ago there were multiple efforts to produce animated PNGs. There was MNG (Multiple-image Network Graphics, which…

APNG is used heavily in the gaming (slot machine) industry.

Really? How curious. Why APNG? I would have assumed slot machines would want specific control over their animations and so would use something actually designed for that sort of control.

Re: Twitter's GIF hack

#157

Post by feld »

eridius wrote: post #156
feld wrote: post #155

Earlier quoted context omitted.

APNG is used heavily in the gaming (slot machine) industry.

Really? How curious. Why APNG? I would have assumed slot machines would want specific control over their animations and so would use something actually designed for that sort of control.

No idea, but when I worked in a casino for several years you could see the APNG files being loaded during boot.

Re: Twitter's GIF hack

#158

Post by cookiecaper »

callahad wrote: post #129
cookiecaper wrote: post #123

Earlier quoted context omitted.

Yes, until we get client-side transcoding, it is that difficult. Video files, even files compressed with modern codecs, are really big, and the concepts are generally really hard for most people to grok. The knowledge is so arcane that if you have a basic grasp of ffmpeg you are already a wizard. Most people's eyes will glaze over the second you start talking about how they should use a different container, or codec,…

> Video files, even files compressed with modern codecs, are really big, and the concepts are generally really hard for most people to grok. As the article points out, the video files are generally much smaller than the source gifs. Similarly, while there are real impediments to transcoding for many developers, I'm confident that the Twitter engineers in this specific case are capable of building a VP8 pipeline. Afte…

Yes, in this specific case, application-specific compression codecs are better for the storage of video, but that doesn't mean that video is small and that you can afford to transcode 4-8 different copies for every single image that would be uploaded (avail resolutions * formats * single piece of content), as that requires both a lot of CPU and a lot of disk space. It also doesn't mean any person around you has the expertise to figure out how to do this in a semi-reasonable manner, because as stated, most people don't understand the concepts used in modern video storage. You also assume that either WebM or H.264 will be supported by the user's browser. I don't think this is entirely correct, and in any case, it may always change, at which point the demand on the individual site becomes that much worse.

I think the real solution is to automate this and allow the client to request on-the-fly transcodes to the formats the browser can support, similar to the way some UPnP servers work.

Re: Twitter's GIF hack

#159

Post by cookiecaper »

sillysaurus3 wrote: post #99
Sir_Cmpwn wrote: post #77

I made a site that drove this innovation, MediaCrush. It's open source. https://github.com/MediaCrush/MediaCrush if you want to try to do this yourself, our code is a good reference. https://mediacru.sh

Why did Gfycat win? I've always wondered.

I think it's probably too early to call Gfycat the "winner", and I don't necessarily see why there isn't room for multiple providers. There's no reason image sharing needs to be a "winner-take-all" game; in fact, it behooves everyone if there is a bit of diversity in the space. I understand that reddit (the organization, not only the community) likes to aggressively favor single image sharing vendors, and that sucks, but surely there are other places where such a site can find usage.

Re: Twitter's GIF hack

#160

Post by cookiecaper »

skrowl wrote: post #67
discreditable wrote: post #66

Earlier quoted context omitted.

In a way they're doing it better. Twitter only seems to support MP4 whereas gfycat et al will deliver WEBM or MP4 based on the user's browser.

Agreed. MP4 doesn't work on Firefox (out of the box).

If you have a H.264 decoder installed (and most computers do), new-ish versions of Firefox will try to use it to play embedded H.264 videos. Unfortunately, a lot of sites are still in "Firefox == Flash" mode and automatically serve up the Flash version to Fx users without even trying a tag.
Post reply on HN