Building a New Image Format

First off, the concept of using the intra-frame compressor of a video format to compress images is unsound. Video compression has constraints that don't apply in still-image compression, and you are dragging that baggage around for no reason. These constraints lead to my other two issues.

One of the biggest problems with JPEG is that block artifacts are often visible even at relatively high quality settings. Post-decoding deblocking algorithms have proven utterly inadequate for fixing this.

One simple and effective leap in mitigating block artifacts is to use larger block sizes. Modern computers are obscenely fast compared to when JPEG was introduced, and we could bump the block size up from 8x8 to 16x16 or even 32x32 without breaking a sweat. This reduces the number of pixels that are at block boundaries, and that means less block artifacts. WebP uses smaller blocks because it's a video format, not an image format.

Another way that you can avoid edge artifacts is to overlap the blocks so that the edge artifacts don't create discontinuities. This is called "time-domain alias cancellation" and it's used in the 1D case by every lossy audio codec you've ever used. It generalizes easily to the 2D case, and it would generalize easily to the 3D case, but for reasons that I don't understand because video compression is not really my field, nobody does video compression by taking the DCT of 3D blocks of video. Instead they use a more ad hoc method of motion compensation, which consists of brute force searching for where blocks in one frame have moved in the next frame. It's not clear how you'd reconcile that with overlapping blocks in the interframe compressor, so VP8 doesn't use overlapping blocks. And by extension, neither does WebM.

Beating JPEG with a new video format is not all that difficult. We have two decades of research between JPEG's introduction and today, and each of us routinely carries hardware in our pocket that would have been considered a major military asset in 1992. It isn't at all surprising that the key frame compressor of a modern video codec would beat JPEG by 25%, but we can do so much better than that. Once we manage to displace JPEG, we're going to be stuck with whatever displaces it for a long time. So we need to make sure we have the best format we can muster before we make that commitment.

I'm not sure what your reasoning is there, but it's incorrect. Nearest-neighbor looks really bad even when scaling down to half size.

Then we don't have to wait for browser makers to implement it in their browsers. You just add a one-line script tag to load webp.js, a script that scans your DOM for img tags with a src that ends with ".webp", and your website has webp support.

If browsers later support webp natively, you can just add a check to webp.js so it turns itself off when a webp-supporting browser is detected.

It's really too bad that Java in the browser never took off; its performance characteristics are far better for writing image decompressors than JS. But JS with a modern HTML5 Canvas running in a modern JS engine on a modern CPU is good enough for the purpose.

Animated images have several advantages:

-Almost all places that support images support also animated image

-Can be downloaded and reposted (as opposed to canvas)

-No worry about audio, there is no audio

-Seamless looping

-Auto play everywhere

-Variable frame rate

These features could be achieved in a video format, but how would you convince websites that support images to support video with these characteristics?

For example, I would never let people upload autoplaying, looping videos as their avatars or as part of their comment unless I could be 100% sure that there will be no audio. I would have to inspect the video for audio and I would need different handling for images and videos.