Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Content-aware image resizing in JavaScript

trekhleb.dev

Post reply on HN
Print view View original

21–30 of 68 posts

Re: Content-aware image resizing in JavaScript

#21

Post by crazygringo »

lbutler wrote: post #14

You can also apply this to each frame in a video for a rather interesting effect. My brother has a YouTube channel full of content-aware scaling videos: https://youtu.be/a8k3b-QNbhs

Wow, that's incredibly interesting!

My first reaction is how actor's faces look surprisingly like traditional caricatures that illustrators do -- e.g. shrinking foreheads and chins which are detail-light but keeping eyes and ears which are detail-heavy.

But my second thought is that the extreme jumpiness in frames occurs because each frame is processed separately. But if you considered each seam not to be a "jagged line" from point A on one edge to point B on the opposite edge of a single frame, but rather a "jagged plane" cutting through a series of frames -- all frames in a single shot -- you could eliminate the jumpiness entirely.

You might need to build a bit more flexibility into it to allow for discontinuities generated from object movement and camera panning, but I wonder if anyone's tried to do something like that?

Though I imagine it might be quite a lot of programming for a tool that might only ever be used as a kind of video filter for entertainment purposes -- I have a hard time imagining a cinematographer ever using it for serious purposes.

Re: Content-aware image resizing in JavaScript

#22

Post by awb »

Very cool for simple images like the demo ones provided. But images with detailed content don't resize well and are much worse than a naive resize.

Try: https://unsplash.com/photos/ZtRuoAKr9vM

Resize: 50% width, 70% height

The basketball hoop is heavily distorted, as is the court, the squares on the building and the 3 point line.

Re: Content-aware image resizing in JavaScript

#23

Post by gchamonlive »

awb wrote: post #22

Very cool for simple images like the demo ones provided. But images with detailed content don't resize well and are much worse than a naive resize. Try: https://unsplash.com/photos/ZtRuoAKr9vM Resize: 50% width, 70% height The basketball hoop is heavily distorted, as is the court, the squares on the building and the 3 point line.

That is explained in the section with the van Gogh painting. It is not like they are advertising the algorithm as a jack of all trades.

Re: Content-aware image resizing in JavaScript

#24

Post by colejohnson66 »

Amazing! I tried it with a schematic (practically all right angles), and it did an impressive job (until it ran out of room and decided to mess with the text). Of course, images work great too :)

I do have one question: I see this is based on RGB, but how good is a "seam carving" implementation using RGB compared to one based on a color space more like human vision (such as CIELAB)?

Re: Content-aware image resizing in JavaScript

#25

Post by ZephyrBlu »

SavantIdiot wrote: post #17
FpUser wrote: post #4

I wish there were more of such interesting and educational articles with great examples and explanations.

Yaz, Right??? One of the biggest complaints I have about HN is that it promotes really crappy "Look at me! I just learned a thing and wrote a 300 word blog doing a crappy job explaining it because I don't really get it but want to pad my CV..." This article is exceptional. Thank you OP.

Unfortunately HN is not immune from lowest common denominator content.

Re: Content-aware image resizing in JavaScript

#26

Post by CyberRabbi »

SavantIdiot wrote: post #17
FpUser wrote: post #4

I wish there were more of such interesting and educational articles with great examples and explanations.

Yaz, Right??? One of the biggest complaints I have about HN is that it promotes really crappy "Look at me! I just learned a thing and wrote a 300 word blog doing a crappy job explaining it because I don't really get it but want to pad my CV..." This article is exceptional. Thank you OP.

Tragedy of the commons. Private hacker communities tend to produce higher quality content.

Re: Content-aware image resizing in JavaScript

#27

Post by aeroheim »

ska wrote: post #13
purplecats wrote: post #10

Is there an opposite of this, where it will expand an image size?

That's more related to a problem known as infilling. Generally throwing out information is a lot easier than generating it. You can do some statistical things by sampling other points of the image into newly created "gap" but it will probably look bad if fully automated, at least on big changes. There's an obvious version of the algorithm in that direction. For one line "seam", it's easy enough, you just pull data fr…

Are you referring to image inpainting? I think that's what it's usually called (please correct me if I'm wrong though!)

Although I'm not really familiar with traditional algorithms for inpainting, I've seen some ML research do some stuff with it that I found to be really impressive.

One demo that really stood out to me was the following: https://shihmengli.github.io/3D-Photo-Inpainting/

The algorithm they describe is able to inpaint pixels AND depth information from existing RGB-D photos, enabling images to be viewed in 3d space and be used with parallax effects. Really cool stuff!

Re: Content-aware image resizing in JavaScript

#29

Post by ZephyrBlu »

ska wrote: post #8

You see lots of demos of this, I think because the algorithm is interesting but also pretty easy to implement. As an approach it seems to do a decent job with either very small changes (e.g. slight change of aspect ratio) or uninteresting images, but aesthetically the results seem bad on most interesting images; I suspect because targeting "low information" regions of the image removes tension that is needed. Often a…

That what I was amazed by. The algorithm is pretty ingenious.

Re: Content-aware image resizing in JavaScript

#30

Post by marzell »

lbutler wrote: post #14

You can also apply this to each frame in a video for a rather interesting effect. My brother has a YouTube channel full of content-aware scaling videos: https://youtu.be/a8k3b-QNbhs

Interesting. A couple thoughts.... this might be a lot smoother on cartoons since there's usually less colors and noise overall.

It seems as though there's an additional effect (for extra... effect) when they scream. Not sure if that is a natural result of the content of the visual scene being processed, or if there's some sort of audio input into the visual processing, or if they manually/intentionally applied some sort of parameter change (at 0:29 and 0:38 in the video) that causes the video to get all chaotic.

Post reply on HN