Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Content-aware image resizing in JavaScript

trekhleb.dev

Post reply on HN
Print view View original

61–68 of 68 posts

Re: Content-aware image resizing in JavaScript

#61

Post by SavantIdiot »

littleninja wrote: post #34
SavantIdiot wrote: post #17

Earlier quoted context omitted.

Yaz, Right??? One of the biggest complaints I have about HN is that it promotes really crappy "Look at me! I just learned a thing and wrote a 300 word blog doing a crappy job explaining it because I don't really get it but want to pad my CV..." This article is exceptional. Thank you OP.

> promotes really crappy "Look at me! I just learned a thing and wrote a 300 word blog doing a crappy job explaining it because I don't really get it but want to pad my CV..." This is a broad brush, are you sure the intent is always resume padding? Some folks (like me) write poorly but I find writing tests what I know (and shows me what I don’t). I share anyway so I can be corrected and learn more, and so others migh…

> Some folks (like me) write poorly but I find writing tests what I know (and shows me what I don’t)

That's fine, just don't have such a big ego you need to share your crap with the world unless you have something important to say. That's why when you try to google something to learn, you have to wade through pages and pages of half-baked crap: all the good stuff has been drowned out.

Re: Content-aware image resizing in JavaScript

#62

Post by crazygringo »

andersource wrote: post #44
crazygringo wrote: post #21

Earlier quoted context omitted.

Wow, that's incredibly interesting! My first reaction is how actor's faces look surprisingly like traditional caricatures that illustrators do -- e.g. shrinking foreheads and chins which are detail-light but keeping eyes and ears which are detail-heavy. But my second thought is that the extreme jumpiness in frames occurs because each frame is processed separately. But if you considered each seam not to be a "jagged l…

That's quite an insight! Actually the authors of the seam carving paper went on to do just that [0]. From the abstract: "We present video retargeting using an improved seam carving operator. Instead of removing 1D seams from 2D images we remove 2D seam manifolds from 3D space-time volumes. To achieve this we replace the dynamic programming method of seam carving with graph cuts that are suitable for 3D volumes." [0]…

Son of a gun, this is why I love HN. Thank you! And it turns out the results are shockingly good, far better than I expected. They have demo videos at:

https://faculty.idc.ac.il/arik/SCWeb/vidret/results/video_re...

My favorite is:

Original: https://faculty.idc.ac.il/arik/SCWeb/vidret/results/videos/w...

Narrowed: https://faculty.idc.ac.il/arik/SCWeb/vidret/results/videos/w...

Widened: https://faculty.idc.ac.il/arik/SCWeb/vidret/results/videos/w...

Just wow.

Re: Content-aware image resizing in JavaScript

#64

Post by ska »

aeroheim wrote: post #27
ska wrote: post #13

Earlier quoted context omitted.

That's more related to a problem known as infilling. Generally throwing out information is a lot easier than generating it. You can do some statistical things by sampling other points of the image into newly created "gap" but it will probably look bad if fully automated, at least on big changes. There's an obvious version of the algorithm in that direction. For one line "seam", it's easy enough, you just pull data fr…

Are you referring to image inpainting? I think that's what it's usually called (please correct me if I'm wrong though!) Although I'm not really familiar with traditional algorithms for inpainting, I've seen some ML research do some stuff with it that I found to be really impressive. One demo that really stood out to me was the following: https://shihmengli.github.io/3D-Photo-Inpainting/ The algorithm they describe is…

> Are you referring to image inpainting?

Yes, too late to edit but that's the more common name.

Re: Content-aware image resizing in JavaScript

#65

Post by andersource »

crazygringo wrote: post #62
andersource wrote: post #44

Earlier quoted context omitted.

That's quite an insight! Actually the authors of the seam carving paper went on to do just that [0]. From the abstract: "We present video retargeting using an improved seam carving operator. Instead of removing 1D seams from 2D images we remove 2D seam manifolds from 3D space-time volumes. To achieve this we replace the dynamic programming method of seam carving with graph cuts that are suitable for 3D volumes." [0]…

Son of a gun, this is why I love HN. Thank you! And it turns out the results are shockingly good, far better than I expected. They have demo videos at: https://faculty.idc.ac.il/arik/SCWeb/vidret/results/video_re... My favorite is: Original: https://faculty.idc.ac.il/arik/SCWeb/vidret/results/videos/w... Narrowed: https://faculty.idc.ac.il/arik/SCWeb/vidret/results/videos/w... Widened: https://faculty.idc.ac.il/arik/…

Gladly! And yeah, the results really are quite good. This is why I like optimization problems - if you can formally capture what you want as an objective, and if you can find a way to optimize it, you can get surprisingly good results. Of course these are two very big IFs...

Re: Content-aware image resizing in JavaScript

#66

Post by foota »

saganus wrote: post #60
foota wrote: post #28

There's an improvement to seam carving using something termed "forward energy", see: https://avikdas.com/2019/07/29/improved-seam-carving-with-fo...

I wonder if further research on this has been done. For example, what if some ML tagging mechanism is used to find the silhouette of interesting objects in the image (people, animals, traffics signs, etc), and then "freezing" them to prevent the energy function from operating on those areas, thus preserving those objects intact, while resizing the rest of the image.

I don't remember where I saw it linked from, but someone did that with face detection: https://github.com/esimov/caire

Re: Content-aware image resizing in JavaScript

#67

Post by HelloNurse »

gchamonlive wrote: post #23
awb wrote: post #22

Very cool for simple images like the demo ones provided. But images with detailed content don't resize well and are much worse than a naive resize. Try: https://unsplash.com/photos/ZtRuoAKr9vM Resize: 50% width, 70% height The basketball hoop is heavily distorted, as is the court, the squares on the building and the 3 point line.

That is explained in the section with the van Gogh painting. It is not like they are advertising the algorithm as a jack of all trades.

In this specific image there are even greater difficulties than in the Van Gogh painting: the unpainted wall portions on the left prevail over the "low energy" surfaces that surround them, and there are many rectilinear structures that demand very good preservation, like court lines, wall edges, iron bars, and worst of all the backboard, located in the middle of lower energy regions (the low contrast purple grid) that attract seams that bite off the corners of the backboard.

This image should look better with a strong penalty against seams that depart from vertical or horizontal lines, but it wouldn't be enough: the purple pillars and panels would be straighter but still squeezed.

Post reply on HN