I remember first seeing content aware image scaling at Adobe MAX in Barcelona I think it was. We were completely dumbfounded by what was surely magic happening on stage. When they showed removing objects we just lost it, jaws on the floor and all. That was a fun conference.
I'm the author of this library. The algorithm permits to remove an object from the image, only it needs somehow to be localized. After localization it can be applied to that area a higher energy map, which means that part will be avoided by the seam carver. I will implement a face detection algorithm to automatically exclude faces to get not altered by the carver.
Out of curiosity, liblqr was not good enough for you? :)
It's super fun but I can't think of many real-world uses. Maybe in subtle, element-wise stretching and reshaping as part of more complex editing? Squeezing images is a huge no-no in any graphic design, and while content-aware resizing is a neat trick to avoid the most glaring problems, it's still image squeezing. You'd rather just crop the image or choose a different one.
> Squeezing images is a huge no-no in any graphic design Surely this is true in journalism. I find it hard to believe it's true in ad work.
I think the previous comment was about changing the aspect ratio. And while you'd rather crop or choose another image, content aware resizing is now the 3rd option
I am always blown away by this stuff, but it still applies some distortion to the image in a way that feels slightly unnatural.
It's similar to the effect that changing focal length on a camera has. eg. Wider angle lenses seem to squish far away objects and enlarge closer objects. However, this affects the entire image in a consistent way whereas changing focal length does not.
I wonder if this could be used to simulate focal length changes?
Happy to see this here! Seam carving is one of those neat little algorithms that hit a sweet spot, being a sum of a few moderately complex algorithms, comes from a readable paper, and gives very satisfying result. I had a lot of fun implementing it in undergrad. One thing that I always wondered is how Photoshop managed to make it so fast that you can resize in real-time. If n is the width or height of the image, then…
Well, there is this paper called "Real-time content-aware image resizing" , maybe that describes the method you are curious about? > In this paper, we present a more efficient algorithm for seam based content-aware image resizing, which searches seams through establishing the matching relation between adjacent rows or columns. We give a linear algorithm to find the optimal matches within a weighted bipartite graph co…
That's the one I implemented -- with a bit of low-level hackery, it could run fast enough to resize at ~5-8 fps on mobile back in 2013. But the result didn't look great on most images if you resized by more than ~15%. The technique works by calculating the "importance" field of the image once, and all the seams upfront. Then you can just number the seams from 1-n and remove the first m seams when you resize the image by m. The runtime is then O(n^2) instead of O(n^3). In contrast, with regular seam carving, you'd often update the "importance" field every time you remove a seam. The result is that the seams that get removed are less likely to introduce artifacts. I think it's a similar problem as the forward energy v.s. backward energy problem discussed elsewhere in the thread, but worse.
Happy to see this here! Seam carving is one of those neat little algorithms that hit a sweet spot, being a sum of a few moderately complex algorithms, comes from a readable paper, and gives very satisfying result. I had a lot of fun implementing it in undergrad. One thing that I always wondered is how Photoshop managed to make it so fast that you can resize in real-time. If n is the width or height of the image, then…
I also thought about augmenting this algorithm with the semantic information output of a neural network, but couldn't come up with a good way to generate training data for semantic segmentation which wouldn't distort objects. For example, if you consider an image of a roof, the roof pixels will all be semantically similar, but if you remove any of them just based on that, the regular structure will be distorted. Do y…
I know little about deep learning though, so I can't really comment on it.
For situations like the roof problem you describe, I think PatchMatch is more likely to work out well. The downside is that it's considerably harder to implement correctly than seam carving. http://gfx.cs.princeton.edu/pubs/Barnes_2009_PAR/patchmatch....
I wrote a version of this in javascript with a live demo you can play with at https://alexander.soto.io/seam-carving
Your JavaScript version is very easy to use, and I'm so glad I can upload my own images instead of just using examples! The algorithm itself looks like it needs face (and body) detection to be really useful, though. Example: My girlfriend and I on holiday in Australia. Original: http://peterburk.free.fr/seamCarving/lighthouse.jpg Shrunk: http://peterburk.free.fr/seamCarving/lighthouse_seamCarving....
Not to downplay the cool factor of a new implementation, Liquid Rescale (long-time GIMP plugin) also has protective masks, which prevent or delay the laying of seams in some parts of the image. Caire seems to intend to do a more automagical subset of that, with the "Face detection" todo, but the better short term solution is to accept masks directly.
Just saying, but GIMP had this for years with Liquid Rescale plugin (I think before even Photoshop): http://liquidrescale.wikidot.com/
Neither GIMP nor Photoshop are libraries ("just sayin'"). It sounds like there were other previous libraries: that would be more relevant of a callback.
Looks cool, will check out soon. I remember I created an image resize web application using seam carving during college for my software engineering project as I was fascinated by the algorithm. Not sure if in working state currently. https://github.com/pixation/pixation