Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adobe Photoshop's 'super resolution' made my jaw hit the floor

petapixel.com

Post reply on HN
Print view View original

161–170 of 171 posts

Re: Adobe Photoshop's 'super resolution' made my jaw hit the floor

#161

Post by atlantis_ »

ISL wrote: post #6

As a metrologist (and photographer), the difficulty with these techniques is that they can over-represent the information contained within an image; they present an image of what was "probably" there, rather than representing what was. These aren't so different from our own brains, which remember what we thought we saw, rather than the light that reached our retinas. These methods are already in extensive use (most s…

I have been using neural-enhance (gh:alexj) and Topaz tools to upscale PAL/NTSC artworks the last three years and would not be so judicious in describing these tools. They are hallucinating what the model assumes an upscaled image should look like and not enhancing in any way as the word is understood. The original image ceases to exist. A more honest term might be “Render As Upscaled” or “Generate Higher Resolution Image” (likewise “ML”, not “AI”).

When playing around funny things happen too: recursive upscale/sharpen and analogue artifacts begin resembling topography, molten metal etc.

Now imagine this being fitted into military drones, which it almost certainly is.

Re: Adobe Photoshop's 'super resolution' made my jaw hit the floor

#162

Post by NoPicklez »

dimatura wrote: post #148
mcrutcher wrote: post #70

This seems highly relevant as to what is actually going on: http://www.johncostella.com/magic/

That's an interesting article, and relevant in the sense that the "magic kernel" can be used for purposes of super-resolution, but Adobe is using a fairly different approach. Instead of using analytically-derived functions Adobe is using a deep learning model trained on a large dataset of Low resolution-High resolution image pairs. The details are propietary, obviously, but it's likely similar to various deep learnin…

This sounds like how nvidia have implemented Deep Learning Super Sampling (DLSS) into computer graphics cards for gaming. Allowing people to run at higher resolutions (e.g. 4k) and in some cases the image looks better than native.

Re: Adobe Photoshop's 'super resolution' made my jaw hit the floor

#163

Post by Sebb767 »

herendin2 wrote: post #129
Sebb767 wrote: post #122

Earlier quoted context omitted.

That was due to compression, however, not upscaling. Related, but not similar :)

Upscaling could be described as the decompression stage of a lossy compression algorithm

A compression algorithm knows which data was lost and can optimize the discarded data for a good lost data/saved space ratio. Data "lost" by a low resolution sensor most definitely does not fit this description. Imagine saving a FullHD png instead of a 4k jpg - the former is most likely far worse.

It's not too dissimilar, I agree, but there are differences.

Re: Adobe Photoshop's 'super resolution' made my jaw hit the floor

#165

Post by serguzest »

aembleton wrote: post #145
serguzest wrote: post #126

Possible practical application: can anyone upscale the beatles abbey road cover to a higher resolution please? I've always wanted it as a wall poster.

91.5cm x 61cm: https://www.ukposters.co.uk/posters/beatles-abbey-road-v6

thanks!

Re: Adobe Photoshop's 'super resolution' made my jaw hit the floor

#166

Post by qayxc »

nwienert wrote: post #150
qayxc wrote: post #84

Earlier quoted context omitted.

> Let’s take the classic example of enhancing a blurry photo to get a license plate. That's not the same as fabricating information, though. A blurry image still contains a whole bunch of information and correlation data that just isn't present in a handful of pixels. This is not super-resolution, but something different entirely. Super-resolution would mean to produce a readable license plate from just a handful of…

I was simply pointing out that AI enhancement to find details otherwise not possible by humans that could be useful/accurate is very possible, and I don’t think you refuted it.

I also never denied that. But there's a difference between finding details that would otherwise go amiss (i.e. in lieu of a microscope revealing features invisible to the unaided eye) and reproducing data that simply isn't there to begin with (as is the core of the "enhance"-trope).

Re: Adobe Photoshop's 'super resolution' made my jaw hit the floor

#167

Post by webmaven »

TeMPOraL wrote: post #111
neogodless wrote: post #106

Earlier quoted context omitted.

I did a web search for "cots" and learned that a cot is... > a small usually collapsible bed often of fabric stretched on a frame But in this case, COTS is apparently... > commercial, off-the-shelf In other words "photo equipment" or "consumer/retail photo equipment." Upon further reading[0] it seems odd to use the term here, but maybe I'm misunderstanding something. It's often used for software and has a key phrase.…

I use it in a way it's used in disciplines that also work with specialty-built, or even custom-built equipment. Such as science, military and some types of engineering (e.g. aerospace). The first sentence of the linked article describes it: "Commercial off-the-shelf or commercially available off-the-shelf[1] (COTS) products are packaged solutions[buzzword] which are then adapted to satisfy the needs of the purchasing…

What you're describing is edging close to the setup for the 2003 novel 'Blind Lake': https://en.m.wikipedia.org/wiki/Blind_Lake_(novel)

In the novel, quantum computers (rather than ML per-se) are tasked with interpolating more and more detailed data from astronomical observations, to the point that tracking individual members of an alien species on a distant world, underground, is possible. Eventually it is noticed that cutting off the astronomical data entirely doesn't interrupt the interpolated data. Then things get weird.

I won't go into further plot details, as that would be spoilery, but it is a pretty good book, reminiscent to me of Greg Egan's oeuvre (the novel is actually by Robert Charles Wilson).

Re: Adobe Photoshop's 'super resolution' made my jaw hit the floor

#168

Post by webmaven »

danaliv wrote: post #52
ISL wrote: post #6

As a metrologist (and photographer), the difficulty with these techniques is that they can over-represent the information contained within an image; they present an image of what was "probably" there, rather than representing what was. These aren't so different from our own brains, which remember what we thought we saw, rather than the light that reached our retinas. These methods are already in extensive use (most s…

> These aren't so different from our own brains, which remember what we thought we saw, rather than the light that reached our retinas. Never mind memories; there are parts of our eyes that aren’t responsive to light at all. We’re always hallucinating.

> Never mind memories; there are parts of our eyes that aren’t responsive to light at all. We’re always hallucinating.

Are you referring to the blind spot, or something else?

Interestingly, the blind spot turns out not to be a design requirement, it is a contingent feature that cephalopods like octopuses (whose eyes evolved independently from vertebrates') don't have.

Re: Adobe Photoshop's 'super resolution' made my jaw hit the floor

#169

Post by webmaven »

HenryBemis wrote: post #82
dejj wrote: post #43

Earlier quoted context omitted.

You probably want go with 0.1+0.2 != 0.3

I would say that it's like pixel's RGB at address 1x1 is 0-0-0 and pixel at address 1x2 is 0-0-2 and squeezing between them a pixel with color 0-0-1 (averaging the two values near it)(assuming doing this on a image that has 1 pixel height and e.g. 2 pixes width; so that the new image would be would be: 1x1 0-0-0 (original) 1x2 0-0-1 (made up) 1x3 0-0-2 (original)

What you're describing is relatively straightforward (bi)linear interpolation. It is worth noting that even at this relatively simple level, going with bicubic interpolation instead will usually give you nicer results, except in cases where the hard edges in the image are only horizontal or vertical.

Re: Adobe Photoshop's 'super resolution' made my jaw hit the floor

#170

Post by webmaven »

TimBurr wrote: post #103
Someone wrote: post #89

I suggest to read the Adobe blogpost at https://blog.adobe.com/en/publish/2021/03/10/from-the-acr-te... instead. It has sample images side-by-side with bicubic upsampling. Even better comparisons are in the blog post for a competing product: https://www.pixelmator.com/blog/2019/12/17/all-about-the-new... (likely the same algorithm, but using a different training set, so results will be different from what Adobes prod…

A few examples in the blog post stood out to me - both refine ambigous letters to clear ones, which demonstrates concerns other folks were talking about in this thread. In this one, i's are nearly illegible, but are corrected. It appears to be correct here, but could be wrong. https://blog.adobe.com/hlx_fa6ee5a67d3f2e5b1945745176e6b955e... This one's weirder. In the original, I can't read many of the letters. It look…

> A few examples in the blog post stood out to me - both refine ambigous letters to clear ones, which demonstrates concerns other folks were talking about in this thread.

Those images show an effect that looks similar to LCD subpixel rendering, which is an artifact of a scanner working at the limit of what it's sensor is capable of producing (typical CCDs have subpixel stripes (or arrays), just like LCD screens.

Scanners typically overcome this by oversampling and then downsampling the raw data to smooth out the effect. In theory you could also do this with less oversampling if you could manage to get the scanner to do subpixel offsets, and oversampling isn't needed at all if the CCD doesn't use striping or a Bayer array but instead layers the RGB detectors on top of each other, like the Foveon X3 CCD.

Anyway, it is pretty clear that the main benefit of the upsampling interpolation in these particular images is in correcting these subpixel color fringes. Downsampling back to the original resolution should still yield an improved image, which is quite intriguing.

Post reply on HN