81–90 of 171 posts
Does the ML algorithm with the millions of images training set get run locally or as a remote service? When I use ML these days is it all hardcore data crunching by remote servers or is some of it running on my phone/laptop?
Earlier quoted context omitted.
It’s like systems that add zeros (floating point math) to the right of the decimal. 18.00000 is not the same as 18. Thoughts?
You probably want go with 0.1+0.2 != 0.3
1x1 0-0-0 (original)
1x2 0-0-1 (made up)
1x3 0-0-2 (original)
I like how he realizes the impact for pro cameras but doesn't highlight the elephant in the room: phone cameras. This means that soon for many people digital cameras outside of their smartphone will become an even more niche product.
This is already the case. I rarely have a need to take out my SLR - it's just too bulky to have a reason for it, unless I'm going on an adventure where photography is one of the or the main purpose. I've gone on hiking trips where my "challenge" was to only use my phone camera. It wasn't much of a challenge for landscapes.
Phone (imho) is for quick and dirty, not for a 'it's time to do proper photography'.
Earlier quoted context omitted.
Absolutely not. If there's not enough information available, there's not enough information, full stop. Plausible (i.e. "good looking" or "believable") results are not the same as actual data, which is why enhance wouldn't work on vehicle licence plates or faces for example. Sure, the result might be a plausible looking face or text, but it's still not a valid representation of what was originally captured. That's th…
No there certainly is a chance for ML to improve here. Let’s take the classic example of enhancing a blurry photo to get a license plate. Humans may not be able to see much in the blur, but an AI trained on many different highly down-res’d images could at least give you plausible outcomes using far less data than a human brain would be able to say anything with confidence. You wouldn’t hold it up as the absolute trut…
That's not the same as fabricating information, though. A blurry image still contains a whole bunch of information and correlation data that just isn't present in a handful of pixels.
This is not super-resolution, but something different entirely. Super-resolution would mean to produce a readable license plate from just a handful of pixels. That is an impossible task, since the pixels alone would necessarily match more than one plate.
The algorithm would therefore have to "guess" and the result will match something that is has been trained on (read: plausible), but by no means the correct one, no matter how many checks you run on a database.
To illustrate the point, I took an image of a random license plate, and scaled it down to 12x6 pixels. 4x super-resolution would bring it to 48x24 pixels and should produce perfectly readable results.
Here's how it looks (original, down-scaled to 48x24, and down-scaled to 12x6 pixels): https://pasteboard.co/JSu3WDU.png
The 48x24 pixel version could easily be upscaled to even make the state perfectly readable. A 4x super-resolution upscale of the 12x6 version, however, would be doomed to fail no matter what.
That's what I'm getting at.
Just for shits and giggles, here's the AI 4x super-resolution result: https://pasteboard.co/JSu7jkP.png
Edit: while I'm having fun with super-resolution, here's the upscaled result from the 48x24 pixel version: https://pasteboard.co/JSu9Qh6.png
I like how he realizes the impact for pro cameras but doesn't highlight the elephant in the room: phone cameras. This means that soon for many people digital cameras outside of their smartphone will become an even more niche product.
This is already the case. I rarely have a need to take out my SLR - it's just too bulky to have a reason for it, unless I'm going on an adventure where photography is one of the or the main purpose. I've gone on hiking trips where my "challenge" was to only use my phone camera. It wasn't much of a challenge for landscapes.
Well if you collapse the problem space to a single point that corresponds to a phone's standard field of view, then it won't be a problem...
But what if you wanted to catch a photo of a rare bird in flight at 500mm equivalent, or a surfer caught at 1/4000th of a second?
Does the ML algorithm with the millions of images training set get run locally or as a remote service? When I use ML these days is it all hardcore data crunching by remote servers or is some of it running on my phone/laptop?
The model is trained on Adobe servers and run locally on your device. The training of the model is much more processor intensive than actually utilizing the trained model, usually by multiple orders of magnitude.
So any model that can be trained for generic use (Eg. Not trying to deep fake my specific face) can presumably be run on local machines.
Thanks!
Hm. So I took the example image, upscaled by 200%, applied a sharpen filter (all in Paint.NET) and compared the result to the AI upscaled image. TBH, I couldn't see a difference. 2x upscaling isn't all that impressive to begin with (e.g. produce 4 pixels from 1) and can be done in fairly high quality using traditional non-learning algorithms. I'm much more impressed by 4x and 8x super-resolution. I'm really not sure…
I was expecting the upscaled image to have extra "invented" detail from the supposed ML, as I've seen elsewhere.
But looking at these upscaled images, there isn't any at all. There's no extra texture, nothing.
I can't find any difference at all, like you say, from just some bicubic interpolation with sharpening.
No jaw dropping here, unfortunately.
Even better comparisons are in the blog post for a competing product: https://www.pixelmator.com/blog/2019/12/17/all-about-the-new... (likely the same algorithm, but using a different training set, so results will be different from what Adobes product does).
It has comparisons with nearest neighbor, bilinear and Lanczos filters and uses a slider to make it easier to see the difference.
Papers on this task: https://paperswithcode.com/task/image-super-resolution
[1] https://www.pixelmator.com/blog/2019/12/17/all-about-the-new...
[2] https://www.pixelmator.com/blog/2020/09/15/pixelmator-photo-...
