Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The limits of "computational photography"

yager.io

Post reply on HN
Print view View original

141–150 of 223 posts

Re: The limits of "computational photography"

#141

Post by foldr »

account42 wrote: post #65

> For example, by stitching together multiple dark images to try to make a brighter one. (Dedicated cameras tend to have better-quality but conceptually similar options like long exposures with physical IS.) However, we are starting to introduce the core sin of modern computational photography: imposing a prior on the image contents. In particular, when we do something like stitch multiple images together, we are mak…

>In particular, when we do something like stitch multiple images together, we are making an assumption: the contents of the image have moved only in a predictable way in between frames

From this way of looking at things a normal long exposure also imposes a prior assumption (that nothing is moving). It's just that we're used to the artefact that's generated when this prior isn't true (motion blur).

Re: The limits of "computational photography"

#142

Post by wyager »

epicycles33 wrote: post #13

Interesting article. I think the real question however is whether imposing complex priors (say driven by a neural net) makes images better _on average_ even if has some failure modes. My guess would be that a fairly weak prior trained on a diverse enough dataset would lead to better average image quality(as judged by everyday people in diverse scenarios) and that's why they are used.

I think it could absolutely make images better on average, assuming their prior is at all representative. The question, though, is whether the expected benefit to the photographer in cases where it improves the image (usually somewhat small) outweighs the costs when it screws up (perhaps relatively large).

Now, are most people going to notice that the iPhone wrecked the text on their subject? Probably not. But they probably also wouldn't notice if the model wasn't applied to the image at all. The median consumer probably mostly benefits from (in terms of how much they like the photo) AE, a bit of curve reshaping (using a smoothed histogram CDF algorithm or something), and maybe some extra saturation.

Re: The limits of "computational photography"

#144

Post by foldr »

wyager wrote: post #139
account42 wrote: post #65

> For example, by stitching together multiple dark images to try to make a brighter one. (Dedicated cameras tend to have better-quality but conceptually similar options like long exposures with physical IS.) However, we are starting to introduce the core sin of modern computational photography: imposing a prior on the image contents. In particular, when we do something like stitch multiple images together, we are mak…

Yes, I meant to imply that dedicated cameras are committing the same "sin" here. I only meant physical IS is better because you don't need to do things like periodically read out the sensor. You are getting a true full-duration exposure that won't produce artifacts like tearing or skips.

The iPhone has physical IS too. It's very good. (Image stabilisation tends to work better for smaller sensors, as you have less weight to move around and less far to move.)

Re: The limits of "computational photography"

#145

Post by foldr »

CarVac wrote: post #93
onphonenow wrote: post #62

A very long article but I’m slightly confused. Is he using proraw? Can you not get unprocessed images from proraw? I’m not sure what the pipeline looks but I thought this type of situation was where pro raw was supposed to be used?

Afaik proraw has the same computational reconstruction, but none of the baked-in contrast, lighting, or color tweaks. This gives you a clean low-noise image with editing flexibility but it does have the flaws of deconvolution and stacking and AI denoising. Actual raw from a cell phone is insanely noisy and hideously soft from diffraction in the best of cases.

>Actual raw from a cell phone is insanely noisy and hideously soft from diffraction in the best of cases.

You can get single shot RAW output from Halide and other third party apps on iPhones. It's actually perfectly usable and not particularly noisy or soft. I haven't personally had any problems with the output of ProRAW (which applies far less aggressive sharpening than the standard JPEG processing). I'm pretty sure the photo in the article would have come out fine if it had been shot using ProRAW.

Re: The limits of "computational photography"

#146

Post by formerly_proven »

hilbert42 wrote: post #80

I accept that there is a place for computational/algorithmic photography but I remain deeply sceptical of its actual benefits (in its current incarnation), moreover my recent bad experiences with it have only strengthened my conviction. I have previously discussed having taken photos with a smartphone where certain objects within some images have been so modified by the processing algorithm as to be almost unrecogniz…

> 6. What's the future. On the hardware side we need better sensors with higher resolution and more sensitivity and improved optics (that, say, use metamaterials etc.). Such developments are on their way but don't hold your breath.

Two interesting developments here are the pixels in Starvis 2 sensors, which as a first afaik use a 2.5D structure to increase full-well capacity by a lot. And another, non-production sensor by Sony where they developed a self-aligning process and pixels are actually split in two layers, with the top layer only carrying the photodiode and the bottom layer entirely dedicated to the readout transistors. That's promising for lower readout noise and also for increasing full-well capacity.

Re: The limits of "computational photography"

#147

Post by foldr »

wyager wrote: post #143
kblev wrote: post #89

Is there a way to disable this extra processing?

Sort of - (I assume for PR/marketing reasons) Apple doesn't let apps get access to actual raw sensor data. It may be possible to skip the steps that are causing the most trouble here.

Why don't you just try shooting it ProRAW, then try shooting a single shot RAW using a third-party app such as Halide? The latter option certainly removes any fancy computational photography from the pipeline.

Re: The limits of "computational photography"

#148

Post by hef19898 »

mtrycz2 wrote: post #90
dusted wrote: post #46

Earlier quoted context omitted.

They already are. However, most photographers do not appreciate this type of distortion being applied to their images. At a glance, my samsung note 22 ultra takes better picture than my nikon d7500. At a glance. However, as soon as you want to actually DO anything to it, like, view it in any real detail, or on anything but a tiny screen, reality returns.. While the phone is absolutely fantastic for a quick snapshot,…

My old DSLR has a RAW+JPEG setting which will do both, I use it all the time.

My favorite setting as well, I can edit the best shots from RAW, but can share any shot as a JPEG. And those are usually not too bad anyway, sometimes I just stick with the camera JPEG. Only exception, and rarely enough, continous high speed shooting. Even in JPEG Fine, the buffer never runs full. In RAW it does so after appr. 10-13 shots.

Re: The limits of "computational photography"

#149

Post by throwanem »

psychphysic wrote: post #79
dusted wrote: post #46

Earlier quoted context omitted.

They already are. However, most photographers do not appreciate this type of distortion being applied to their images. At a glance, my samsung note 22 ultra takes better picture than my nikon d7500. At a glance. However, as soon as you want to actually DO anything to it, like, view it in any real detail, or on anything but a tiny screen, reality returns.. While the phone is absolutely fantastic for a quick snapshot,…

Similarly sometimes I just want to apply whatever ai magic elixir my phone does to the raw image from my DSLR. Messing around in lightroom etc is just not worth it for hundreds of shots I take per day sometimes.

If you're making the same or very similar edits to a lot of shots by hand, you don't have to do that. You can create a preset from one shot's edits, then apply it to as many others as you like in a single step.

Re: The limits of "computational photography"

#150

Post by dwringer »

causi wrote: post #130
dusted wrote: post #46

Earlier quoted context omitted.

They already are. However, most photographers do not appreciate this type of distortion being applied to their images. At a glance, my samsung note 22 ultra takes better picture than my nikon d7500. At a glance. However, as soon as you want to actually DO anything to it, like, view it in any real detail, or on anything but a tiny screen, reality returns.. While the phone is absolutely fantastic for a quick snapshot,…

I recently started collecting a few of the digital cameras I lusted after as a kid. It made me realize there's a huge population of cameras out there that would produce incredible photos if not for small details. Details like not having readily available batteries, or an odd cable or memory card limitation. The biggest one for me personally is CCD degradation in cameras that don't have a built in pixel mapping functi…

I have used a program called Pixel Fixer which can learn a profile of each of your cameras and directly process out the dead pixels in the RAW files in a batch operation.

It hasn't been updated in about 8 years so it doesn't necessarily support anything too recent, and is not open source, but I had some success with using it.

Post reply on HN