Live data from Hacker News

The limits of "computational photography"

yager.io

181–190 of 223 posts

Re: The limits of "computational photography"

#181
Does anybody know what post-processing is applied to ProRaw images from iOS? I'm guessing true raw (Halide, etc) have none at all, but I recall reading the ProRaw had some applied. I just haven't seen a summary of which steps are applied.

90% of the time, my iPhone photos are fine straight out of the camera as HEIC. But every once in a while, I get something like is described here (or in several other recent similar articles).

Re: The limits of "computational photography"

#182

I would like to see computational photography applied to raw images from DSLRs and MILCs with APS-C and larger sensors. Perhaps Canon, Nikon, Sony, and Fujifilm could have built-in options in their cameras for ‘social media mode’, with a modicum of noise reduction (honestly unnecessary at ISOs lower than about 1600 for modern cameras), but drastically improved HDR and white balance. Many of these cameras are able to…

I've got an Olympus M43 camera, which has an image sensor that's both significantly smaller than full frame, and significantly larger than a phone. I've found that I can do absolutely amazing things to the RAW image later. There are these processes now that will use ML to remove noise, and it blows my mind every time. I can shoot at ISO 6400 now without even thinking about it. I used to cringe when I had to hit 1600.

Olympus (OM Systems?) should build this stuff into their cameras. I used to have a bit of inkling to some day "upgrade" to full frame, but not any more.

Re: The limits of "computational photography"

#183

Earlier quoted context omitted.

There is no reason to do this. If you want to apply the algorithms and Iphone uses, simply take a burst and post process later for HDR or whatever. I remember being in middle school and messing around with Hugin and my Minolta bridge camera… People value quick shots/edits and don’t care about quality or editing things later don’t mind an iPhone doing all this behind the scene - but it is irreversible. The sort of err…

There is no consumer software that can do what the iPhone does with its sensors. iPhone HDR produces an HDR image file. Consumer HDR apps do the opposite - they take an HDR raw and tone map an sRGB JPEG out of it. The only portable format that really supports HDR images is EXR, so if you're not generating that you're not getting it. (I don't think there's anything that can do deep fusion either, though obviously you…

> iPhone HDR produces an HDR image file

What file format is this? iPhone produces tone mapped HEIC, yes, but that’s not HDR as you pointed out.

Or do you mean HDR video?

Re: The limits of "computational photography"

#184

Does anybody know what post-processing is applied to ProRaw images from iOS? I'm guessing true raw (Halide, etc) have none at all, but I recall reading the ProRaw had some applied. I just haven't seen a summary of which steps are applied. 90% of the time, my iPhone photos are fine straight out of the camera as HEIC. But every once in a while, I get something like is described here (or in several other recent similar…

They are already demosaiced (this I know for sure and is easy to verify). I also believe that they are the result of stacking several photos to increase the dynamic range and reduce noise (see [0][1]).

Even for the "true raw" ones, I don't know if they're truly raw. Do they have distortion and light fall-off correction applied?

[0]: https://ai.googleblog.com/2021/04/hdr-with-bracketing-on-pix... [1]: https://dl.acm.org/doi/10.1145/3355089.3356508

Re: The limits of "computational photography"

#185

Earlier quoted context omitted.

There is no reason to do this. If you want to apply the algorithms and Iphone uses, simply take a burst and post process later for HDR or whatever. I remember being in middle school and messing around with Hugin and my Minolta bridge camera… People value quick shots/edits and don’t care about quality or editing things later don’t mind an iPhone doing all this behind the scene - but it is irreversible. The sort of err…

There is no consumer software that can do what the iPhone does with its sensors. iPhone HDR produces an HDR image file. Consumer HDR apps do the opposite - they take an HDR raw and tone map an sRGB JPEG out of it. The only portable format that really supports HDR images is EXR, so if you're not generating that you're not getting it. (I don't think there's anything that can do deep fusion either, though obviously you…

I don't think that's true. Capture One introduced HDR merging in one of their recent releases [0].

It does raw in "raw" out. You get a DNG out which is demosaiced. I have not yet looked to see what it's doing under the hood, and how the result compares to EXR. (But in my experience, it seems to struggle with large exposure differences, even when on a tripod).

[0]: https://support.captureone.com/hc/en-us/articles/44100147302...

Re: The limits of "computational photography"

#186

This looks like iPhone postprocessing problem more than anything else. I have used 3 Google pixel phones (up to pixel 4) and none of them does bad post-processing. In fact, it improves the resolution of whatever you are taking picture of https://ai.googleblog.com/2018/10/see-better-and-further-wit... I never saw any of these phones altering the details like in article.

Google Pixel phones still do "deep fusion" processing like an iphone, but instead with Google's secret sauce. The photo your phone is showing you is what machine learning thinks the picture should look like, and not the picture you took.

Re: The limits of "computational photography"

#187

Earlier quoted context omitted.

If you're making the same or very similar edits to a lot of shots by hand, you don't have to do that. You can create a preset from one shot's edits, then apply it to as many others as you like in a single step.

Yeah, Photoshop has actions too. See, photographers have been doing computational photography too, this whole time. It’s just that we prefer to put it together with a general purpose tool, rather than what formula Apple or Samsung has decided would be best.

Which is the point I failed to touch upon in my smartassed earlier comment about the macro rig. We use most of (maybe all) the same transformations in a tool like Lightroom, but prefer to compose them by hand precisely because the assumptions made by phone camera ISPs are not at all guaranteed to hold over the range of raw images we use our cameras to produce. (They don't even hold over the range of raw images people use phone cameras to produce!)

Re: The limits of "computational photography"

#188

Earlier quoted context omitted.

There is no consumer software that can do what the iPhone does with its sensors. iPhone HDR produces an HDR image file. Consumer HDR apps do the opposite - they take an HDR raw and tone map an sRGB JPEG out of it. The only portable format that really supports HDR images is EXR, so if you're not generating that you're not getting it. (I don't think there's anything that can do deep fusion either, though obviously you…

> iPhone HDR produces an HDR image file What file format is this? iPhone produces tone mapped HEIC, yes, but that’s not HDR as you pointed out. Or do you mean HDR video ?

No, the HEIC has a proprietary attachment with the HDR data as well. You can see the iPhone display adapting when you view it in Photos.

Re: The limits of "computational photography"

#189

I always shoot in Raw (using the Lightroom iPhone App) to make sure that this kind of defects never occur. Noise is generally preferable and acceptable that the disaster trail left by denoising et al. At least you can do it yourself in a way that pleases you instead of having a ruined photograph.

ProRAW appears to not be as processed, but still processed: "Apple ProRAW combines the information of a standard RAW format along with iPhone image processing" [1]

1: https://support.apple.com/en-gb/HT211965

Re: The limits of "computational photography"

#190
post #43

Earlier quoted context omitted.

as long as there is anything to key onto it's possible to remove the shake algorithmically

> as long as there is anything to key onto it's possible to remove the shake algorithmically Why the requirement? The phone already has accelerometers, doesn't it? In any case, less hake should still be easier for the phone to deal with. Those algorithms aren't flawless.

There's no way that the accelerators/gyrocscope would be accurate enough to remove the shake. According to google, the iphone gyroscope is accurate to within about 0.5 degrees, roughly two orders of magnitude away from being pixel accurate in a 1x zoom image.
Post reply on HN