Live data from Hacker News

A reality bending mistake in Apple's computational photography

appleinsider.com

11–20 of 414 posts

Re: A reality bending mistake in Apple's computational photography

#11

So this wasn't just rolling shutter?

No, the difference is too large for it to be rolling shutter. (not sure if the rolling shutter is even vertical or horizontal). It seems like it takes a short video and stitches together the sections with the least motion blur in each frame.

Re: A reality bending mistake in Apple's computational photography

#12
Purists will always hate the idea of computational photography, but I love how well my iPhone captures handheld nighttime, low-light, and motion images. I’ve captured some photos at night that I never would have thought possible from a physically small sensor due to the laws of physics.

Is it 100% pixel perfect to what was happening at the time? No, but I also don’t care.

I’ve used HDR exposure stacking in the past. I’ve used focus stacking in the past for shallow depth of field. I’ve even played with taking multiple photos of a crowded space and stitching them together to make an image of the space without any people or cars. None of them are pixel perfect representations of what I saw, but I don’t care. I was after an image that captured the subject and combining multiple exposures gets the job done.

Re: A reality bending mistake in Apple's computational photography

#15

So this wasn't just rolling shutter?

No. Rolling shutter should typically be too fast for a shot like that. Much more likely it's a result of 3 photos taken at slightly different times and automatically composited. Appears to be a well known effect.

Re: A reality bending mistake in Apple's computational photography

#16

The article says that “The final composite image should be the best, most realistic interpretation of that moment.” But that doesn’t make any sense. If the there were three real people, rather than one person and two reflections, the algorithm would have created an image of a moment that never existed, stitching together different images taken at different times. The only difference is that we might not notice and mi…

Imagine such an image being used as evidence is a court case. E.g. showing someone pointing a gun at another person when that actually never happened.

Seems extremely unlikely. The phone isn’t going to make drastic alterations to reality. It’s just combining multiple exposures. It can’t make someone point a gun if they’re not pointing a gun. You could try to imagine a scenario where someone was moving extremely fast through the frame while someone was whipping a gun around at high speed in the same direction and the phone just happened to stitch two moments a fraction of a second apart such that they look slightly closer together, but to get to that point you’d still need someone pointing a gun in the same direction that someone is going.

It’s really hard to imagine a scenario where two exposures a fraction of a second apart could be stitched together to tell a completely false story, but maybe it exists. In that case, I suspect the lawyers would be all over this explanation to try to dismiss the evidence.

Re: A reality bending mistake in Apple's computational photography

#17
post #3

I had a related experience with my Pixel 7 a few months ago. In this picture of a deer: https://photos.app.goo.gl/qChwaw9C29WVdAmr6 If you zoom in, you'll note that .. everything at a detail level looks like an oil painting, especially the deer's face and the wall behind it. Very weird effect, and that's certainly not what the wall or deer actually look like. No filters applied. Computational photography is really aw…

That’s denoising. Those mosaic like patterns are more pleasing to the eye than a noisy photograph.

The laws of physics put a limit on how well a tiny phone lens and sensor can capture light. We’re not at the limit yet even though hardware is quite good. Still, you wouldn’t like if your phone spat out raw, noisy images everywhere without any processing (beyond what’s necessary to translate it to the right pixel grid and color space).

Re: A reality bending mistake in Apple's computational photography

#18

Earlier quoted context omitted.

Imagine such an image being used as evidence is a court case. E.g. showing someone pointing a gun at another person when that actually never happened.

Seems extremely unlikely. The phone isn’t going to make drastic alterations to reality. It’s just combining multiple exposures. It can’t make someone point a gun if they’re not pointing a gun. You could try to imagine a scenario where someone was moving extremely fast through the frame while someone was whipping a gun around at high speed in the same direction and the phone just happened to stitch two moments a fract…

It’s unlikely, true. But precisely that makes it so dangerous. If such a picture is presented as evidence in a murder case, the possibility that it is telling the wrong story will be discounted and someone may go to prison for the rest of their lives.

Re: A reality bending mistake in Apple's computational photography

#19

So this wasn't just rolling shutter?

No. A rolling shutter deals with different parts of the sensor being exposed at different times within one frame. It generally requires a much faster object (think airplane propeller or light with PWM). This is effectively multiple frames captured normally and a mistake in the algorithm stitching them into one frame.

Re: A reality bending mistake in Apple's computational photography

#20

So this wasn't just rolling shutter?

Rolling shutter could in principle explain this, but it’s probably too fast to capture these three poses which must be temporally rather far apart.

Not probably too fast - definitely too fast. Rolling shutter occurs over a single read-out of the sensor, so she'd have to have held all the three poses during the 1/100th of a second or so exposure for it to be possibly related to rolling shutter.
Post reply on HN