Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A reality bending mistake in Apple's computational photography

appleinsider.com

Post reply on HN
Print view View original

251–260 of 414 posts

Re: A reality bending mistake in Apple's computational photography

#251

Post by TeMPOraL »

jcynix wrote: post #239
TeMPOraL wrote: post #195

Earlier quoted context omitted.

> No photographer thinks images the they get on film are perfect reflections of reality. The lens itself introduces flaws/changes as does film and developing. Don't fall into this trap. A lens and computational photography are not alike . One is a static filter, doing simple(ish) transformation of incoming light. The other is arbitrary computation operating in semantic space , halfway between photography and generati…

While I agree with the gist of your comment, I cannot leave this detail uncommented >Or, put another way: you can undo effects of a lens, or the way photo was developed classically, because each pixel is still correlated with reality, You cannot undo each and every effect. Polarizing filters (filters, as are lens coatings, are part of a classical lens in my opinion), gradual filters, etc effectively disturb this corr…

> You cannot undo each and every effect.

Fair enough.

> Polarizing filters

Yeah, I see it. This one is as pure signal removal as it comes in analog world. And they can, indeed, drop significant information - not just reflections, but also e.g. by blacking out computer screens - but they don't introduce fake information either, and lost information could in principle be recovered -- because in reality, everything is correlated with everything else.

> But yes, I agree that computational photography offers a different kind of reality distortion.

A polarizing filter or choice of photographic paper won't make e.g. shadows come out the wrong way. Conversely, if you get handed a photo with wrong shadows, you not only can be sure it was 'shopped, but could use those shadows and other details to infer what was removed from the original photo. If you tried the same trick with computational photograph, your math would not converge. The information in the image is no longer self-consistent.

That's as close as I can come up to describing the difference between the two kinds of reality distortion; there's probably some mathematical framework to classify it better.

Re: A reality bending mistake in Apple's computational photography

#252

Post by zimpenfish »

callalex wrote: post #164
j16sdiz wrote: post #64

Earlier quoted context omitted.

Without better evidence, the court will keep accepting them. Just look at how bad fingerprints are accepted

See also: polygraphs.

See also: bite mark evidence[0], blood splatter[1], etc.[2]

[0] https://www.theguardian.com/us-news/2022/apr/28/forensics-bi...

[1] https://features.propublica.org/blood-spatter/blood-spatter-...

[2] https://www.currentaffairs.org/2023/01/we-need-to-get-junk-s...

Re: A reality bending mistake in Apple's computational photography

#253

Post by TeMPOraL »

mplewis wrote: post #177
autoexec wrote: post #99

Earlier quoted context omitted.

> computational photography does mean you can just push a button without thinking and get a clear picture that captures a memory. I'd argue that the women trying to take a photo of herself in her wedding dress did not get a clear picture that captures a memory. She got a very confusing picture that captured something which never happened. There are lots of great automatic camera features which are super helpful, but…

Without the iPhone’s computational camera, she wouldn’t have this photo at all because she wouldn’t have a camera in her pocket that could get a good picture in this situation.

On the contrary, she would have a perfectly good phone with a perfectly good camera making perfectly good pictures that don't fake reality and turn her into a vampire.

Re: A reality bending mistake in Apple's computational photography

#254

Post by mirsadm »

mplewis wrote: post #177
autoexec wrote: post #99

Earlier quoted context omitted.

> computational photography does mean you can just push a button without thinking and get a clear picture that captures a memory. I'd argue that the women trying to take a photo of herself in her wedding dress did not get a clear picture that captures a memory. She got a very confusing picture that captured something which never happened. There are lots of great automatic camera features which are super helpful, but…

Without the iPhone’s computational camera, she wouldn’t have this photo at all because she wouldn’t have a camera in her pocket that could get a good picture in this situation.

A 10 year old smartphone would have taken a perfectly good photo in that scenario. There's nothing challenging about it.

Re: A reality bending mistake in Apple's computational photography

#255

Post by foldr »

kazinator wrote: post #209
eproxus wrote: post #190

Earlier quoted context omitted.

No lens is going to reproduce a perfectly straight line without lens compensation either. Lenses are bending light to distill a moment in space time into a 2D representation which by definition will always be imperfect. What counts as “photography” really? GP> In the end, this tech just takes control from you. In the end, any tech is always going to take control away from you one way or another. That’s the whole poin…

Though it may be that no lens will reproduce a perfectly straight line, the transformation which warps the line is a relatively simple and invertible mapping which preserves local as well as global features. It might not be a conformal mapping but, say, if the image of a grid is projected through the lens, the projected image will have the same number of divisions and crossings, in the same relation to each other. We…

There’s no AI transformation altering her body posture. It’s just that the phone is blending multiple exposures during which she was posing in different ways. The phone doesn’t realize that the images of the women in the mirror correspond to the woman in the center, so it hasn’t felt it necessary to use the same keyframe as the reference for the different parts of the photo. People are jumping to crazy explanations for this when there is a much simpler explanation that’s obvious to anyone who’s ever done a bit of manual HDR.

Ironically, the solution to this problem is more AI, not less. The phone needs to understand mirrors in order to make smarter HDR blending decisions.

HN has a strangely luddite tendency when it comes to photography. Lots of people are up in arms about phones somehow ruining photography by doing things that you could easily do in a traditional darkroom. (It's not difficult to expose multiple negatives onto the same sheet of photo paper.)

Re: A reality bending mistake in Apple's computational photography

#256

Post by dsego »

kazinator wrote: post #147
autoexec wrote: post #55

Earlier quoted context omitted.

> Is it 100% pixel perfect to what was happening at the time? No, but I also don’t care. No photographer thinks images the they get on film are perfect reflections of reality. The lens itself introduces flaws/changes as does film and developing. You don't have to be a purist to want the ability to decide what gets captured or to have control over how it looks though. Those kinds of choices are part of what make photo…

No lens is going to produce a straight arm out of one that was bent at the elbow, or vice versa, while not disturbing anything else. It is not "photography".

If you use long exposure, you will capture a blurry blob instead. I don't know about you but I don't see blurry hands with my eyes (mostly because our brain also does computational photography).

Re: A reality bending mistake in Apple's computational photography

#257

Post by Dah00n »

astrange wrote: post #172
easygenes wrote: post #63

Earlier quoted context omitted.

Phones over-beautifying faces by default already happened with the iPhone XS, and it wasn't received well. See #beautygate: https://www.imore.com/beautygate

"Over-beautifying" never happened. Noise reduction/smoothing happens in camera processing even if you don't try to do it, because it's trying to preserve signal and that's not signal. If you want that noise, you have to actually put in processing that adds it back.

Yes, it actually did. And as it says in the link:

>".... if they can detect and preserve fabric, rope, cloud, and other textures, why not skin texture as well?"

The phone changed the image on skin and not on random other things that wasn't a face. That is a filter, not normal image processing, when it happens only on X but not on Y. The only way to not get this filter on the phone is using RAW.

Re: A reality bending mistake in Apple's computational photography

#258

Post by PUSH_AX »

ClumsyPilot wrote: post #235

So many comments, not a single one pointing out that someone will go to prison over this 'photo', when it shows something that never actually happened.

We've been able to create images and video of things that didn't happen for decades.

Re: A reality bending mistake in Apple's computational photography

#259

Post by TeMPOraL »

cesaref wrote: post #246
Aurornis wrote: post #12

Purists will always hate the idea of computational photography, but I love how well my iPhone captures handheld nighttime, low-light, and motion images. I’ve captured some photos at night that I never would have thought possible from a physically small sensor due to the laws of physics. Is it 100% pixel perfect to what was happening at the time? No, but I also don’t care. I’ve used HDR exposure stacking in the past.…

It's best to just think of it as a different art form. B&W film photography + darkroom printing is an art form, as is digital photography + photoshop. These modern AI assisted digital photography methods are another art form, one with less control left to the photographer, but there's nothing inherently wrong with that. I wouldn't want to say which is better, it's not really an axis that you can use to compare art is…

The mistake is in thinking photography is only, or even primarily, art. It's also measurement. A recorded observation of reality.

People use their cameras - especially phone cameras - for both these purposes, and often which one is needed is determined after the fact. So e.g. I might appreciate the camera touching up my selfie for the Instagram post today, but if tomorrow I discover a rash on my face and I want to figure out how long it was developing, discovering that all my recent selfies have been automatically beautified would really annoy me.

Or, you know, just try being a normal person and make a photo of your kid's rash to mail/MMS to a doctor, because it's a middle of a fucking pandemic, your pediatrician is only available over the phone, and now the camera plain refuses to make a clear picture of the skin condition, because it knows better.

I'm also reminiscing about that Xerox fiasco, with copy machines that altered numbers on copied documents due some over-eager post-processing. I guess we'll need to repeat that with photos of utility meters, device serial numbers, and even scans of documents (which everyone does with their phone camera today) having their numbers computationally altered to "look better".

EDIT:

Between this and e.g. Apple's over-eager, globally-enabled autocorrect at some point auto-incorrecting drug names in medical documents written by doctors, people are going to get killed by this bullshit, before we get clear indication and control over those "magic" features.

Re: A reality bending mistake in Apple's computational photography

#260

Post by dsego »

TeMPOraL wrote: post #216
autoexec wrote: post #120

Earlier quoted context omitted.

I agree. I'd have no issue with this if it were something people could disable and it was clearly disclosed and explained as the default.

Doesn't even need to have an off-switch, if it would preserve the inputs that went into AI stitching magic, so that one could recover reality if/when they needed it.

I think you can with live photos, basically you get a short video included.
Post reply on HN