Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A reality bending mistake in Apple's computational photography

appleinsider.com

Post reply on HN
Print view View original

191–200 of 414 posts

Re: A reality bending mistake in Apple's computational photography

#191

Post by jahnu »

atomlib wrote: post #187
gnabgib wrote: post #14

Is maybe the original article[0] (referenced by this short recap piece) a better source? [0]: https://petapixel.com/2023/11/16/one-in-a-million-iphone-pho...

Why not link directly to Instagram instead then? https://www.instagram.com/p/CzPGNmJIebC/

Off topic, but does anyone know why when I click on the above link in Firefox the back history is gone as if it was opened in a fresh tab? IG doing sketchy stuff to discourage navigating back to where I came from? Or perhaps the Firefox Facebook container protecting me?

Re: A reality bending mistake in Apple's computational photography

#192

Post by yorwba »

atomlib wrote: post #187
gnabgib wrote: post #14

Is maybe the original article[0] (referenced by this short recap piece) a better source? [0]: https://petapixel.com/2023/11/16/one-in-a-million-iphone-pho...

Why not link directly to Instagram instead then? https://www.instagram.com/p/CzPGNmJIebC/

For one, the Instagram post says "Full story in my highlights (THE MIRROR)" but I can't figure out how to actually view those highlights. (Without creating an account at least.)

Re: A reality bending mistake in Apple's computational photography

#193

Post by [deleted] »

dahart wrote: post #154
autoexec wrote: post #99

Earlier quoted context omitted.

> computational photography does mean you can just push a button without thinking and get a clear picture that captures a memory. I'd argue that the women trying to take a photo of herself in her wedding dress did not get a clear picture that captures a memory. She got a very confusing picture that captured something which never happened. There are lots of great automatic camera features which are super helpful, but…

> don’t falsify events […] Cameras that outright lie […] AI generated monstrosity of what apple thinks Oh the irony of framing things (pun intended) so hyperbolically. Somehow it never seems to dawn on people that like to throw around the word ‘lie’ that they’re doing exactly what they’re complaining about, except intentionally, which seems way worse. Nobody sat down to say bwahahah let’s make the iphone create fake…

[deleted]

Re: A reality bending mistake in Apple's computational photography

#194

Post by vlovich123 »

wkat4242 wrote: post #178
astrange wrote: post #170

Earlier quoted context omitted.

Also, many more people buy phones than standalone cameras, so they can afford a lot more software R&D.

Well yes but a lot of that can be reused across devices I would guess.

Naw, astrange is correct. At peak in 2010, 121 standalone cameras were shipped worldwide. By 2021, that number was down to ~8 million. By comparison, 83 million smartphones are shipped each quarter (~1.4 billion for the year). Those kinds of economies of scale means there's more R&D revenue to sustain a larger amount of HW & SW innovation. Even though smartphone shipments should come down over the next few years as they incremental jump each year is minimal & the market is getting saturated, there's always going to be more smartphone shipments.

Individual camera vendors just can't compete as much and I don't think there was possibly anything they could have done to compete because of the ergonomics of a smartphone camera. The poor UX and lack of computational photography techniques doesn't matter as much because the pro market segment is less about the camera itself and more about the lenses / post-processing. Even professional shoots that use smartphones (as Apple likes to do for their advertising) ultimately tend to capture RAW + desktop/laptop post-processing when they can because of the flexibility / ergonomics / feature set of that workflow. The camera vendors do still have the advantage of physical dimensions in that they have a bigger sensor and lenses for DSLR (traditional point & click I think is basically dead now), but I expect smartphones to chip away at that advantage through new ways of constructing lenses / sensors. Those techniques could be applied to DSLRs potentially for even higher quality, but at the end of the day the market segment will just keep shrinking as smartphones absorb more use-cases (interchangeable lenses will be the hardest to overcome).

Honestly, I'm surprised those device manufacturers haven't shifted their DSLR stack to just be a dumb CMOS sensor, lens, a basic processor for I/O, and a thunderbolt controller that you slot the phone into. Probably heat is one factor, the amount of batteries you'd need would go up, the BOM cost for that package could be largely the same, & maybe the external I/O isn't quite yet fast/open enough for something like that.

Re: A reality bending mistake in Apple's computational photography

#195

Post by TeMPOraL »

autoexec wrote: post #55
Aurornis wrote: post #12

Purists will always hate the idea of computational photography, but I love how well my iPhone captures handheld nighttime, low-light, and motion images. I’ve captured some photos at night that I never would have thought possible from a physically small sensor due to the laws of physics. Is it 100% pixel perfect to what was happening at the time? No, but I also don’t care. I’ve used HDR exposure stacking in the past.…

> Is it 100% pixel perfect to what was happening at the time? No, but I also don’t care. No photographer thinks images the they get on film are perfect reflections of reality. The lens itself introduces flaws/changes as does film and developing. You don't have to be a purist to want the ability to decide what gets captured or to have control over how it looks though. Those kinds of choices are part of what make photo…

> No photographer thinks images the they get on film are perfect reflections of reality. The lens itself introduces flaws/changes as does film and developing.

Don't fall into this trap. A lens and computational photography are not alike. One is a static filter, doing simple(ish) transformation of incoming light. The other is arbitrary computation operating in semantic space, halfway between photography and generative AI. Those are qualitatively different.

Or, put another way: you can undo effects of a lens, or the way photo was developed classically, because each pixel is still correlated with reality, just modulo a simple, reversible transformation. It's something we intuitively understand, which is why we often don't notice. In contrast, computational photography decorrelates pixels from reality. It's not a mathematical transformation you can reverse - it's high-level interpretation, and most of the source data is discarded.

Is this a big deal? I'd say it is. Not just because it rubs some the wrong way (it definitely makes something no longer be a "photo" to me). But consider that all camera manufacturers, phone or otherwise, are jumping on this bandwagon, so in a few years it's going to be hard to find a camera without built-in image-correcting "AI" - and then consider just how much science and computer vision applications are done with COTS parts. A lot of papers will have to be retracted before academia realizes they can no longer trust regular cameras in anything. Someone will get hurt when a robot - or a car - hits them because "it didn't see them standing there", thanks to camera hardware conveniently bullshitting them out of the picture.

(Pro tip for modern conflict: best not use newest iPhones for zeroing in artillery strikes.)

Ultimately you're right, though: this is an issue of control. Computational photography isn't bad per se. It being enabled by default, without an off-switch, and operating destructively by default (instead of storing originals plus composite), is a problem. It wasn't that big of a deal with previous stuff like automatic color corrections, because it was correlated with reality and undoable in a pinch, if needed. Computational photography isn't undoable. If you don't have the inputs, you can't recover them.

Re: A reality bending mistake in Apple's computational photography

#196

Post by lukeschlather »

nucleardog wrote: post #135
autoexec wrote: post #76

Earlier quoted context omitted.

For decades people have had the ability to get great photos using cell phones that included useful features like automatically adjusting focus, or exposure, or flash all without their phones inventing total misrepresentations of what the camera was pointed at. I mean, at a certain point taking a less than perfect photo is more important than getting a fake image that looks good. If I see a pretty flower and want to t…

The case in the article is obviously one that any idiot could have taken without these tools. But "in difficult scenarios", as the GP comment put it, your mistake is assuming people have been taking those photos all along no problem. They have not. People have been filling their photo albums and memory cards up with underexposed blurry photos that look more like abstract art than reality. That's where this sort of te…

In circumstances where it really matters having a prettied up image might be worse than having no image at all. If you rely on the image being correct to make some consequential decision, you could convict someone of a crime, or if you were trying to diagnose some issue with some machine you might cause damage. While if the camera gave an honest but uninterpretable picture you would be forced to try again.

Re: A reality bending mistake in Apple's computational photography

#197

Post by edent »

TeMPOraL wrote: post #184

Tangent, but: > A U.K. comedian and actor named Tessa Coates Why is it always some kind of celebrity that "discovers" stuff like this? Was it luck (yet again), or are those extreme failure modes of computational photography already somewhat known, just too nerdy to report on until they can be attached to a public person?

Because the typical Instagram account is only followed by a handful of people. So anything unusual is unlikely to get any traction.

Re: A reality bending mistake in Apple's computational photography

#198

Post by Kye »

amluto wrote: post #158
autoexec wrote: post #155

Earlier quoted context omitted.

> Not in these light conditions. Again, decades of people photographing themselves in wedding dresses while in dress shops (which tend to be pretty well lit) would disagree with you. Also, the things that help most with lighting (like auto-exposure) aren't the problem here. That's not why her arms ended up in three different positions at once.

> Apple's horrible tech featured in the article had nothing to do with the lighting. Of course it did. iPhones take an “exposure” (scare quotes quite intentional) of a certain length. A conventional camera taking an exposure literally integrates the light hitting each sensor pixel (or region of film) during the exposure. iPhone do not — instead (for long enough exposures), iPhones take many pictures, aka a video, and…

This is probably harder for people to believe if they've never seen a progress video of stacking long exposures.

I've seen the kinds of images people get out of stacking images for astrophotography. Individually, the images are mostly noise. Put enough together and you get stuff like this: https://web.archive.org/web/20230512210222/https://imgur.com... (https://old.reddit.com/r/astrophotography/comments/3gx29m/st...)

The phone is operating under way less harsh conditions since there's usually quite a bit of light even in most night scenes.

The iPhone is actually too good at this. You can't do light trails: it over-weights the first image and removes anything too divergent when stacking, so you get a well-lit frozen scene of vehicles on the road. I can get around it shooting in burst mode and stack in something like Affinity Photo, but that's work.

Re: A reality bending mistake in Apple's computational photography

#199

Post by jack_pp »

andrei_says_ wrote: post #189
frogblast wrote: post #68

Earlier quoted context omitted.

> In the end, this tech just takes control from you. In these difficult scenarios, the alternative photo I'd get using such a small camera without this kind processing would be entirely unusable. I couldn't rescue those photos with hours of manual edits. That may be "in control", but it isn't useful.

Is an argument about corner cases where computational improvement does make sense an argument in good faith when used as an excuse to take away control in all cases?

Afaik there are photo apps on the iphone that let you have all the control you want so you can use those when you want control and use the default when you just want a quick photo knowing the pros/cons.

Re: A reality bending mistake in Apple's computational photography

#200

Post by eproxus »

wkat4242 wrote: post #142
FalconSensei wrote: post #138

Earlier quoted context omitted.

> The SW backing it is much more powerful than what you get in standalone cameras that is very due to camera makers not giving a sh*t though. Maybe now they do, but it's overdue. And I'm not even talking about the phone apps for connection, or image processing, but more on a user interface and usability PoV, and also interesting modes for the user. Fuji and Ricoh are the only ones that I see are trying to make things…

I don't think it's only that. The processing in a phone is expensive. If you'd build a high end camera with a Snapdragon 8 gen 3 it would raise the price a lot. In a phone that's not an issue because you need the chip there anyway for other tasks. So there's just much more compute power available, not to mention a connection to the cloud with even more potential compute.

Would be cool if there ever were a product like a large external sensor and lens hardware dongle for phones (essentially all parts of a system camera without the computer) that would use the phone as a software processing platform. They would “just” ship the raw data over to the phone for post processing.
Post reply on HN