Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unbounded High Dynamic Range Photography Using a Modulo Camera

media.mit.edu

Post reply on HN
Print view View original

51–60 of 98 posts

Re: Unbounded High Dynamic Range Photography Using a Modulo Camera

#51

Post by slr555 »

I am probably less technical than most of you but the upside of this technology (at the point it is real world ready) might be producing what I would call a RAW file that is RAW(ER). By that I mean the file would simply contain a wider range of scene information that could be accessed by Adobe Camera Raw or the like. For the class of photographers that have no interest in RAW the information would help the JPEG engine in the camera make a "better" informed JPEG (or whatever the standard is at that time). To me capturing more scene info is better than less assuming there are minimal penalties being paid elsewhere (file size, burst shooting etc).

As a side note, it seems to me that aperture and shutter speed would still be essential as they affect not only exposure but depth of field and subject motion. Perhaps a technique that allows proper exposure independent of fstop/shutter/ASA would allow exploration of extremely wide aperture with slow shutter but no need for ND filters.

In any event it will be interesting to see what happens.

Re: Unbounded High Dynamic Range Photography Using a Modulo Camera

#53

Post by mortenjorck »

dr_zoidberg wrote: post #10

> No more will photographers or even ordinary people have to fumble with aperture size and exposure length. This is a "computer scientists" understanding of photography, and this phrase alone can even be seen as "dangerous" by photographers. There's more to aperture size and exposure length than "how much light reaches the sensor", like focus, depth of field, motion blur and bokeh, to name a few, that is not aknowled…

I'm surprised no one in the comments so far has used the word "clipping", which is what the real breakthrough here is in. I've been doing street photography somewhat seriously for about five years now, and always shoot full-manual: I rarely have problems with motion blur because I control the shutter speed; I rarely have problems with focus because I use assisted manual; the only thing that gets me with any regularity is highlight or shadow clipping.

I often shoot with a histogram overlay which helps a lot, but there are still high-contrast scenes where I have to decide where I'm going to bias the exposure. I can usually tease out some more detail in Lightroom from the RAW file, but direct sunlight plus shadow, or a quick-draw moment where I fumbled the exposure to make the shot, often leave some clipped areas.

Not having to worry about clipping would be an amazing breakthrough, and I'd even accept some other limitations in a camera that could manage this.

Re: Unbounded High Dynamic Range Photography Using a Modulo Camera

#54

Post by paulmd »

Tloewald wrote: post #41
dr_zoidberg wrote: post #10

> No more will photographers or even ordinary people have to fumble with aperture size and exposure length. This is a "computer scientists" understanding of photography, and this phrase alone can even be seen as "dangerous" by photographers. There's more to aperture size and exposure length than "how much light reaches the sensor", like focus, depth of field, motion blur and bokeh, to name a few, that is not aknowled…

> This is a "computer scientists" understanding of photography, and this phrase alone can even be seen as "dangerous" by photographers. The actual headline of the article is actually more informative and less misleading than the subhead: "Unbounded High Dynamic Range Photography Using a Modulo Camera" In photographer's terms, they're referring to ending blown highlights (caused by filling the wells in a sensel) not s…

I identified the same issue with cycle time, but to me the biggest problem is the assumption of an accurate, sensitive, non-destructive-read sensor. They're assuming that they can charge a capacitor from a photodiode accurately, and also hook a voltage comparator (which is effectively the same as an ADC) to it constantly to get an accurate pulse to count.

Assuming you have an accurate sensor that can be read non-destructively, then none of this modulus stuff is necessary anyway. Read once to set the proper gain on the ADC, then read again to get the fine bits. That way you still get the "unbounded HDR", but you don't need an ADC for every single receptor site, and you don't waste space on the counter or comparator. Instead, all of that lives on the image processor where it's not going to cause heat noise.

That's such an incredibly enormous hand-wave to me that I feel I've got to be missing something. Why make it so complex with all the modulus stuff if we have a magic sensor?

Re: Unbounded High Dynamic Range Photography Using a Modulo Camera

#55

Post by Tloewald »

paulmd wrote: post #54
Tloewald wrote: post #41

Earlier quoted context omitted.

> This is a "computer scientists" understanding of photography, and this phrase alone can even be seen as "dangerous" by photographers. The actual headline of the article is actually more informative and less misleading than the subhead: "Unbounded High Dynamic Range Photography Using a Modulo Camera" In photographer's terms, they're referring to ending blown highlights (caused by filling the wells in a sensel) not s…

I identified the same issue with cycle time, but to me the biggest problem is the assumption of an accurate, sensitive, non-destructive-read sensor. They're assuming that they can charge a capacitor from a photodiode accurately, and also hook a voltage comparator (which is effectively the same as an ADC) to it constantly to get an accurate pulse to count. Assuming you have an accurate sensor that can be read non-dest…

And we haven't even gotten into issues like thermal noise... Who knows, it may be great, but right now it just looks like a press release :-)

Oh the 1/10,000,000th comes from assuming they need to reset 128 * 8000 times per second to merely equal current sensor performance before losing sensitivity to misc losses.

If they want one more bit of DR they need to cycle in half as much time, and so on. Similarly, if we generously assume they're only losing 50% efficiency to cycle time and real estate then double again.

Re: Unbounded High Dynamic Range Photography Using a Modulo Camera

#56

Post by paulmd »

Tloewald wrote: post #55
paulmd wrote: post #54

Earlier quoted context omitted.

I identified the same issue with cycle time, but to me the biggest problem is the assumption of an accurate, sensitive, non-destructive-read sensor. They're assuming that they can charge a capacitor from a photodiode accurately, and also hook a voltage comparator (which is effectively the same as an ADC) to it constantly to get an accurate pulse to count. Assuming you have an accurate sensor that can be read non-dest…

And we haven't even gotten into issues like thermal noise... Who knows, it may be great, but right now it just looks like a press release :-) Oh the 1/10,000,000th comes from assuming they need to reset 128 * 8000 times per second to merely equal current sensor performance before losing sensitivity to misc losses. If they want one more bit of DR they need to cycle in half as much time, and so on. Similarly, if we gen…

Also don't forget that you actually need to cycle significantly faster than just 128 * 8000 times per second, since presumably you can't integrate light into the capacitor while you're also zeroing it. Whatever goes into the capacitor while it's draining is lost, because it drains to zero.

Seems like a better approach would be to add a "gain pixel" to traditional sensors.

So imagine you've got your Bayer grid (or a layout that functions similarly), one of the pixels in the grid gets read out first. That one sets the gain for the ADC for the surrounding pixels. The downside is your iteration couldn't be linear and the gain control would be a lot more complex, but you could do it with technology that actually exists, and you wouldn't lose half of your vertical resolution by doing it a whole line at a time like Magic Lantern's dual-ISO mode.

You'd have to do fancy processing around high-contrast edges, but what else is new.

Re: Unbounded High Dynamic Range Photography Using a Modulo Camera

#57

Post by Tloewald »

paulmd wrote: post #56
Tloewald wrote: post #55

Earlier quoted context omitted.

And we haven't even gotten into issues like thermal noise... Who knows, it may be great, but right now it just looks like a press release :-) Oh the 1/10,000,000th comes from assuming they need to reset 128 * 8000 times per second to merely equal current sensor performance before losing sensitivity to misc losses. If they want one more bit of DR they need to cycle in half as much time, and so on. Similarly, if we gen…

Also don't forget that you actually need to cycle significantly faster than just 128 * 8000 times per second, since presumably you can't integrate light into the capacitor while you're also zeroing it. Whatever goes into the capacitor while it's draining is lost, because it drains to zero. Seems like a better approach would be to add a "gain pixel" to traditional sensors. So imagine you've got your Bayer grid (or a l…

All good points. (And my figure actually assumes 10x faster and no loss of time between clearing wells and capturing photons -- i.e. 90% of time is spent capturing photons and 10% is spent clearing the well with no downtime -- to be generous. And, again, just to equal current performance.)

The underlying idea is sound -- storing bits is a linear problem but storing photons is an exponential problem. Of course you're still going to have to dump exponential amounts of energy... (So forget the "unbounded" part.)

Re: Unbounded High Dynamic Range Photography Using a Modulo Camera

#58

Post by stevensanderson »

vlamingsjef wrote: post #14
Mithaldu wrote: post #3

What's with the completely useless thumbnail at the bottom? That said, the technological explanation sounds simple and solid. Why aren't cameras doing that already?

High resolution image here: http://web.media.mit.edu/~hangzhao/images/modulo_teaser.png

Note that this is a simulated example, not one actually taken with a modulo camera (if such a thing has been built yet).

That image of Tintern Abbey comes from this 2011 web page: http://people.csail.mit.edu/sparis/publi/2011/siggraph/addit...

Re: Unbounded High Dynamic Range Photography Using a Modulo Camera

#59

Post by paulmd »

Tloewald wrote: post #57
paulmd wrote: post #56

Earlier quoted context omitted.

Also don't forget that you actually need to cycle significantly faster than just 128 * 8000 times per second, since presumably you can't integrate light into the capacitor while you're also zeroing it. Whatever goes into the capacitor while it's draining is lost, because it drains to zero. Seems like a better approach would be to add a "gain pixel" to traditional sensors. So imagine you've got your Bayer grid (or a l…

All good points. (And my figure actually assumes 10x faster and no loss of time between clearing wells and capturing photons -- i.e. 90% of time is spent capturing photons and 10% is spent clearing the well with no downtime -- to be generous. And, again, just to equal current performance.) The underlying idea is sound -- storing bits is a linear problem but storing photons is an exponential problem. Of course you're…

Yeah I was thinking that too. No matter how tiny a capacitor, you have (tens of) millions of them on the sensor, one per photoreceptor, and you're charging and discharging them 100k-10M times per second. That's a lot of energy being accumulated and dissipated right next to the photoreceptors. Thermal noise is gonna be nuts on a modulus sensor, too.

No system is ever unbounded, period, unless you live in a world of frictionless spherical cows. At some point you run into issues caused by accuracy, energy, capacity, etc.

Re: Unbounded High Dynamic Range Photography Using a Modulo Camera

#60

Post by sctb »

darkmighty wrote: post #7

The title here is terrible because over-saturation is an effect of poor tone mapping, which is inevitable if you want to display an HDR image in a low dynamic range display. An accurate title would be the end of over- exposed images, but why not just keep the original?

We updated the submission title to the article's.
Post reply on HN