Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Learning to See in the Dark (2018)

github.com

Post reply on HN
Print view View original

161–170 of 179 posts

Re: Learning to See in the Dark (2018)

#161

Post by y7r4m »

randyrand wrote: post #160
y7r4m wrote: post #141

Hi, I'm a developer at NexOptic[0] and we are a company that was deeply inspired by this paper when it was first published. We had a lot of early success when attempting to replicate the results on our own and ended up running with it, and extending it into our own product line under our ALIIS brand of AI powered solutions. For those curious, our current approach differs in some very significant ways to the author's…

It would be really cool if you could feed the network a photo with flash that it could use for gathering more information, but then recreated a photo without flash from the non-flash raw. Often flash is not the look people are going for, but would be okay with the flash firing in order to improve the non-flash photo.

Absolutely! We recently rebranded our AI solutions from ALLIS (Advanced Low Light Imagine Solution) to ALIIS (All Light Intelligent Imaging Solution) specifically because we are beginning to branch out to handle use cases such as this!

As a proof of concept that this task can be tackled directly, a quick search brought up "DeepFlash: Turning a Flash Selfie into a Studio Portrait"[0]

Beyond denoising, we are already running experiments with very promising results on haze, lens flare, and reflection removal; super resolution; region adaptive white balancing; single exposure HDR; and a fair bit more.

One of the other cooler things we are doing is putting together a unified SDK where our algorithms and neural nets will be able to run pretty much anywhere, on any hardware, using transparent backend switching. (e.g. CPU, GPU, TPU, NPU, DSP, other accelerator ASICs, etc..)

[0] https://arxiv.org/abs/1901.04252

Re: Learning to See in the Dark (2018)

#162

Post by anigbrowl »

barrkel wrote: post #131
anigbrowl wrote: post #128

Earlier quoted context omitted.

Of course, but what are the odds that the algorithm just lucked into the correct book title and other cover text? It doesn't have a dictionary or semantic network. You are right that the raw sensor data should always be preserved. But sticking with the license plate example, you could challenge a picture of a single car with a visible license plate far away in a wooded area, but it would be hard to refute a picture o…

Xerox used to replace numbers in documents while copying: https://www.theregister.co.uk/2013/08/06/xerox_copier_flaw_m... License plates are an ideal breeding ground for false enhancement owing to standardisation of appearance; an ML algo trained on lots of examples might, without due care, learn to replace as a well-known texture.

The pre-emptive certification I mention would be a validation of due care. It doesn't matter how many theoretical arguments you want to throw up against this, once there's sufficient empirical evidence for its reliability (and there will be) it will be accepted as evidence.

Also, y'all need to think more like prosecutors. Say you are dragged you into court on the basis of photos showing your car in the dark, and you object that the photo is from the ML 9000 security camera and it might be just imaging your license plate. The police/prosecutors will just 'borrow' your car and leave it there for a night and leave it up to the jury.

Forensic evidence can be and is regularly abused, but it can also be quite easily validated and it's massively persuasive to juries.

Re: Learning to See in the Dark (2018)

#164

Post by chias »

LukeShu wrote: post #89
chias wrote: post #60

Earlier quoted context omitted.

Interestingly, this effect is notably visible in their example image [0]. Notice the distinctly "plasticized" appearance of the book cover, and how the text is not intelligible in the low-contrast areas of the reflection. [0]: https://raw.githubusercontent.com/cchen156/Learning-to-See-i...

Note (a) and (b) are separate photographs (different angles and everything), and that (c) is based on (a), not (b); comparing the glare between (b) and (c) isn't quite an even comparison.

Oh gosh. Taking one second to think about it, _of course_ (a) and (b) are separate photographs -- that is the entire point of that diagram. Somehow my brain farted right over that when making my previous comment.

Thank you, not only for setting me straight, but also for doing so as kindly as you did.

Re: Learning to See in the Dark (2018)

#166

Post by barrkel »

anigbrowl wrote: post #162
barrkel wrote: post #131

Earlier quoted context omitted.

Xerox used to replace numbers in documents while copying: https://www.theregister.co.uk/2013/08/06/xerox_copier_flaw_m... License plates are an ideal breeding ground for false enhancement owing to standardisation of appearance; an ML algo trained on lots of examples might, without due care, learn to replace as a well-known texture.

The pre-emptive certification I mention would be a validation of due care. It doesn't matter how many theoretical arguments you want to throw up against this, once there's sufficient empirical evidence for its reliability (and there will be) it will be accepted as evidence. Also, y'all need to think more like prosecutors. Say you are dragged you into court on the basis of photos showing your car in the dark, and you…

I personally think juries are a great way to convict innocent people, and that adversarial court systems privilege people who can afford to pay for the best storyteller, so arguments from that direction start out hobbled.

Re: Learning to See in the Dark (2018)

#167

Post by exikyut »

randyrand wrote: post #160
y7r4m wrote: post #141

Hi, I'm a developer at NexOptic[0] and we are a company that was deeply inspired by this paper when it was first published. We had a lot of early success when attempting to replicate the results on our own and ended up running with it, and extending it into our own product line under our ALIIS brand of AI powered solutions. For those curious, our current approach differs in some very significant ways to the author's…

It would be really cool if you could feed the network a photo with flash that it could use for gathering more information, but then recreated a photo without flash from the non-flash raw. Often flash is not the look people are going for, but would be okay with the flash firing in order to improve the non-flash photo.

The way I mistakenly initially parsed this comment gave rise to a potentially-dumb idea/question:

What would happen if you

- begin capturing video (unsure of fps) on a phone-quality sensor in a near-dark environment

- pulse the phone's flash LED(s) like you're taking a photo

- do super-resolution on the resulting video to extract a photo...

- ...while factoring in the decay in brightness/saturation in consecutive video frames produced by the flash pulse?

I vaguely recall reading somewhere that oversaturated photos have more signal in them and are easier to fix than undersaturated. Hmm.

IIRC super-resolution worked with 30fps source video for better quality; I wonder if 60fps or 120fps source video would produce better brightness decay data, or whether super-resolution could actually help extract more signal out of the decay sequence too.

On the other hand, I'm not sure if super-resolution fundamentally requires largely consistent brightness in order to work as well as it does. :/

Perhaps individual networks could be trained/tuned to specific slices/windows of the brightness gradient. I also wonder if it would be useful to factor the superresolution process into each of the brightness-specific stages or just to do it at the end.

Re: Learning to See in the Dark (2018)

#168

Post by exikyut »

y7r4m wrote: post #161
randyrand wrote: post #160

Earlier quoted context omitted.

It would be really cool if you could feed the network a photo with flash that it could use for gathering more information, but then recreated a photo without flash from the non-flash raw. Often flash is not the look people are going for, but would be okay with the flash firing in order to improve the non-flash photo.

Absolutely! We recently rebranded our AI solutions from ALLIS (Advanced Low Light Imagine Solution) to ALIIS (All Light Intelligent Imaging Solution) specifically because we are beginning to branch out to handle use cases such as this! As a proof of concept that this task can be tackled directly, a quick search brought up "DeepFlash: Turning a Flash Selfie into a Studio Portrait"[0] Beyond denoising, we are already r…

Before reading your reply to OP's comment I got to thinking about how the super-resolution process and flash photography might interact (https://news.ycombinator.com/item?id=22905317). I get the impression you left the point I got to a long time ago :)

Re: Learning to See in the Dark (2018)

#169

Post by edjrage »

jdmichal wrote: post #69
edjrage wrote: post #65

Earlier quoted context omitted.

> the model actually had [...] information Wait, did it? Isn't the middle photo being shown for comparison only, rather than as an input?

I guess I might have misinterpreted the goal. If the goal was to make the image look like it was daylight, then maybe whitewashing that light reflection was the correct choice. If the goal was to "see in the dark", then it seems like a very bad choice. EDIT: Finally got the paper to load via the helpful wayback machine link provided in another thread. It looks like the goal was to simulate a long exposure with a shor…

Regardless of the goal, there's no way to get a more readable result if the data just isn't there. Whitewashing might simply be a result of that absence.

Re: Learning to See in the Dark (2018)

#170

Post by y7r4m »

exikyut wrote: post #167
randyrand wrote: post #160

Earlier quoted context omitted.

It would be really cool if you could feed the network a photo with flash that it could use for gathering more information, but then recreated a photo without flash from the non-flash raw. Often flash is not the look people are going for, but would be okay with the flash firing in order to improve the non-flash photo.

The way I mistakenly initially parsed this comment gave rise to a potentially-dumb idea/question: What would happen if you - begin capturing video (unsure of fps) on a phone-quality sensor in a near-dark environment - pulse the phone's flash LED(s) like you're taking a photo - do super-resolution on the resulting video to extract a photo... - ...while factoring in the decay in brightness/saturation in consecutive vid…

For the most part, our effort has been focused on single exposure image enhancement, however we are beginning to use recurrent models to improve quality when video information is available.

Nonetheless, it's kinda a neat idea, so I tried testing the feasibility of it. I set up a recent flagship phone that claims to have 960fps super-slow-motion video capture next to another phone with a strobe app at 12Hz with a short delay in between pulses.

https://www.dropbox.com/s/ha51ntucl3klkcb/cell_flash_960fps....

There are definitely a few frames where the LED is at an intermediate brightness, however teasing out the exact timings between the flash and the camera may prove to be difficult to correctly synchronize.

As for over-saturated images having more signal... although the PSNR calculation may give you a better number, in practice, a region that is over-saturated is just a blob of 1s on the image (assuming float64 pixel values of 0-1) and there is no information there to extract. With a black level near but not at 0, we've found there is often more information hidden in the 'dark noise' than can be discerned by the human eye alone.

Post reply on HN