Colorizing and restoring old images with deep learning • Hacker News Board

Re: Colorizing and restoring old images with deep learning

#31

Post by onemoresoop » Fri, Nov 02, 2018, 6:18 PM UTC

dsfyu404ed wrote: post #8

iforgotpassword wrote: post #3

I've to admit I have no clue about machine learning, but what I notice is that this seems to have preferred colors for things that can actually have many different colors, most notably clothes. They're almost always this blueish slightly purple color here, even the samurai. Don't get me wrong, this is still awesome and I might try this on some old photos from my grandparents. I'm just wondering if and how one can pre…

The Seneca Native in 1908 example seems the most absurd to me. I know the software has no notion of a "fabric" or "clothing" but it's very rare for brown or beige things to fade to blue (or vise versa). In real life things when transition from brown/beige to another color that other color tends to be a red orange or yellow. I know from the known issue that it likes blue but it still seems very odd that it chose to fa…

To me the Seneca native's skin on the hand seems a bit to reddish. I find these photos to have very high saturation. I think this could be adjusted and get subtler effects. It's still amazing that this is possible with no human intervention but at the same time, from a different perspective, I find that the originals have their own charm that I would leave it like that.

Re: Colorizing and restoring old images with deep learning

#32

Post by bpye » Fri, Nov 02, 2018, 6:34 PM UTC

searine wrote: post #27

Colorizing single images will always been a bespoke task. There are just too much missing data in the image to be able to create high quality colorizations from the photo alone. However, I think the real application here is colorizing frames of movies. Imagine being able to turn black and white historical footage into color. It won't be as good looking as a single image, but it would be good enough i bet.

I imagine applying this to a video will perhaps be better than the stills, motion helps to hide errors.

Re: Colorizing and restoring old images with deep learning

#33

Post by kinos » Fri, Nov 02, 2018, 7:05 PM UTC

Can someone run this against some manga?

Re: Colorizing and restoring old images with deep learning

#34

Post by gowld » Fri, Nov 02, 2018, 7:12 PM UTC

eagsalazar2 wrote: post #24

When I saw "restoring" in the title I was expecting higher resolution. For example seeing in modern photo level detail eyelashes, wrinkles, etc. I get that, like the colors, this would require the adding lots of made up information about scene and feature details but IMO it would blur the lines between restoration and reconstruction/storytelling in a really awesome way. Old photos are cool in their own way but their…

https://www.youtube.com/watch?v=gg0F5JjKmhA

Re: Colorizing and restoring old images with deep learning

#35

Post by amelius » Fri, Nov 02, 2018, 7:18 PM UTC

compsciphd wrote: post #16

nine_k wrote: post #2

The most interesting exhibit for me is "People watching a television set for the first time", where everything is colorized except the TV image , which correctly remains B&W. I wonder what kind of a training set provided the neural network with this notion.

it didn't, the tv picture is much more blue (at least to my eyes) than the original.

Yes.

Re: Colorizing and restoring old images with deep learning

#36

Post by amelius » Fri, Nov 02, 2018, 7:19 PM UTC

whoisjuan wrote: post #28

nine_k wrote: post #2

The most interesting exhibit for me is "People watching a television set for the first time", where everything is colorized except the TV image , which correctly remains B&W. I wonder what kind of a training set provided the neural network with this notion.

There's some haze on the original image. I think it simples ignores the hazy portion of an image or attempts a very light colorization which seems to be the case here.

I'd suggest to run the code on a modified image, where the TV screen image is more clear.

Re: Colorizing and restoring old images with deep learning

#37

Post by citnaj » Fri, Nov 02, 2018, 7:22 PM UTC

nine_k wrote: post #2

The most interesting exhibit for me is "People watching a television set for the first time", where everything is colorized except the TV image , which correctly remains B&W. I wonder what kind of a training set provided the neural network with this notion.

Author here- So I'll just be brutally honest on that one- not all renders are doing that. I cherry picked the one that did that because yeah, it's amazing. There's a simple explanation for why it sometimes doesn't pick up on the guy on tv to color it- The source material is fuzzy and small.

I wish I could claim it was something more awesome than that but that's the truth! I'm treating these outputs as an art of selection to a certain extent because it's simply not 100% consistent yet. That's one of the things I'm going to continue to try to improve upon.

Re: Colorizing and restoring old images with deep learning

#38

Post by amelius » Fri, Nov 02, 2018, 7:22 PM UTC

Seems like an easy problem for DL, as you have an enormous amount of data available (just take any color image, convert it to grayscale and you have a pair of training images).

(This is also the case for e.g. the superresolution problem.)

Re: Colorizing and restoring old images with deep learning

#39

Post by citnaj » Fri, Nov 02, 2018, 7:23 PM UTC

david_draco wrote: post #9

iforgotpassword wrote: post #3

I've to admit I have no clue about machine learning, but what I notice is that this seems to have preferred colors for things that can actually have many different colors, most notably clothes. They're almost always this blueish slightly purple color here, even the samurai. Don't get me wrong, this is still awesome and I might try this on some old photos from my grandparents. I'm just wondering if and how one can pre…

This is mentioned in "known issues" and near "Generator Loss"

Author here. Yes, it drives me nuts :)

Re: Colorizing and restoring old images with deep learning

#40

Post by citnaj » Fri, Nov 02, 2018, 7:26 PM UTC

crazygringo wrote: post #15

Very interesting... seems to basically learn that: Faces -> some variety of flesh-colored from light to dark Fabric/clothing -> blue Sky -> blue Vegetation -> green Wood -> brown Blank -> turquoise or tan Small details -> fascinating variety of colors, but often a brilliant red Which all seems fairly reasonable. For many things (like wood or skin) it seems accurate. Obviously things like clothes come in such a variet…

Author here. Yeah you're basically right. GANs vastly improve the situation though because being safe with "green for grass, blue for sky, brown as default" doesn't work in the Generative-Adversarial setting. The critic will assign lower scores if the generator keeps outputting brown. Now I'd think the generator would get more creative than going for blue constantly but that might just be a matter or more/better training (...?)