Very nice, but it looks like the blues are kind of off in several pictures.
Colorizing and restoring old images with deep learning
81–90 of 107 posts
Re: Colorizing and restoring old images with deep learning
#82Earlier quoted context omitted.
Author here- So I'll just be brutally honest on that one- not all renders are doing that. I cherry picked the one that did that because yeah, it's amazing. There's a simple explanation for why it sometimes doesn't pick up on the guy on tv to color it- The source material is fuzzy and small. I wish I could claim it was something more awesome than that but that's the truth! I'm treating these outputs as an art of selec…
Shooting from the hip here, but I'd much rather you optimized for speed and allowed selection from a rotating palette of local maxima. I noticed your sadness about the limitations on the picture of the indian woman leaning against a tree whose hand came out red, presumably because of vignetting or some chemical inconsistency in the film substrate. But that superposition of possible interpretations on noisy data is so…
> I noticed your sadness about the limitations on the picture of the indian woman leaning against a tree whose hand came out red,
I think the biggest problem is that picture is not the hand (its very visible and it could be easily fixed in post processing), it's the blue shade in the clothes that just should not be there. Otherwise, the colors are great (skin and all look very real).
Re: Colorizing and restoring old images with deep learning
#8315 years ago I was in the first cohort of a brand new college program in Digital Imaging Technology. I spent the cost of a college diploma and over 10,000 hours learning to do this by hand. Now it's AI on Github for all. The Times They Are A Changin'
Re: Colorizing and restoring old images with deep learning
#84There is something wrong with that woman’s hand. It is either extremely swollen, a glove, or not a real hand (wooden?). Perhaps your model didn’t make a mistake after all
Re: Colorizing and restoring old images with deep learning
#85This is quite interesting! The pictures were basically perfect to myeyes, until I scrolled down to the "gotchas" section, at which point I started to notice a lot of details that are wrong, mostly fading colors, on clothes or otherwise. Now, there seems to be a distinct loss of details in the restored images. The network being resolution-limited, is the black-and-white image displayed at full resolution besides the r…
Author here. That's actually what I find quite fascinating myself about the results- that they look almost perfect at first glance, yet you drill down a bit closer and you see another "zombie hand". The resolution issue you mention is definitely something I'm painfully aware of- it just comes down to lack of memory resources to support bigger renderings. That's going to be something I'm going to try to attack next.
Thanks for being so present in the comments :)
I don't know much about ML, but would it be possible to use some kind of attention model to iteratively construct the final colouring? The memory limit of the GPU would then limit the attention region size, but not the maximum image size. Talkin' outta my rear here, though.
Re: Colorizing and restoring old images with deep learning
#86Earlier quoted context omitted.
I believe that Peter Jackson's recent endeavour in cleaning up WW1 footage employs significant ML for de-noising, frame interpolation, and colorising. I haven't seen the final film, but some of the clips are staggeringly good: https://www.bbc.com/news/av/entertainment-arts-45884501/pete... Edit: Here's maybe a better link -- https://www.bbc.com/news/av/entertainment-arts-45803977/pete...
I'm actually not sure much ML was involved here - depends where you draw the line I guess, but denoising and interpolation for restoration typically use more traditional wavelet and optical flow algorithms. The work for this was done by Park Road Post and StereoD, which are established post-production facilities using fairly off-the-shelf image processing software. The colorisation likely leant heavily on manual roto…
Huh, I recall seeing an article at one point (can't find the link) where it said or suggested that ML was involved. Of course this could have just been a journalist failing to make the distinction; I've seen everything from linear regression on up naively lumped into the ML bucket.
In any case the results are damned impressive -- can't say I've seen anything like it before.
Re: Colorizing and restoring old images with deep learning
#87Earlier quoted context omitted.
> since obviously you can't really know what color some jacket really was. That’s why colorizing companies employ historians and researchers. You can have a pretty accurate idea of this color with enough research, but it takes time (and thus money).
That might work when the job is colorizing Hollywood productions, but for documentary photos, it's not going to be possible in most cases. You just won't have any leads at all about the origin of the garment, apart from whatever you can observe of its style. An expert can certainly suggest a few colours that don't look anachronistic, but that doesn't make the end result historically "accurate", just plausible/convinc…
Is there enough money in colorizing old movies to bother?
Re: Colorizing and restoring old images with deep learning
#88The most interesting exhibit for me is "People watching a television set for the first time", where everything is colorized except the TV image , which correctly remains B&W. I wonder what kind of a training set provided the neural network with this notion.
Author here- So I'll just be brutally honest on that one- not all renders are doing that. I cherry picked the one that did that because yeah, it's amazing. There's a simple explanation for why it sometimes doesn't pick up on the guy on tv to color it- The source material is fuzzy and small. I wish I could claim it was something more awesome than that but that's the truth! I'm treating these outputs as an art of selec…
If you had like to brutely honest, you should put randomly selected set, along with the hand-picked set - labeling each set how it was selected. This is a cancer in current deep learning research. You see paper with such a glowing cool examples but in reality they are just hiding all problematic cases while being fully aware of it. If this happened anywhere else in any other domain people would say they got ripped off and outright lied to.
Re: Colorizing and restoring old images with deep learning
#89This seems almost too good to be true. One thing I find very striking is how it gets skin tones very plausible across people of different ethnicities (though the majority of subjects in the picture appear of european descent). Unless a) my brain is applying more interpretation to these pictures than I realize or b) the author (intentionally or not) picked out pictures that show the best results
One thing I find very striking is how it gets skin tones very plausible across people of different ethnicities (though the majority of subjects in the picture appear of european descent).
Look at the Chinese Opium Smokers in 1880. They appear slightly too caucasian-coloured to me.
At the extreme, it'd be interesting to see what it would do to these, for example: https://mashable.com/2015/01/31/former-slaves-photos-united-...
Re: Colorizing and restoring old images with deep learning
#90One of the reasons why these photos look so convincingly realistic is the same reason https://en.wikipedia.org/wiki/Chroma_subsampling is done --- the human eye has less sensitivity to colour resolution, and so even relatively vague blobs of colour can evoke the right perception as long as there is sufficient luma detail (provided by the original monochrome image); but if you inspect the photos closely, you'll see there are plenty of unnatural gradients in clothes and such, and the colours of objects blend into each other.