Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Colorizing and restoring old images with deep learning

github.com

Post reply on HN
Print view View original

61–70 of 107 posts

Re: Colorizing and restoring old images with deep learning

#61

Post by anigbrowl »

citnaj wrote: post #37
nine_k wrote: post #2

The most interesting exhibit for me is "People watching a television set for the first time", where everything is colorized except the TV image , which correctly remains B&W. I wonder what kind of a training set provided the neural network with this notion.

Author here- So I'll just be brutally honest on that one- not all renders are doing that. I cherry picked the one that did that because yeah, it's amazing. There's a simple explanation for why it sometimes doesn't pick up on the guy on tv to color it- The source material is fuzzy and small. I wish I could claim it was something more awesome than that but that's the truth! I'm treating these outputs as an art of selec…

Shooting from the hip here, but I'd much rather you optimized for speed and allowed selection from a rotating palette of local maxima. I noticed your sadness about the limitations on the picture of the indian woman leaning against a tree whose hand came out red, presumably because of vignetting or some chemical inconsistency in the film substrate. But that superposition of possible interpretations on noisy data is something that shouldn't be thrown away - it's the same 'error' that makes optical illusions interesting when they seem to flip back and forth between being a vase and 2 faces or a duck vs. a rabbit. The model is doing such a great job that trying to push it too far in any one direction risks overfitting.

What I'd love to see in the future are compound networks where a few nodes like this can be mixed with a few nodes that extract vector data, a few others that infer depth maps from images, modulated by similarity detectors that match objects and individuals.

I'm very impressed by the work you've already done - I have a huge library of images I'd like to run it against for both forensic and aesthetic purposes.

Re: Colorizing and restoring old images with deep learning

#62

Post by nkoren »

neuromantik8086 wrote: post #58

> And yes, I'm definitely interested in doing video As someone familiar with the libraries space, I'd actually be very interested in seeing a machine learning model that could deal with "cleaning up" old film (I've actually brought this up w/ several of my ML friends occasionally). One of the biggest challenges in the world of media preservation is migrating analogue content to digital media before physical deteriora…

I believe that Peter Jackson's recent endeavour in cleaning up WW1 footage employs significant ML for de-noising, frame interpolation, and colorising. I haven't seen the final film, but some of the clips are staggeringly good: https://www.bbc.com/news/av/entertainment-arts-45884501/pete...

Edit: Here's maybe a better link -- https://www.bbc.com/news/av/entertainment-arts-45803977/pete...

Re: Colorizing and restoring old images with deep learning

#64

Post by joshvm »

How stable is the result with respect to augmentation?

If you get an image with a funny artifact, like a super-red hand, can you fix it by running the network on a slightly augmented image? For this kind of work, it seems reasonable that you could keep re-colorising an image until you got one that was acceptable (as in the case with the B+W TV).

Re: Colorizing and restoring old images with deep learning

#65

Post by WalterBright »

The colorized photos on https://www.reddit.com/r/Colorization/ are just marvelous. If that could be combined with the AI colorization to colorize old BW movies, that would make them so much more watchable. Other attempts at colorizing them, like what Turner did in the 80's, were a commendable attempt but didn't turn out well.

Re: Colorizing and restoring old images with deep learning

#67

Post by draugadrotten »

onemoresoop wrote: post #31
dsfyu404ed wrote: post #8

Earlier quoted context omitted.

The Seneca Native in 1908 example seems the most absurd to me. I know the software has no notion of a "fabric" or "clothing" but it's very rare for brown or beige things to fade to blue (or vise versa). In real life things when transition from brown/beige to another color that other color tends to be a red orange or yellow. I know from the known issue that it likes blue but it still seems very odd that it chose to fa…

To me the Seneca native's skin on the hand seems a bit to reddish. I find these photos to have very high saturation. I think this could be adjusted and get subtler effects. It's still amazing that this is possible with no human intervention but at the same time, from a different perspective, I find that the originals have their own charm that I would leave it like that.

Is it unthinkable that the seneca girl actually had her hand painted red, for decoration or as a symbol of something? Perhaps her father/brother etc was a fighter and this was a way to keep spirits up while he was in the war?

Re: Colorizing and restoring old images with deep learning

#68

Post by slg »

eagsalazar2 wrote: post #24

When I saw "restoring" in the title I was expecting higher resolution. For example seeing in modern photo level detail eyelashes, wrinkles, etc. I get that, like the colors, this would require the adding lots of made up information about scene and feature details but IMO it would blur the lines between restoration and reconstruction/storytelling in a really awesome way. Old photos are cool in their own way but their…

There reminds me of a recent 99% Invisible episode [1] in which they discuss the same topic in the world of dinosaurs. It details how dinosaurs used to be depicted with the goal of only showing the things that we are confident in being true (although what we are confident in obviously changes over time). This results in mostly just greenish-brown skin draped over a muscle structure attached to the fossilized skeletons.

In recent decades there has been a push to show the animals more realistically. The fossilized evidence is studied and compared to the skeletal structure of animals that exist today. Inferences and educated guesses are made from there to project a more realistic but more subjective image of the dinosaurs. We now get much more varied and interesting depictions with feathers, bright coloring, fat deposits, and other features that can neither be completely confirmed or ruled out based on the evidence.

[1] - https://99percentinvisible.org/episode/welcome-to-jurassic-a...

Re: Colorizing and restoring old images with deep learning

#69

Post by blackflame7000 »

draugadrotten wrote: post #67
onemoresoop wrote: post #31

Earlier quoted context omitted.

To me the Seneca native's skin on the hand seems a bit to reddish. I find these photos to have very high saturation. I think this could be adjusted and get subtler effects. It's still amazing that this is possible with no human intervention but at the same time, from a different perspective, I find that the originals have their own charm that I would leave it like that.

Is it unthinkable that the seneca girl actually had her hand painted red, for decoration or as a symbol of something? Perhaps her father/brother etc was a fighter and this was a way to keep spirits up while he was in the war?

If you compare the tone of her hand to her face in the black and white image you can clearly see that the colors are different.

Re: Colorizing and restoring old images with deep learning

#70

Post by theoh »

hk__2 wrote: post #54
iforgotpassword wrote: post #3

I've to admit I have no clue about machine learning, but what I notice is that this seems to have preferred colors for things that can actually have many different colors, most notably clothes. They're almost always this blueish slightly purple color here, even the samurai. Don't get me wrong, this is still awesome and I might try this on some old photos from my grandparents. I'm just wondering if and how one can pre…

> since obviously you can't really know what color some jacket really was. That’s why colorizing companies employ historians and researchers. You can have a pretty accurate idea of this color with enough research, but it takes time (and thus money).

That might work when the job is colorizing Hollywood productions, but for documentary photos, it's not going to be possible in most cases. You just won't have any leads at all about the origin of the garment, apart from whatever you can observe of its style. An expert can certainly suggest a few colours that don't look anachronistic, but that doesn't make the end result historically "accurate", just plausible/convincing.
Post reply on HN