Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Colorizing and restoring old images with deep learning

github.com

Post reply on HN
Print view View original

51–60 of 107 posts

Re: Colorizing and restoring old images with deep learning

#51

Post by sova »

cmpb wrote: post #48
citnaj wrote: post #37

Earlier quoted context omitted.

Author here- So I'll just be brutally honest on that one- not all renders are doing that. I cherry picked the one that did that because yeah, it's amazing. There's a simple explanation for why it sometimes doesn't pick up on the guy on tv to color it- The source material is fuzzy and small. I wish I could claim it was something more awesome than that but that's the truth! I'm treating these outputs as an art of selec…

Might make a good addition to the "Known Issues" section or somewhere else. Just somewhere to indicate all the little points of human intervention.

to demand documentation of art is to impede the process

Re: Colorizing and restoring old images with deep learning

#52

Post by mrharrison »

citnaj wrote: post #37
nine_k wrote: post #2

The most interesting exhibit for me is "People watching a television set for the first time", where everything is colorized except the TV image , which correctly remains B&W. I wonder what kind of a training set provided the neural network with this notion.

Author here- So I'll just be brutally honest on that one- not all renders are doing that. I cherry picked the one that did that because yeah, it's amazing. There's a simple explanation for why it sometimes doesn't pick up on the guy on tv to color it- The source material is fuzzy and small. I wish I could claim it was something more awesome than that but that's the truth! I'm treating these outputs as an art of selec…

That seems like it would be quite difficult. If it is colorizing a black and white image wouldn't it colorize the black and white image on the TV screen. You would almost have to train it to recognize old TV's that produce black and white images, so that it wouldn't colorize the TV screen. Unless you can get a unique signature from a black and white photo of a black and white screen. Fun stuff.

Re: Colorizing and restoring old images with deep learning

#54

Post by hk__2 »

iforgotpassword wrote: post #3

I've to admit I have no clue about machine learning, but what I notice is that this seems to have preferred colors for things that can actually have many different colors, most notably clothes. They're almost always this blueish slightly purple color here, even the samurai. Don't get me wrong, this is still awesome and I might try this on some old photos from my grandparents. I'm just wondering if and how one can pre…

> since obviously you can't really know what color some jacket really was.

That’s why colorizing companies employ historians and researchers. You can have a pretty accurate idea of this color with enough research, but it takes time (and thus money).

Re: Colorizing and restoring old images with deep learning

#55

Post by MayeulC »

This is quite interesting!

The pictures were basically perfect to myeyes, until I scrolled down to the "gotchas" section, at which point I started to notice a lot of details that are wrong, mostly fading colors, on clothes or otherwise.

Now, there seems to be a distinct loss of details in the restored images. The network being resolution-limited, is the black-and-white image displayed at full resolution besides the restored one?

What I would like to see is the output of the network to be treated as chrominance only.

Take the YUV transform of both the input and output images, scale back the UV matrix of the restored one to match the input, and replace the original channels. I'd be really curious to look at the output (and would do it myself if I was not on asmartphone)!

Nevertheless, that's some awesome work, and I can't wait to see where it goes!

Re: Colorizing and restoring old images with deep learning

#56

Post by citnaj »

cmpb wrote: post #48
citnaj wrote: post #37

Earlier quoted context omitted.

Author here- So I'll just be brutally honest on that one- not all renders are doing that. I cherry picked the one that did that because yeah, it's amazing. There's a simple explanation for why it sometimes doesn't pick up on the guy on tv to color it- The source material is fuzzy and small. I wish I could claim it was something more awesome than that but that's the truth! I'm treating these outputs as an art of selec…

Might make a good addition to the "Known Issues" section or somewhere else. Just somewhere to indicate all the little points of human intervention.

Yeah I tried pointing that out in the Known Issues section by alluding to adjusting the size parameter as a means to get the best images. But I think I'll just go ahead and be crystal clear on the "art of selection" part so that this doesn't come across as snake oil.

Re: Colorizing and restoring old images with deep learning

#57

Post by citnaj »

citnaj wrote: post #56
cmpb wrote: post #48

Earlier quoted context omitted.

Might make a good addition to the "Known Issues" section or somewhere else. Just somewhere to indicate all the little points of human intervention.

Yeah I tried pointing that out in the Known Issues section by alluding to adjusting the size parameter as a means to get the best images. But I think I'll just go ahead and be crystal clear on the "art of selection" part so that this doesn't come across as snake oil.

Added this to the Known Issues section.

Re: Colorizing and restoring old images with deep learning

#58

Post by neuromantik8086 »

> And yes, I'm definitely interested in doing video

As someone familiar with the libraries space, I'd actually be very interested in seeing a machine learning model that could deal with "cleaning up" old film (I've actually brought this up w/ several of my ML friends occasionally). One of the biggest challenges in the world of media preservation is migrating analogue content to digital media before physical deterioration kicks in. Oftentimes, libraries aren't able to migrate content quickly enough, and you end up with frames that have been partially eaten away by mold.

As a heads-up, these are some of the problems you might encounter on the film front (which you might not otherwise find with photos due to differences in materials used, etc):

https://www.nyu.edu/tisch/preservation/program/05fall/physic...

https://www.filmpreservation.org/preservation-basics/vinegar...

Re: Colorizing and restoring old images with deep learning

#59

Post by citnaj »

sova wrote: post #51
cmpb wrote: post #48

Earlier quoted context omitted.

Might make a good addition to the "Known Issues" section or somewhere else. Just somewhere to indicate all the little points of human intervention.

to demand documentation of art is to impede the process

LOL I think that'll be my go to excuse for lack of documentation for now on....

Re: Colorizing and restoring old images with deep learning

#60

Post by citnaj »

MayeulC wrote: post #55

This is quite interesting! The pictures were basically perfect to myeyes, until I scrolled down to the "gotchas" section, at which point I started to notice a lot of details that are wrong, mostly fading colors, on clothes or otherwise. Now, there seems to be a distinct loss of details in the restored images. The network being resolution-limited, is the black-and-white image displayed at full resolution besides the r…

Author here. That's actually what I find quite fascinating myself about the results- that they look almost perfect at first glance, yet you drill down a bit closer and you see another "zombie hand". The resolution issue you mention is definitely something I'm painfully aware of- it just comes down to lack of memory resources to support bigger renderings. That's going to be something I'm going to try to attack next.
Post reply on HN