Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

‘2001: A Space Odyssey’ rendered in the style of Picasso

bhautikj.tumblr.com

Post reply on HN
Print view View original

51–60 of 97 posts

Re: ‘2001: A Space Odyssey’ rendered in the style of Picasso

#51

Post by brian_cloutier »

vintermann wrote: post #47
Udik wrote: post #38

There is something that escapes me regarding this very cool neural style transfer technique. One would expect it to need at least three starting images: the one to transform, the one used as a source for the style, and a non-styled version of the source. This last one should give the network hints on how to transform the unstyled version in the styled one. For example, what does a straight line end up being in the st…

It's right that it needs additional information to distinguish style from content, but they get that from selected layers from established, pre-trained neural nets for image recognition. I don't entirely understand why it works myself, but it seems to.

From the way you described it, you could consider the pre-trained network to be the "missing image". It already has an idea of what images should look like so when it detects an object the "style" is what makes that object different than the stereotypical one it's already modeled.

Re: ‘2001: A Space Odyssey’ rendered in the style of Picasso

#52

Post by vintermann »

brian_cloutier wrote: post #51
vintermann wrote: post #47

Earlier quoted context omitted.

It's right that it needs additional information to distinguish style from content, but they get that from selected layers from established, pre-trained neural nets for image recognition. I don't entirely understand why it works myself, but it seems to.

From the way you described it, you could consider the pre-trained network to be the "missing image". It already has an idea of what images should look like so when it detects an object the "style" is what makes that object different than the stereotypical one it's already modeled.

Right. But it's more complicated than that, the choice of layer(s) to use matters a lot, and I have no idea why they do as they do. Seems it's a bit of dark magic to get it to work well - takes lot of aesthetic judgements too.

I think Alex J. Champandard's implementation is probably the best one out there right now. It has a ton of knobs to twist and is very fast.

Re: ‘2001: A Space Odyssey’ rendered in the style of Picasso

#53

Post by fractallyte »

Possibly the finest painting software currently available is Synthetik's Studio Artist (http://synthetik.com/). Compared to Adobe's powerhouse software, it's relatively unknown, but that doesn't make it any less innovative.

It uses an algorithmic 'paint synthesizer' to generate brushes (with hundreds of presets) and auto-paint canvases, and is designed for animation (rotoscoping) as well as static artwork. The output can be reminiscent of the style of the movie 'A Scanner Darkly', but the software is hugely flexible. Here are a couple of rather amazing examples: http://studioartist.ning.com/video/auto-rotoscoped-dancers and http://studioartist.ning.com/video/dance-styles-animation-re...

Also, unlike most other 'painterly' software, the graphics are resolution independent - meaning that they can be scaled up to any size without loss of detail.

Re: ‘2001: A Space Odyssey’ rendered in the style of Picasso

#54

Post by i7i »

veli_joza wrote: post #45
jkscm wrote: post #40

Earlier quoted context omitted.

Another interesting problem would be the generation of filler pictures (I don't know the correct term). Normally there is a person who draws keyframes at a much lower framerate. Other animators then fill in the frames between to increase the framerate.

They're called inbetweens or tweens. According to recent HN article, they are outsourced to South Korea. Generating them with algorithm would be interesting, but often incorrect and against artist wishes. For example, objects in motion sometimes needs to be blurred; some characters need to have ghost duplicates; shapes get distorted and exaggerated. I think at this moment it is not possible to instruct algorithm to t…

there's this https://www.youtube.com/watch?v=zICe4hYAJBE

Re: ‘2001: A Space Odyssey’ rendered in the style of Picasso

#57

Post by gh2k »

Udik wrote: post #38

There is something that escapes me regarding this very cool neural style transfer technique. One would expect it to need at least three starting images: the one to transform, the one used as a source for the style, and a non-styled version of the source. This last one should give the network hints on how to transform the unstyled version in the styled one. For example, what does a straight line end up being in the st…

I'm learning about ANNs at the moment, so I'm not really getting this but I'd like to.

Aside from the 'missing image' part, what is the fitness example here for training? How does the training process determine what a good image is given there aren't many (any?) examples of a source image -> picasso mapping?

Re: ‘2001: A Space Odyssey’ rendered in the style of Picasso

#58

Post by neutronicus »

stepvhen wrote: post #7

I have two opinions: 1) I don't think cubism transfers well into a motion picture format, 2) I think these experiments, as they are currently, attempt to merge two styles and end up with neither, and nothing novel in its place; there is little Kubrick or Picasso in the final piece. I think it's superficial and doesn't do either source justice.

Specifically, what the algorithm seems to have done is leave the shapes of objects (furniture, hallways, etc) alone and decorate any blank space with random, Picasso-ish noise. It's chaotic, but isn't really Cubist. To be cubist it'd have to figure out some way to fuck with the rules of perspective much more than it does at the moment.

Re: ‘2001: A Space Odyssey’ rendered in the style of Picasso

#59

Post by JD557 »

Udik wrote: post #38

There is something that escapes me regarding this very cool neural style transfer technique. One would expect it to need at least three starting images: the one to transform, the one used as a source for the style, and a non-styled version of the source. This last one should give the network hints on how to transform the unstyled version in the styled one. For example, what does a straight line end up being in the st…

Disclaimer: I'm probably wrong about this, this is just how I believe "Neural style transfer" works. I never tried this out and there's probably a lot of problems with my explanation.

I believe that this is done using Restricted Boltzman Machines[1] trained with the stylised image.

Think of it as a network that receives an image on the input layer, sends it to one or more hidden layers with less nodes (like an auto-encoder), and then tries to reconstruct the image on the output nodes. This is like a lossy compressor-decompressor overfitted to the stylized image.

Now, just pass the real image as an input to your network and the output should be a stylized version of the input.

[1]http://deeplearning4j.org/restrictedboltzmannmachine

Re: ‘2001: A Space Odyssey’ rendered in the style of Picasso

#60

Post by jamesdwilson »

minimaxir wrote: post #26
jamesdwilson wrote: post #20

Serious question: how is this different than one of the many photoshop filters that could be applied iteratively to each frame?

There likely isn't a Picasso Photoshop filter that can match the fidelity of the style transfer used in the video. (I checked; the best you can get is a macro to automate faux-cubism)

ok, if there was, would there be a difference?
Post reply on HN