Understanding Aesthetics with Deep Learning
devblogs.nvidia.com
Understanding Aesthetics with Deep Learning
1–10 of 31 posts
Re: Understanding Aesthetics with Deep Learning
#2I wonder how much our sense of aesthetics has to do with the perceived scarcity or effort of creation needed. I remember the first HDR photos looked absolutely mind-blowing to me, but now, as the process has been automated and is ubiquitous, it just looks tacky.
Re: Understanding Aesthetics with Deep Learning
#3I wonder how much our sense of aesthetics has to do with the perceived scarcity or effort of creation needed. I remember the first HDR photos looked absolutely mind-blowing to me, but now, as the process has been automated and is ubiquitous, it just looks tacky.
HDR only looks tacky when it's done tacky, ridiculously amping up the saturation and removing all the shadows
Re: Understanding Aesthetics with Deep Learning
#4I believe that there are factors that go beyond visible composition. Just a though experiment, I imagine that the brain would evaluate easthetics of two similar images differently depending on whether it is an image of an object it recognizes or not - when evaluating the image with an object other qualities of the object (that are not necessarily visible in the image) will be taken into account.
Re: Understanding Aesthetics with Deep Learning
#5I believe that there are factors that go beyond visible composition. Just a though experiment, I imagine that the brain would evaluate easthetics of two similar images differently depending on whether it is an image of an object it recognizes or not - when evaluating the image with an object other qualities of the object (that are not necessarily visible in the image) will be taken into account.
Yeah, exactly. No good critic of photography thinks that subject matter is irrelevant, that you can understand pictures as if they were abstract compositions of light and color. You'd might as well try to read a poem in an unknown language. This algorithm might learn to identify certain cliches, but it'll never learn what makes a picture powerful.
Re: Understanding Aesthetics with Deep Learning
#6I believe that there are factors that go beyond visible composition. Just a though experiment, I imagine that the brain would evaluate easthetics of two similar images differently depending on whether it is an image of an object it recognizes or not - when evaluating the image with an object other qualities of the object (that are not necessarily visible in the image) will be taken into account.
Yeah, exactly. No good critic of photography thinks that subject matter is irrelevant, that you can understand pictures as if they were abstract compositions of light and color. You'd might as well try to read a poem in an unknown language. This algorithm might learn to identify certain cliches, but it'll never learn what makes a picture powerful.
You have to wonder, though. Is it impossible that there's a "music theory" for images/paintings/art that explains the mechanics of what makes them more compelling vs less compelling? I suspect there is, at least to some degree.
Obviously images convey much more information than music, so any theory that doesn't encompass the semantics of the subject will miss most of the signal. But is there a theory for the presentation and composition of the subject? To some degree, I'm confident there is.
Some of the methods used to debug the deep learning of images already do a fair job of showing the locus of focus in the image where the DNN found maximum information. I can see such a technique discovering many of the techniques used by artists and photographers to direct the observer's eye or juxtapose objects that conflict.
Re: Understanding Aesthetics with Deep Learning
#7I wonder how much our sense of aesthetics has to do with the perceived scarcity or effort of creation needed. I remember the first HDR photos looked absolutely mind-blowing to me, but now, as the process has been automated and is ubiquitous, it just looks tacky.
"fashion is clothing so terrible we have to change them every six months"
Re: Understanding Aesthetics with Deep Learning
#8I believe that there are factors that go beyond visible composition. Just a though experiment, I imagine that the brain would evaluate easthetics of two similar images differently depending on whether it is an image of an object it recognizes or not - when evaluating the image with an object other qualities of the object (that are not necessarily visible in the image) will be taken into account.
I agree. Although one could imagine that concepts conveyed in a photograph could be extracted and abstracted as vectors -- just like word2vec and its successors. Of course, there is a long way to go before we hit "human understanding" parity, but I think ideas from [1], [2] and [3] could be extrapolated in doing just that.
[1] Deep Visual-Semantic Alignments for Generating Image Descriptions - cs.stanford.edu/people/karpathy/cvpr2015.pdf
[2] Deep Learning for Content-Based Image Retrieval - www.research.larc.smu.edu.sg/mlg/papers/MM14-fp336-hoi.pdf
[3] Deep Learning for Content-Based Image Retrieval - www.cs.rutgers.edu/~elgammal/pub/MTA_2014_Saleh.pdf
Re: Understanding Aesthetics with Deep Learning
#9I don't think computers would be able to understand aesthetics. It is a really high-level concept. Plus, I think deep-learning is a marketing mambo-jambo and does not perform much better than a linear SVM.
Re: Understanding Aesthetics with Deep Learning
#10[deleted]