Live data from Hacker News

Understanding Aesthetics with Deep Learning

devblogs.nvidia.com

11–20 of 31 posts

Re: Understanding Aesthetics with Deep Learning

#11
post #2

I wonder how much our sense of aesthetics has to do with the perceived scarcity or effort of creation needed. I remember the first HDR photos looked absolutely mind-blowing to me, but now, as the process has been automated and is ubiquitous, it just looks tacky.

read adorno

Re: Understanding Aesthetics with Deep Learning

#12
post #9

I don't think computers would be able to understand aesthetics. It is a really high-level concept. Plus, I think deep-learning is a marketing mambo-jambo and does not perform much better than a linear SVM.

Then why are we using deep convolutional networks for state of the art vision and speech when we could just plug an SVM with handcrafted features? From what I know, error rates in vision dropped from 25% to less than 5% since deep learning. That's no trifle, especially at the higher end of the accuracy scale. It's very hard to conquer those last few percents.

Re: Understanding Aesthetics with Deep Learning

#13
post #6

Earlier quoted context omitted.

Yeah, exactly. No good critic of photography thinks that subject matter is irrelevant, that you can understand pictures as if they were abstract compositions of light and color. You'd might as well try to read a poem in an unknown language. This algorithm might learn to identify certain cliches, but it'll never learn what makes a picture powerful.

You have to wonder, though. Is it impossible that there's a "music theory" for images/paintings/art that explains the mechanics of what makes them more compelling vs less compelling? I suspect there is, at least to some degree. Obviously images convey much more information than music, so any theory that doesn't encompass the semantics of the subject will miss most of the signal. But is there a theory for the presenta…

Perhaps it's not quite analogous to music theory, but what you're describing in the first paragraph would be referred to as the formal elements of art or simply the elements of art.

Analysis of these elements (form, line, space, color, and texture) is usually a part of the sort of art criticism you'd find in academic studio art, art history, or even just the New York Times art section.

The visual design field has a similar, extended set of elements for describing the formal elements of a design piece.

In both art and design, works are usually considered effective if they use the formal elements of art/design to support what you refer to as the semantics of the subject. That's a broad generalization, but you see it in practice a lot, so it seems like a fair thing to say.

Academic art history is starting to feel the influence of machine learning and computer vision precisely because computers can be trained to recognize the formal elements of art and associate their use with movements and historical periods. There are way more detailed articles than this one, but this will get you started if you're interested in this sort of thing:

https://www.technologyreview.com/s/537366/the-machine-vision...

Re: Understanding Aesthetics with Deep Learning

#14
post #6

Earlier quoted context omitted.

Yeah, exactly. No good critic of photography thinks that subject matter is irrelevant, that you can understand pictures as if they were abstract compositions of light and color. You'd might as well try to read a poem in an unknown language. This algorithm might learn to identify certain cliches, but it'll never learn what makes a picture powerful.

You have to wonder, though. Is it impossible that there's a "music theory" for images/paintings/art that explains the mechanics of what makes them more compelling vs less compelling? I suspect there is, at least to some degree. Obviously images convey much more information than music, so any theory that doesn't encompass the semantics of the subject will miss most of the signal. But is there a theory for the presenta…

The problem is that you'll hit a wall when it comes to understanding "what makes art." You can do all the theory you want, and people do, of course. You can analyze all that has ever been done, and come up with rules for describing and even generating music and art. But there is no guarantee that these will allow you to predict what makes future art. Just like with financial markets, in art, what happened in the past is not a good predictor of the future. That is the mistake that "art theorists" tend to make, have made for decades and decades, and are carrying over rather simplistically to statistical analysis via machine learning.

This is particularly challenging in art (as compared e.g. to financial markets) because much of what defines new art is specifically what makes it different from what has come before it. That is to say, art, by its nature, will always beat any rules you try to design, because that is what it does, indeed, what is must do.

The proof is in the pudding: that machine learning systems can be designed to learn the statistical trends in a body of works and then generate similar art, done since at least the 80s if not earlier, evokes the very definition of the detractive term "cookie cutter art." "Good" art then, by contradiction, is exactly that art that does not fit into such a model -- plus "something".

Surely it is that "something" we'd like to find, but I am afraid that using rule- or statistically-based analysis to help curators sort through art, even with the prescribed notion that this should help them find "diamonds in the rough", it will generate an echo chamber in which the next diamond, which by definition is quite different from diamonds that came before it, to remain undiscovered, buried in a pile of sorted spam.

It is for this reason that I believe that despite the advances in machine learning, nothing will ever replace the past-time of "crate digging" for finding gems. The DJs job will never completely die.

... I will add: That is not to say that tools for automatically understanding and measuring aspects of a photo or piece of music are not useful for artists as a way of judging their own work and making decisions. But it is exactly those artists that will look at the "goodness indicator" drop one notch while they make a change, and say, "I'm fine with that", who will produce the next important work.

Re: Understanding Aesthetics with Deep Learning

#15
post #2

I wonder how much our sense of aesthetics has to do with the perceived scarcity or effort of creation needed. I remember the first HDR photos looked absolutely mind-blowing to me, but now, as the process has been automated and is ubiquitous, it just looks tacky.

"fashion is clothing so terrible we have to change them every six months"

"Fashion is a form of ugliness so intolerable that we have to alter it every six months."

Re: Understanding Aesthetics with Deep Learning

#16
post #11
post #2

I wonder how much our sense of aesthetics has to do with the perceived scarcity or effort of creation needed. I remember the first HDR photos looked absolutely mind-blowing to me, but now, as the process has been automated and is ubiquitous, it just looks tacky.

read adorno

Care to elaborate? Does he go into this?

Re: Understanding Aesthetics with Deep Learning

#17

I believe that there are factors that go beyond visible composition. Just a though experiment, I imagine that the brain would evaluate easthetics of two similar images differently depending on whether it is an image of an object it recognizes or not - when evaluating the image with an object other qualities of the object (that are not necessarily visible in the image) will be taken into account.

Yeah, exactly. No good critic of photography thinks that subject matter is irrelevant, that you can understand pictures as if they were abstract compositions of light and color. You'd might as well try to read a poem in an unknown language. This algorithm might learn to identify certain cliches, but it'll never learn what makes a picture powerful.

We listen to foreign music a lot and still find pleasure in the voices. Although I'm mostly speaking from my experience with English before I had learned it, which is yet rather close to German, so YMMV

>No True Scotsman thinks that subject matter is irrelevant

fixed that for you

Re: Understanding Aesthetics with Deep Learning

#18
Aesthetics is a game of cat and mouse. Artists create some new things. Then critics and theorists observe the patterns of composition, color, proportion, etc., that are popular. These rules are canonized in books. Then artists challenge the rules.

The comparisons to music theory in this thread are apt. Music theory is always behind music production.

How can you understand aesthetics without understanding creativity?

Re: Understanding Aesthetics with Deep Learning

#19
post #11
post #2

I wonder how much our sense of aesthetics has to do with the perceived scarcity or effort of creation needed. I remember the first HDR photos looked absolutely mind-blowing to me, but now, as the process has been automated and is ubiquitous, it just looks tacky.

read adorno

Have you read him? If so, why not synthesize his arguments so we can all understand?

Re: Understanding Aesthetics with Deep Learning

#20

Earlier quoted context omitted.

Yeah, exactly. No good critic of photography thinks that subject matter is irrelevant, that you can understand pictures as if they were abstract compositions of light and color. You'd might as well try to read a poem in an unknown language. This algorithm might learn to identify certain cliches, but it'll never learn what makes a picture powerful.

We listen to foreign music a lot and still find pleasure in the voices. Although I'm mostly speaking from my experience with English before I had learned it, which is yet rather close to German, so YMMV > No True Scotsman thinks that subject matter is irrelevant fixed that for you

No idea what the No True Scotsman fallacy has to do with this; it involves dismissing an example, but no example was offered. You're welcome to offer one, if you'd like.

We listen to foreign music and find pleasure in the voices because a voice itself is expressive, independent of language: a cry, a laugh, an imprecation. The same is true of light, shape, etc., but artists who work with these qualities independent of their reference to objects tend to choose media other than photography, for obvious reasons.

Post reply on HN