Live data from Hacker News

Gender and verbs across 100K stories: a tidy analysis

varianceexplained.org

31–40 of 178 posts

Re: Gender and verbs across 100K stories: a tidy analysis

#32

The reference at the end talking about comparing changes over time reminded me of a problem I've been kicking around. For this particular system that's not too hard, since you're putting each word on a one-dimensional femininity/masculinity scale, so you could plot a word on a line graph or something. But what do you do if you want to evaluate the changes of more complex relationships over time? Not just mapping word…

One way is to learn clusters in the data over the whole time period and then calculate the cluster distribution of the same clusters for each time interval separately. You can then track the proportion of each cluster over time as a time series.

So for example if you have a word cluster that describes computers, you could see it start growing in the seventies, while having near-zero proportion in 19th century etc.

Re: Gender and verbs across 100K stories: a tidy analysis

#33

Earlier quoted context omitted.

As in "She Apache helicoptered him in the face"?

Some people identify as "attack helicopter".

TIL, and it's even a first page search result.

> I’m beautiful. I’m having a plastic surgeon install rotary blades, 30 mm cannons and AMG-114 Hellfire missiles on my body. From now on I want you guys to call me “Apache” and respect my right to kill from above

Re: Gender and verbs across 100K stories: a tidy analysis

#34
post #6

It's interesting how thoroughly ingrained sexist concepts are in the language. Even a verb that's fairly active like 'resist' assumes a power relation in which they are in a worse position. I'd like to see this done by country or year or language or genre.

Is it sexist if it's reflective of reality? Men commit more aggressive crimes than women by far.

It's probably still sexist. I think you would probably agree that literature spoke the same way about black people a couple hundred years ago. Cultures can be sexist or racist even if everyone subscribes to the same discriminatory standards.

However, I think the interesting point is that this is the culmination of 100,000 plot descriptions. At that point, it's not a reflection of bias in an author, it's a culturally accepted position on the roles of genders. (whether right or wrong)

...or I guess it could be a bunch of biased wikipedia editors :D

Re: Gender and verbs across 100K stories: a tidy analysis

#36

The reference at the end talking about comparing changes over time reminded me of a problem I've been kicking around. For this particular system that's not too hard, since you're putting each word on a one-dimensional femininity/masculinity scale, so you could plot a word on a line graph or something. But what do you do if you want to evaluate the changes of more complex relationships over time? Not just mapping word…

The Gapminder charts very nicely illustrates multi dimensional data over time:

http://www.gapminder.org/tools/#_locale_id=en;&chart-type=bu...

I imagine a dataset that charted the word use over time would fit into a chart like this.

Re: Gender and verbs across 100K stories: a tidy analysis

#37

> what verbs are used after “he” and “she”, and therefore what roles male and female characters tend to have within stories. It might've been better had the author (and the Jane Austen article's author) used some NLP processing to see whether the pronoun was actually the subject of those verbs. But I'll grant that it's usually the case. Also interesting: gender and the object of those verbs. EDIT: after some research…

Can you give an example sentence where this isn't the case?

Re: Gender and verbs across 100K stories: a tidy analysis

#38

The reference at the end talking about comparing changes over time reminded me of a problem I've been kicking around. For this particular system that's not too hard, since you're putting each word on a one-dimensional femininity/masculinity scale, so you could plot a word on a line graph or something. But what do you do if you want to evaluate the changes of more complex relationships over time? Not just mapping word…

What about 1d protection and simple line plots over time? You'll need 2 or 3 of them over some interesting projections, but it's better than unfamiliar and complex representation.

Alternatively 1d protection + style change for another axis. Like changing the colour, thickness of the line, etc.

Re: Gender and verbs across 100K stories: a tidy analysis

#39
> I think this paints a somewhat dark picture of gender roles within typical story plots. Women are more likely to be in the role of victims- “she screams”, “she cries”, or “she pleads.” Men tend to be the aggressor: “he kidnaps” or “he beats”. Not all male-oriented terms are negative- many, like “he saves”/”he rescues” are distinctly positive- but almost all are active rather than receptive.

Got mathematical analysis, but terrible semantic interpretation.

There are only 3 really negative verbs for men ("murdered", "kills", "kidnaps"), and 3 distinctly positive ("saves", "defeats", "rescues"). "Beat" is ambiguous, personally I would more likely interpret it as "defeats" rather than "hit"/"punch". This is in no way a "dark" picture, and the only relevant conclusion is the one he briefly mentions in passing: women are described as more passive, men as more active.

Re: Gender and verbs across 100K stories: a tidy analysis

#40

Earlier quoted context omitted.

Is it sexist if it's reflective of reality? Men commit more aggressive crimes than women by far.

It's probably still sexist. I think you would probably agree that literature spoke the same way about black people a couple hundred years ago. Cultures can be sexist or racist even if everyone subscribes to the same discriminatory standards. However, I think the interesting point is that this is the culmination of 100,000 plot descriptions. At that point, it's not a reflection of bias in an author, it's a culturally…

If you took 100,000 historical books describing the periods of US slavery, you'd have a strong correlation between "black" and "slave", but concluding that the books are therefore racist would be extremely illogical.
Post reply on HN