Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rappers, Sorted by Size of Vocabulary

rappers.mdaniels.com.s3-website-us-east-1.amazonaws.com

Post reply on HN
Print view View original

231–240 of 276 posts

Re: Rappers, Sorted by Size of Vocabulary

#231

Post by danielsf »

plorg wrote: post #101
sfrank2147 wrote: post #8

Earlier quoted context omitted.

The problem with average per song is that you "use up" words in every new song, so all things being equal each marginal song has progressively fewer new words.

I bet you could get something insightful from plotting "unique words" versus "total words" - That might give a good idea of the amount of repetition over time, the length or quantity of output, and the total vocabulary.

here's what this looks like. ugly as sin as useless for comparing rappers.

http://www.mdaniels.com/vocab/scatter.png

love your other ideas – hopefully can do them later.

Re: Rappers, Sorted by Size of Vocabulary

#232

Post by danielsf »

iLoch wrote: post #143
orblivion wrote: post #4

This looks at the first so many lyrics in each rapper's career. Aesop Rock came out with some weird stuff right off the bat. I wonder if some of these other rappers became more sophisticated over time. Maybe an average per song would be better, or average uniques per word, would be better.

I agree, perhaps the 35,000 most recent words would be better.

OP here: the challenge is that most artists' best work is in their earlier years. I'd rather have Jay-z's first album than last, ya know?

Re: Rappers, Sorted by Size of Vocabulary

#233

Post by danielsf »

coherentpony wrote: post #73

Maybe this is just me, but it's a little unfair to compare to literary texts . Humour me for a moment. When an artist writes a song, he (or she) has constraints. Most rappers would like to rhyme the ends of their sentences. I know sometimes they don't (like poetry), but it's certainly pleasing to the ear to have that constraint. Artists endeavour to make their songs catchy, that's highly correlated with the gross sal…

OP here: the shakespeare thing is really just a hook, food for thought rather than an academic/cultural judgement.

I also had several suggestions to use shakespeare's sonnets rather than plays, which I should have done.

and yes, this is all just pinch of salt barbership discussion :)

Re: Rappers, Sorted by Size of Vocabulary

#234

Post by danielsf »

nmac wrote: post #16

Its a nice touch including portmanteaus and 'incorrect' ebonics on the list (like "ery'day"), since authors like shakespeare, joyce and others took the same liberties with language. Arguably, that's how language develops and makes it interesting to study and think about. The OP could have easily stuck to words in the OED, kudos.

op here: thanks yo!

Re: Rappers, Sorted by Size of Vocabulary

#235

Post by danielsf »

dmourati wrote: post #17

I'm a big fan of the project and the way it is presented. Not sure why Wu-Tang features so prominently but I guess I'm okay with that. Kool Keith should be broken down further into his constituent parts. I also would have thought the Beastie Boys would have run higher.

OP here: many thanks!

Re: Rappers, Sorted by Size of Vocabulary

#236

Post by danielsf »

kenshiro_o wrote: post #133
danielsf wrote: post #39

author here: hit me up with questions you've got.

Before making this study, what were your predictions? Would have have expected Wu-Tang and GZA to be near the top? What did you expect the average to be? It would be very interesting to do something similar for rockers too.

I only have hip hop data, sadly.

I didn't expect wu-tang to be on the top since it would go across 11 different vocabs. I felt like the law of averages would basically play out.

Re: Rappers, Sorted by Size of Vocabulary

#237

Post by danielsf »

pandler wrote: post #168
danielsf wrote: post #39

author here: hit me up with questions you've got.

Any chance you would release your code for this? I'd love to run an analysis on some lesser known rappers and play around with some of the filters. Awesome project btw. EDIT: The reason I ask is that I assume you don't have the time or desire to add every rapper every person asks you for.

the code is pretty straightforward, just chapter one of the NLTK book and a txt file.

nltk.org/book

Post reply on HN