Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Estimate your English vocabulary size

testyourvocab.com

Post reply on HN
Print view View original

321–330 of 331 posts

Re: Estimate your English vocabulary size

#321

Post by jtheory »

derrida wrote: post #41

I was honest and was shocked by my result being in the bottom third! I have an IQ > 125! I think I need to read more.

Self-selected survey respondents != actual population.

Though I suppose, also -- IQ doesn't say much at all about what your vocabulary will be. If you don't read much, or mostly read popular literature, it doesn't matter how clever your are; your vocabulary won't be any bigger than the words you've encountered enough to form (or look up) a definition.

Though it's also worth noting that beyond a certain point, building a broader vocabulary isn't very useful; when you communicate, you generally need to limit yourself to vocabulary your audience knows. Likewise, when you're reading, it's pretty common that the authors using particularly obscure vocabulary are using it to muddy their meaning, not clarify it.

Re: Estimate your English vocabulary size

#322

Post by lurker14 »

dsr_ wrote: post #260
_delirium wrote: post #206

Earlier quoted context omitted.

Even writers of high-brow literature use a much smaller vocabulary than the one they can read. Probably more than the newspaper, but not likely 35,000 words. For non-native speakers, I would guess that there are quickly diminishing returns past about 10,000 words. Much more important is how well you're able to use those first 10,000 (or maybe even 5,000). It's one thing to recognize a word when reading it, and anothe…

Googling says that Shakespeare used 31534 words , of which 14376 appear only once, 4343 twice: http://biomet.oxfordjournals.org/content/63/3/435.abstract Not a modern corpus, but reasonable. Strong's Exhaustive Concordance of the Bible (that would be the King James version) lists 8674 Hebrew root words for the pentateuch, and 5624 Greek root words for the New Testament. Wikipedia suggests that full literacy in Chines…

Of those unique words, how many did Shakespear invent?

Making up new words is an easy way to boost vocabulary.

Re: Estimate your English vocabulary size

#323

Post by lurker14 »

Willwhatley wrote: post #93
natural219 wrote: post #61

19,600. I'm willing to accept this, although I'm not going to lie -- I'm very upset at myself. I'm used to scoring 99th percentile in every standardized test; it's kind of a shock to realize that I'm nowhere near the median of even my age group , let alone the general populace (I'm 20). That said, I'm currently reading A Dance with Dragons and there are tons of words in this series (A Song of Ice and Fire) that I'm n…

Howdy, I scored a little over 40k on this, and did well on other verbal tests for the general population when I was still taking tests. I attribute much of my facility to 1)reading fantasy and 2)looking up words I don't know. Since I loathe interrupting the flow of a story, I read with a pencil and make a list of words to batch learning later. Best,

I filled pages of word lists while reading Cryptonomicon. It was amazing.

Re: Estimate your English vocabulary size

#324

Post by Robin_Message »

jtheory wrote: post #319
Robin_Message wrote: post #315

Earlier quoted context omitted.

I just counted, and you've used at least 12685 dictionary words in HP:MOR [1]. I was expecting it to be more to be honest, but it didn't seem right not to post just because it didn't match my expectations. It certainly seems unlikely that someone who can produce an excellent 500,000 word work of fiction (aside: thanks very much by the way,) in addition to reams of technical writing, has a vocabulary not in the 95th p…

There's a huge difference between writing something targeted at a selected audience, in a given time period, limited range of topics the characters will discuss, etc. and listing out all of the vocabulary words you know personally across all domains of knowledge. Even though HP:MoR Harry has a broad vocabulary, for example, and may use some words not all readers will know, there are still tons of topics (with their o…

Thanks to the internet†, I can reveal a surprise: with the same methodology, the dictionary words in the seven real Harry Potter books concatenated is 19,245 and the total unique words is 21,441.

Total word count is 1,122,131 which is longer than HP:MoR by a factor of three. Plotting mean unique word count for the whole, halves and quarters of MoR gives a fit of uniques=168*length^0.3357, which makes sense given Zipf's law. That formula predicts about 18,050 words for a work of the same length as the original HP.

(Edit to add obvious test in the other direction.) The first 386,829 words of the original HP contain 12,255 unique words. The last 386,829 words contain 13,635 uniques. So, its comparable but perhaps slightly more varied (MoR had 12,685).

In light of those figures, is it possible Eliezer's vocabulary is less good than he thinks (Dunning-Kruger)? Especially as the Harry Potter book were written for children and presumably edited as such.

On the other hand, the fact that Eliezer seems to have used fewer words in his writing than you'd expect if his vocab was excellent doesn't mean that his known vocab is poor — he might just not use all the words he knows in writing.

Additionally, given the success of J K Rowling as an author, you might expect her vocabulary to be excellent, so it is conceivable that he's good and she's better.

† I have all the Harry Potter books on a shelf at home. Is torrenting the pdfs at work so I can word count them infringing copyright? I could have done it manually, it just would have taken longer.

Re: Estimate your English vocabulary size

#325

Post by johnsonjohnson »

pbhjpbhj wrote: post #261
johnsonjohnson wrote: post #220

Putting aside all the comparing. Many non-native speakers here say they read and watch many things in english. I do that do and I'm quite positive that 95% of all the reading and listening I do each day is in english. Now with a low score of 17500 I wonder, if it isn't enough to completely endulge oneself in the language, what is? Of course, watching the Simpsons all day won't teach me some of the rarely used words.…

http://www.urbandictionary.com/define.php?term=endulge Sorry, couldn't resist. For a minute I thought I'd missed out on an alternate spelling. Then I realised that was unpossible ;0)>

Too bad, I'm not from Ohio. But this proves my point. Improve spelling? Increase vocabulary? It seems that swimming in the language isn't enough.

Re: Estimate your English vocabulary size

#326

Post by sc00ter »

Eliezer wrote: post #29

I got 37,300. They claim this is not quite 95th percentile, which I am a tad skeptical accurately represents my vocabulary-size percentile relative to the general population. Perhaps this survey is being forwarded around unusually literate people at the top end, or more than 5% of responders are cheating. Where are the fake words to catch cheaters? I Googled a lot of what I didn't recognize, and everything I checked…

By contrast, I scored 23,300 putting me on par with the average result for a 15 year old, if this test is to be believed. I was brutally honest about not selecting words I only recognised and might be able infer meaning from in context, but couldn't articulate a clear definition for (of which there were a surprising number).

And yet I look at the vocabulary used in the comment posts of those claiming high scores, and wonder how they ever scored so highly (accepting that comments are not neccessarily reflective of ones general writing or vocabulary). I believe that as this test is so open to cheating, that using responses to it as the corpus for determining median scores renders the entire exercise completely meaningless.

Re: Estimate your English vocabulary size

#327

Post by rweba »

crazygringo wrote: post #224

Site creator here -- what a surprise to wake up this morning, see my inbox full of messages about this site, and discover that everyone was coming from Hacker News, which I visit every day! Thanks for all the participation, and comments -- I didn't submit this myself, so thanks, mike_esspe. I've responded to a few points down below; there's a lot more info on the details of the test at: http://testyourvocab.com/faq.p…

Hi crazygringo,

I was intrigued by your estimation procedure and was thinking of a way of playing around with it:

(1) Create a random bit vector of size 45,000. (2) Pick 40 positions in the vector at random and use those positions to estimate the proportion of ones in the original bit vector (so far this is easy). (3) Select an additional 120 positions and use the more sophisticated procedure to refine the estimate from part (2).

I was wondering if you have code or pseudo-code for how you implemented part(3) specifically how you choose the 120 words for extra testing. It seems there are a lot of different natural ways you can do it. Did you write an academic paper about this?

Thanks!

Re: Estimate your English vocabulary size

#328

Post by khafra »

Robin_Message wrote: post #324
jtheory wrote: post #319

Earlier quoted context omitted.

There's a huge difference between writing something targeted at a selected audience, in a given time period, limited range of topics the characters will discuss, etc. and listing out all of the vocabulary words you know personally across all domains of knowledge. Even though HP:MoR Harry has a broad vocabulary, for example, and may use some words not all readers will know, there are still tons of topics (with their o…

Thanks to the internet†, I can reveal a surprise: with the same methodology, the dictionary words in the seven real Harry Potter books concatenated is 19,245 and the total unique words is 21,441. Total word count is 1,122,131 which is longer than HP:MoR by a factor of three. Plotting mean unique word count for the whole, halves and quarters of MoR gives a fit of uniques=168*length^0.3357, which makes sense given Zipf…

I thought Eliezer's Lesswrong sequences might give different results. Applying your tests to those (from http://jb55.com/lesswrong/), I get 257,646 total words, 11,666 unique dictionary words, and 12,721 unique words (I'm surprised there aren't more unique words, given that the quantum physics sequence is in there). 168*257,646^.3357 = 11,010, so the sequences seem to be at about HP level.

Excellent work, by the way; thanks for the analysis.

Re: Estimate your English vocabulary size

#329

Post by khafra »

Eliezer wrote: post #29

I got 37,300. They claim this is not quite 95th percentile, which I am a tad skeptical accurately represents my vocabulary-size percentile relative to the general population. Perhaps this survey is being forwarded around unusually literate people at the top end, or more than 5% of responders are cheating. Where are the fake words to catch cheaters? I Googled a lot of what I didn't recognize, and everything I checked…

None of the words were domain-specific; they were all general vocabulary. You're a polymath, but a lot of your knowledge is focused into technical areas that weren't represented on the test.

Re: Estimate your English vocabulary size

#330

Post by kissickas »

Robin_Message wrote: post #315
Eliezer wrote: post #29

I got 37,300. They claim this is not quite 95th percentile, which I am a tad skeptical accurately represents my vocabulary-size percentile relative to the general population. Perhaps this survey is being forwarded around unusually literate people at the top end, or more than 5% of responders are cheating. Where are the fake words to catch cheaters? I Googled a lot of what I didn't recognize, and everything I checked…

I just counted, and you've used at least 12685 dictionary words in HP:MOR [1]. I was expecting it to be more to be honest, but it didn't seem right not to post just because it didn't match my expectations. It certainly seems unlikely that someone who can produce an excellent 500,000 word work of fiction (aside: thanks very much by the way,) in addition to reams of technical writing, has a vocabulary not in the 95th p…

Does this include word derivations?
Post reply on HN