BlaBlaMeter detects how much bullshit is in your text • Hacker News Board

Re: BlaBlaMeter detects how much bullshit is in your text

#41

Post by slashcom » Thu, Aug 30, 2012, 1:43 PM UTC

shanelja wrote: post #3

Any chance of an overview of the algorithm you're using to filter out the text? My thinking is you are measuring word count versus commonly used marketing or political jargon count, but that's probably too simple.

Here's a few things I've gleamed from experimenting with it:

- It uses a unigram language model. You can take the same text, randomly permute the words, and you get the same score. This means it also can't be using things like POS tagging, phrases, etc.

- It normalizes words by making all letters lowercase. The exact same text in all upper case has the same score.

- The score is eventually normalized by the length of the text. The same text copied multiple times gets the same score.

- It does not form a valid probability distribution, as someone's managed to get some 1.16's. This makes me believe it's not a Naive Bayes classifier giving you the P(Bullshit|Text). Though this is what I originally thought it would be.

Re: BlaBlaMeter detects how much bullshit is in your text

#42

Post by JoeAltmaier » Thu, Aug 30, 2012, 1:44 PM UTC

I'm writing a story; its a 0.12, acceptable.

Re: BlaBlaMeter detects how much bullshit is in your text

#43

Post by skeltoac » Thu, Aug 30, 2012, 1:48 PM UTC

The text of the site itself scores a surprisingly low 0.14: "Your text shows only a few indications of 'bullshit'-English."

This is not a useful tool because it does not define or point to the problems in the text.

Re: BlaBlaMeter detects how much bullshit is in your text

#44

Post by brianjyee » Thu, Aug 30, 2012, 1:49 PM UTC

It would be nice if they gave even the smallest hint of what they're measuring.

Re: BlaBlaMeter detects how much bullshit is in your text

#45

Post by slashcom » Thu, Aug 30, 2012, 1:50 PM UTC

DallaRosa wrote: post #25

grabeh wrote: post #12

The main reason I posted was in the hope of triggering discussion over the method used to analyse the text. I've seen users getting a lower score simply by separating out a block of text into numbered paragraphs which would seem to point to quite a simplistic method. http://ipdraughts.wordpress.com/2012/08/25/cutting-down-on-t...

I'll withstand my statement: model based on a corpus of PR, scholar, licenses and the like texts. If they are into real statistical NLP. Or just esthetic rules + word dictionary.

If I were to make the software, the corpus of PR, licenses, etc. would be the way I go. But "they did it statistically" doesn't answer the question "what is the model?" There are many different statistical models one could use. My other post has a few things we've figured out.

But I'm starting to think a rule-based lexicon isn't out of the question, given these >1 scores on some texts.

Re: BlaBlaMeter detects how much bullshit is in your text

#46

Post by nhebb » Thu, Aug 30, 2012, 1:50 PM UTC

kitsune_ wrote: post #37

President's Obama's AMA answers score 0.21 (Your text shows some indications of 'bullshit'-English, but is still within an acceptable range.)

I tried that too, along with Paul Ryan's speech (which got 0.14 - Your text shows only a few indications of 'bullshit'). Then, I tried the copy on my landing page and got a 0.25. It's shameful to realize that I'm a bigger bullshitter than two politicians.

Re: BlaBlaMeter detects how much bullshit is in your text

#47

Post by makmanalp » Thu, Aug 30, 2012, 1:50 PM UTC

derrida wrote: post #16

Derrida, Chapter 2 'Of Grammatology' >Bullshit Index :0.26 Your text shows some indications of 'bullshit' some indications ... I'd say this thing is broken.

You're just bitter that that era of philosopher in general used an artificially complicated writing style, complete with invented terminology, that borders bullshit :)

Re: BlaBlaMeter detects how much bullshit is in your text

#48

Post by ff0066mote » Thu, Aug 30, 2012, 1:52 PM UTC

"I am a beautiful flower. I sway in the breeze of my meadow, drinking in the sunshine. I love the little bees who come to visit me."

---

Your text: 130 characters, 28 words Bullshit Index :0.05 Your text shows no or marginal indications of 'bullshit'-English.

Re: BlaBlaMeter detects how much bullshit is in your text

#49

Post by ff0066mote » Thu, Aug 30, 2012, 1:56 PM UTC

ff0066mote wrote: post #48

"I am a beautiful flower. I sway in the breeze of my meadow, drinking in the sunshine. I love the little bees who come to visit me." --- Your text: 130 characters, 28 words Bullshit Index :0.05 Your text shows no or marginal indications of 'bullshit'-English.

I was simply demonstrating, as everyone else in this thread is, that the tool doesn't match one's intution of what is bullshit.

Re: BlaBlaMeter detects how much bullshit is in your text

#50

Post by tokenadult » Thu, Aug 30, 2012, 1:59 PM UTC

My longest FAQ for HN

http://news.ycombinator.com/item?id=4270768

got

"Your text: 8044 characters, 1160 words Bullshit Index :0.17 Your text shows only a few indications of 'bullshit'-English."

I too would like to know what the model is for the online ratings. Has there been a validation study of the model?

AFTER EDIT: Paul Graham's essay "Why Nerds Are Unpopular"

http://www.paulgraham.com/nerds.html

(which is the first writing of his that I ever read) appears to reach the maximum length (by character count) that the program will evaluate, and comes out like this:

"Your text: 15000 characters, 2698 words Bullshit Index :0.09 Your text shows no or marginal indications of 'bullshit'-English."

Maybe Paul's procedure of having friends look over his essays and give suggestions helps cut out the bullshit.