Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Graph of wikipedia articles semantic similarity (LSI, Python, d3.js)

similarityapi.appspot.com

Post reply on HN
Print view View original

11–14 of 14 posts

Re: Show HN: Graph of wikipedia articles semantic similarity (LSI, Python, d3.js)

#11

Post by 3pt14159 »

lucamartinetti wrote: post #2

Small experiment of visualization of wikipedia articles as a graph using d3.js. Articles with more traffic are bigger. I computed the semantic similarity using LSI with python (gensim) You have to scroll down/right a bit! http://similarityapi.appspot.com/graph/?title=blade%20runner There is also a JSON api: http://similarityapi.appspot.com/api/v1/?limit=100&title... All feedback is appreciated: @lucamartinetti luca@l…

I've had much, much better results with LDA than LSI. Give that a shot if you have a chance, you'll be blown away. Stop word ratios are important, and make the max number of tokens 500,000.

Re: Show HN: Graph of wikipedia articles semantic similarity (LSI, Python, d3.js)

#12

Post by Radim »

lucamartinetti wrote: post #2

Small experiment of visualization of wikipedia articles as a graph using d3.js. Articles with more traffic are bigger. I computed the semantic similarity using LSI with python (gensim) You have to scroll down/right a bit! http://similarityapi.appspot.com/graph/?title=blade%20runner There is also a JSON api: http://similarityapi.appspot.com/api/v1/?limit=100&title... All feedback is appreciated: @lucamartinetti luca@l…

how much data did you use for the semantic analysis?

Re: Show HN: Graph of wikipedia articles semantic similarity (LSI, Python, d3.js)

#13

Post by lucamartinetti »

stephengoodwin wrote: post #8

Does the font size for a node represent it's similarity with the query page?

It represents the traffic of the article. Ten most related articles are displayed for each expanded node. Articles with more inbound links are darker

Re: Show HN: Graph of wikipedia articles semantic similarity (LSI, Python, d3.js)

#14

Post by lucamartinetti »

Radim wrote: post #12
lucamartinetti wrote: post #2

Small experiment of visualization of wikipedia articles as a graph using d3.js. Articles with more traffic are bigger. I computed the semantic similarity using LSI with python (gensim) You have to scroll down/right a bit! http://similarityapi.appspot.com/graph/?title=blade%20runner There is also a JSON api: http://similarityapi.appspot.com/api/v1/?limit=100&title... All feedback is appreciated: @lucamartinetti luca@l…

how much data did you use for the semantic analysis?

The whole text of all articles from wikipedia english (then filtered those with more the 1k views last month)
Post reply on HN