https://www.sixdegreesofwikipedia.com/?source=Chaka%20Khan&t...
1 degree between Chaka khan and Ghengis Khan!
Show HN: Six Degrees of Wikipedia
151–160 of 354 posts
Re: Show HN: Six Degrees of Wikipedia
#152Earlier quoted context omitted.
Yeah unfortunately I don't know of any way to differentiate the different types of links. Wikipedia's pagelinks database doesn't different them. I agree it's undesireable but I just cannot figure out how to cull them.
I'm not sure how the backend is structured, but it seems that you must parse the individual pages at some point or another. I took a quick look at the Wikipedia HTML for a few pages and I would suggest stripping out anything within (or nested inside) of classes like "mw-cite-backlink", "reference-text", "citation book", "citation journal", etc. Also, you can probably strip out anything inside of a HTML tag. I'm sure…
I never actually touch any of the source HTML. I think that would simply take way too long and would probably result in some very high bandwidth charges. I use three tables from a public dump of Wikipedia's database, which unfortunately don't differentiate between where the links occur on the page. Check out the first section of my README[1] for more information.
Re: Show HN: Six Degrees of Wikipedia
#153Interesting. I just got sucked into checking a whole bunch of connections. I like it.
Re: Show HN: Six Degrees of Wikipedia
#154Earlier quoted context omitted.
Great work! Do you simply do a BFS to find the shortest paths? If so, are you doing any tricks to avoid the path explosion problem?
Thanks! I'm glad you asked. I actually do what I call a bi-directional breadth first search[1]. The gist of it is that instead of just doing a BFS from the source node until I reach the target node, I do a reverse BFS from the target node as well and wait until the two searches overlap. That helps with the exploding path problem, although that still becomes an issue for longer paths (>= 5 degrees generally). I also p…
When I was first playing with this, I actually really expected you to be using an expensive performant solution like neo4j. When I read that you were using sqlite, I didn't believe it at first and thought it was a mistype from dev until I looked over the source.
That's an impressive and well thought out performance enhancement, and that the app runs so blazingly fast on sqlite is very impressive.
Re: Show HN: Six Degrees of Wikipedia
#155Direct link (== 1 deg) between "Lisp (programming languages)" and "C++". However, 5 degrees between Emacs and Vim!
> However, 5 degrees between Emacs and Vim!
5 degrees if you pick Vim (disambiguation page). If you pick Vim (text editor) you get 1 degree. Which is interesting, since Vim (disambiguation page) links to Vim (text editor).
Re: Show HN: Six Degrees of Wikipedia
#156Not sure if you deliberately designed it this way, but I noticed when spot checking some results that it includes the bibliography section links as connections. This seems like it may not be desirable. Example, I did a search that went from the Crusades to Buzz Aldrin and I noticed that Routledge was the first hop from the Crusades. It strikes me as odd that Routledge (a publishing company) would be mentioned on the…
Yeah unfortunately I don't know of any way to differentiate the different types of links. Wikipedia's pagelinks database doesn't different them. I agree it's undesireable but I just cannot figure out how to cull them.
That might be a problem of wikipedia itself. This project can hardly fix that. I mean, That link has hardly any business of being there; "Oxford University Press" isn't linked either, though the page exists, and why would it?
Re: Show HN: Six Degrees of Wikipedia
#157Anime --> Obesity https://www.sixdegreesofwikipedia.com/?source=Anime&target=O... Somehow, I didn't expect a one-stop layover in Dubai.
I went and made it political: Anime -> Alt-right.
The result was a little more predictable... I guess I shouldn't have had to look it up.
https://www.sixdegreesofwikipedia.com/?source=Anime&target=A...
Re: Show HN: Six Degrees of Wikipedia
#158Creator here. Six Degrees of Wikipedia is a side project I've been sporadically hacking on over the past few years. It was an interesting technical challenge and it's fun to play with the end result. Here's the tech stack: * Frontend: React (Create React App) * Backend: Python Flask * Database: SQLite * Web (frontend) hosting: Firebase Hosting * Server (backend) hosting: Google Compute Engine (it runs fine on a tiny…
Why not use a graph database like Neo4j instead of SQLite? This seems like the perfect use case. Is it because of the resources required to run one versus SQLite?
I actually had a friend suggest it to me and the Neo4j docs happen to be one of the many tabs I currently have open. I was already so far into using SQLite for this project and I wanted to ship it, so I decided to stick with what I had. I would be interested to see how Neo4j performs with such a big dataset (the resulting SQLite file is around 9 GB with nearly 6 million nodes and 500 billion links). I was a bit worried that Neo4j wouldn't be able to scale to a graph of that size, but that is a completely untested and ignorant opinion. If you have any experience with Neo4j, I'd love to hear your thoughts.
Re: Show HN: Six Degrees of Wikipedia
#159Earlier quoted context omitted.
make it so that if I visit the page and just click the "go" button, it will use the placeholder examples as the start and end points. i did this and got an error message stating "You'll probably want to choose the start and end pages before you hit that." that was annoying. the placeholders that were auto chosen were actually really interesting.
'Please' and 'thank you' go a long when requesting additional features for an OSS project.
There's a certain irony to the lack of tact in this post suggesting better manners. I suspect the point would be better received were it more politely made.
I, of course, am merely propagating the cycle.
Re: Show HN: Six Degrees of Wikipedia
#160Actually, interestingly, I've been introducing this concept as a "party game" with other nerds at RL gatherings for some years now. The goal is to start on a random page and find the shortest path to another random page by only clicking links in the articles. It can be quite a lot of fun, despite what you're thinking! And anybody can understand the challenge and compete and have fun. It's not just something for geeks…
What is an RL gathering?
Real Live