Live data from Hacker News

Why it took a long time to build the tiny link preview on Wikipedia

blog.wikimedia.org

151–160 of 255 posts

Re: Why it took a long time to build the tiny link preview on Wikipedia

#151

Earlier quoted context omitted.

Wikipedia's markup is just terrible for trying to do any sort of scraping or analysis. I once tried to write a script that pulled the latest version of macOS from the sidebar of this article[1] and I gave up because it was difficult and brittle in a way that made it nearly impossible. I'd probably have better results parsing the HTML with a regex. Likewise, I know a friend who literally had to scrap an entire project…

Can't you use their API? https://en.wikipedia.org/api/rest_v1/#!/Page_content/get_pag...

I haven't used their API in a long time, but I don't think there is a reliable way to get the sidebar that isn't worse than HTML.

For example https://en.wikipedia.org/w/api.php?action=query&prop=revisio...

returns

    {{Infobox website | name = Hacker News | logo = File:hackernews_logo.png | logo_size = 100px | type = [[News aggregator]] | url = {{url|https://news.ycombinator.com/}} | screenshot = File:hn_screenshot.png | registration = Optional | programming_language = [[Arc (programming language)|Arc]] | founder = [[Paul Graham (computer programmer)|Paul Graham]] | launch date = {{start date and age|2007|02|19}} | current status = Online | owner = [[Y Combinator (company)|Y Combinator]] | language = [[English language|English]] }} '''Hacker News''' is a [[social news]] website focusing on [[Computer Science|computer science]] and [[Startup company|entrepreneurship]]. It is run by [[Paul Graham (computer programmer)|Paul Graham]]'s investment fund and startup incubator, [[Y Combinator (company)|Y Combinator]]. In general, content that can be submitted is defined as "anything that gratifies one's intellectual curiosity".{{cite news | first = Paul | last = Graham | title = Hacker News Guidelines | url = http://ycombinator.com/newsguidelines.html | accessdate = 2009-04-29 }}
Which isn't very easy to parse either. From a cursory search, a better format doesn't seem possible without using a 3rd party like dbpedia.

https://www.quora.com/Does-the-Wikipedia-API-give-structured...

I tried it for OSX, and it actually just returns a redirect statement to MacOS. So expect your tool consuming the API to break if you don't handle that in advance.

https://en.wikipedia.org/w/api.php?action=query&prop=revisio...

And then when trying it for MacOS, I can't actually find the version info anywhere in the response data. So you couldn't even get that data without scraping the page.

https://en.wikipedia.org/w/api.php?action=query&prop=revisio...

Re: Why it took a long time to build the tiny link preview on Wikipedia

#152
post #8

> People seem to like it — we are seeing 5,000 hits to our API a minute to serve those cards that show when you hover over any link. Uhm, no. It just means that people are hovering over links with their mouse. It does not imply any opinion about the previews. > The original idea was conceived four years ago When I was active at Wikipedia/Wikibooks 12 years ago, there was a user script floating around that did the sam…

I do mean to express dislike, as I'm one of the millions of users giving Wikipedia a false metric and sense of satisfaction. I've seen the pop-up hundreds of times, and have considered it a hindrance to my process every single time. Something that just happens accidentally is a bug, no matter how useful it might be if it were not happening accidentally.

You can disable it either from the screen, or if you have an account, permanently from your settings.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#154

One of the best things about ad blockers is that they are really just general purpose content blockers. Want to disable this permanently? Here's an Adblock Plus-compatible filter to block the div if you find it annoying. Tested and working on uBlock Origin: ! Disable link preview popups on Wikipedia en.wikipedia.org##.mwe-popups It's just a div with the class ".mwe-popups", and using your ad blocker will persist the…

Also, they are javascript based. They do not appear when running NoScript set to blockk wikipedia's javascript from executing.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#155

There's a lot of negativity in these comments (which is to be expected, as it is still HN after all) but I've been using the preview boxes for a while now and have to say that I absolutely love them. I use Wikipedia a LOT for primative/secondary research and being able to even just figure out the dates someone lived, the very basic information, or even sometimes just a photo saves me from so many instances of new-tab…

Absolutely agree.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#156

Earlier quoted context omitted.

You can disable them by clicking the little gear icon on the pop up itself. I figured that out by googling aroung for half an hour. It's just wretched, backwards UI all around.

As OP stated, that's tied to your cookies though and won't be a "permanent" solution.

I would expect the setting to persist if you're logged in but that's probably not something people do on Wikipedia.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#157

There's a lot of negativity in these comments (which is to be expected, as it is still HN after all) but I've been using the preview boxes for a while now and have to say that I absolutely love them. I use Wikipedia a LOT for primative/secondary research and being able to even just figure out the dates someone lived, the very basic information, or even sometimes just a photo saves me from so many instances of new-tab…

To be honest, it's been like this for a long time. If you haven't noticed the negativity, you've been reading a different "hacker's news" than everyone else on the internet.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#158

There's a lot of negativity in these comments (which is to be expected, as it is still HN after all) but I've been using the preview boxes for a while now and have to say that I absolutely love them. I use Wikipedia a LOT for primative/secondary research and being able to even just figure out the dates someone lived, the very basic information, or even sometimes just a photo saves me from so many instances of new-tab…

It's a very noticeable feature too. I was very delighted to recently discover it, and it really does help Wikipedia to continue to be the revolutionary platform it is. Kudos to those who implemented it.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#159

I personally always use the mobile version https://en.m.wikipedia.org/wiki/Main_Page for desktop these days. I much prefer it to the standard version.

If you have a Wikipedia account you can set this theme as your default theme in your account preferences without having to visit the m. domain.

Mind giving a pointer? Would love to default to the mobile view, but haven't been able to find the setting you're referring to. Google hasn't helped.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#160
This is a great collection of comments! because it exposes a deep bias in the readers, who are mainly coders and dev-culture.

Guess what ? perfectly machine-readable data is called a DATABASE, and it works well in its scope.. and if you think that all of human knowledge, history, arts and culture are perfectly represented in a DATABASE, then congratulations, you are already more computer than human in your preconceptions.

There are several important layers to this onion.. lets call one "participation by non-specialists" .. a second "human factors, aesthetics and publishing arts" .. another is "imperfect intermediate products enable evolution" .. yet another is "taking rules before content" ..

Each topic might be an essay in itself.. Generally, I am happy to see XML essentially proposed as the answer to all human information challenges, because it takes less time to blink than to refute it, for me.

Post reply on HN