Live data from Hacker News

Why it took a long time to build the tiny link preview on Wikipedia

blog.wikimedia.org

41–50 of 255 posts

Re: Why it took a long time to build the tiny link preview on Wikipedia

#41
post #9

I am continually impressed by the markup Wikipedia generates. They've managed to pull in pretty link previews, scientific notation and a grid layout, whilst building a highly nested markup structure? The remarkable part? Wikipedia works great inside a text browser like elinks. It works great in a modern browser. Without sacrificing the interactivity people have grown to expect.

Wikipedia's markup is just terrible for trying to do any sort of scraping or analysis. I once tried to write a script that pulled the latest version of macOS from the sidebar of this article[1] and I gave up because it was difficult and brittle in a way that made it nearly impossible. I'd probably have better results parsing the HTML with a regex. Likewise, I know a friend who literally had to scrap an entire project…

Have you tried using the mediawiki API [1], or any of the alternatives?[2]

I don't know how well they work, but the built-in parser should give you the text without markup. And since they switched to Parsoid [3] to support the Visual Editor, the've polished the wikitext formal specification so all instances of markup have a well-defined structure.

[1] https://www.mediawiki.org/wiki/API:Parsing_wikitext

[2] https://www.mediawiki.org/wiki/Alternative_parsers

[3] https://www.mediawiki.org/wiki/Parsoid

Re: Why it took a long time to build the tiny link preview on Wikipedia

#42
post #15

Earlier quoted context omitted.

Last time I checked, their markup was pretty much nightmare fuel, but it should work fine with IE5, I'm sure :)

It works well for the latest (and last) version of IE, at least.

Using IE11 with the emulator switched to IE5 I can assure you, it works in IE5. At least the main layout, no popup though :D Still equal parts scary and astonishing.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#43

I had no idea what this was talking about, and it appears to be because they've defaulted it to off for existing logged-in users. Maybe that's a way of reducing pushback. In case you want to turn it on (or off), it's under Preferences->Appearance->Page previews. I think I'll probably leave it off personally. I like the previews that have already existed for a while in the mobile app, but on desktop not so sure.

> they've defaulted it to off for existing logged-in users I see these even when logged out.

That's what I meant, the previews are on by default for not-logged-in users, but appear to be off by default for logged-in users (or at least the preference was set to 'off' for me).

Re: Why it took a long time to build the tiny link preview on Wikipedia

#44

I personally always use the mobile version https://en.m.wikipedia.org/wiki/Main_Page for desktop these days. I much prefer it to the standard version.

I hate that I don't have the bar on the left to be able to switch to a different language conveniently. I use it all the time and I've no idea if it's even accessible at all in the mobile version. The mobile version also wastes a ton of screen real-estate but I guess that's fashionable these days.

I really enjoy Wikipedia but the #1 item on my feature wishlist is "redirect me towards the non-mobile version of the site when I click a mobile link on the desktop".

Re: Why it took a long time to build the tiny link preview on Wikipedia

#45
post #23
post #8

> People seem to like it — we are seeing 5,000 hits to our API a minute to serve those cards that show when you hover over any link. Uhm, no. It just means that people are hovering over links with their mouse. It does not imply any opinion about the previews. > The original idea was conceived four years ago When I was active at Wikipedia/Wikibooks 12 years ago, there was a user script floating around that did the sam…

> Uhm, no. It just means that people are hovering over links with their mouse. It does not imply any opinion about the previews. When you evaluate features using engagement metrics, there are only two possibilities. If the metric is high, users love the feature. If the metric is low, users don't know the feature exists, and more alerts or "unread" badges must be added to help them learn.

But when the feature only needs you to hover over a link to activate (which people do anyway) you have no basis to claim that this is users "liking" the feature - it might just be noise from everyday regular navigation.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#46

One downside is that when I move the cursor along while reading an article all sorts of links now pop up at me. What I mean is using the mouse or trackpad to keep my place in the article; sometimes I drag the cursor to highlight the text, especially when skimming. Surely I'm not the only one who does this?

The bigger issue in my opinion, at least from the point that it's the wrong behavior 10/10 times, is that these previews pop up when you are just scrolling. Obviously I did not want to read a preview when page under my cursor changes. There should be a check if the cursor had actually moved.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#47

Earlier quoted context omitted.

> they've defaulted it to off for existing logged-in users I see these even when logged out.

That's what I meant, the previews are on by default for not-logged-in users, but appear to be off by default for logged-in users (or at least the preference was set to 'off' for me).

I'm seeing them in both cases.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#48

Can I access this API externally, e.g. by sending a http get request to some endpoint, to get the summary content in JSON format?

Yes you can, there is a nice API returning the Popup-data in JSON

https://stackoverflow.com/a/48527419/4398037

Documentation: https://en.wikipedia.org/api/rest_v1/#!/Page_content/get_pag...

Re: Why it took a long time to build the tiny link preview on Wikipedia

#49
post #17

This thing is not exactly customizable. I couldn't care less for the huge image in the popup layer, scrollability and extra content would be much nicer.

You can switch to the classic Navigation popups gadget,[1] which has a lot more configuration options (including links that shows the history or whole article text in the popup).

https://en.wikipedia.org/wiki/Wikipedia:Tools/Navigation_pop...

Post reply on HN