Live data from Hacker News

Why it took a long time to build the tiny link preview on Wikipedia

blog.wikimedia.org

61–70 of 255 posts

Re: Why it took a long time to build the tiny link preview on Wikipedia

#61
post #9

I am continually impressed by the markup Wikipedia generates. They've managed to pull in pretty link previews, scientific notation and a grid layout, whilst building a highly nested markup structure? The remarkable part? Wikipedia works great inside a text browser like elinks. It works great in a modern browser. Without sacrificing the interactivity people have grown to expect.

Wikipedia's markup is just terrible for trying to do any sort of scraping or analysis. I once tried to write a script that pulled the latest version of macOS from the sidebar of this article[1] and I gave up because it was difficult and brittle in a way that made it nearly impossible. I'd probably have better results parsing the HTML with a regex. Likewise, I know a friend who literally had to scrap an entire project…

How hard did you try?

These are not hand-written pages, and their output is actually pretty clean compared to the crazy things I've tried to scrape. They have tons of APIs to access the backing data, and that should be your first stop.

Even if you insist on scraping, in your case you're just looking for a whose immediately preceding contains the text "Latest Release", and that's something any XPath-based scraper can give you straight out of the box[1].

A more resilient choice, if you still insist on (or have to use) scraping, is use the underlying template data - a regex is good enough in that case[2]

[1] e.g. http://html-agility-pack.net/ [2] https://en.wikipedia.org/w/index.php?title=Template:Latest_s...

Re: Why it took a long time to build the tiny link preview on Wikipedia

#62
post #8

> People seem to like it — we are seeing 5,000 hits to our API a minute to serve those cards that show when you hover over any link. Uhm, no. It just means that people are hovering over links with their mouse. It does not imply any opinion about the previews. > The original idea was conceived four years ago When I was active at Wikipedia/Wikibooks 12 years ago, there was a user script floating around that did the sam…

I personally do not care for the link preview feature as I am one of those people who like to hover over a link so I can see the url leads (which is also why I detest URL shorteners).

I also have to wonder how much bandwidth Wikipedia is going to end up using to display unneeded previews.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#64
post #24
post #8

> People seem to like it — we are seeing 5,000 hits to our API a minute to serve those cards that show when you hover over any link. Uhm, no. It just means that people are hovering over links with their mouse. It does not imply any opinion about the previews. > The original idea was conceived four years ago When I was active at Wikipedia/Wikibooks 12 years ago, there was a user script floating around that did the sam…

Also, is 5K hits/minute really that impressive? According to https://stats.wikimedia.org/EN/Sitemap.htm , English Wikipedia gets 88K views per minute.

I imagine most people spend more time reading on Wikipedia as opposed to trying to hover over every link available?

Re: Why it took a long time to build the tiny link preview on Wikipedia

#66
post #28
post #24

Earlier quoted context omitted.

Also, is 5K hits/minute really that impressive? According to https://stats.wikimedia.org/EN/Sitemap.htm , English Wikipedia gets 88K views per minute.

I'm thinking it has to be heavily cached.

"5000 hits to our API per minute" does not say anything about whether they're measuring in front of or behind the cache. For all intents and purposes, the cache's endpoint is the API endpoint.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#67

One downside is that when I move the cursor along while reading an article all sorts of links now pop up at me. What I mean is using the mouse or trackpad to keep my place in the article; sometimes I drag the cursor to highlight the text, especially when skimming. Surely I'm not the only one who does this?

At the bottom of every popup there is a cog icon. Clicking the cog icon gives a popup where you can choose to Disable and Enable the feature.

Thanks. I’ll do that!

Re: Why it took a long time to build the tiny link preview on Wikipedia

#68
post #52
post #44

Earlier quoted context omitted.

I hate that I don't have the bar on the left to be able to switch to a different language conveniently. I use it all the time and I've no idea if it's even accessible at all in the mobile version. The mobile version also wastes a ton of screen real-estate but I guess that's fashionable these days. I really enjoy Wikipedia but the #1 item on my feature wishlist is "redirect me towards the non-mobile version of the sit…

It's at the top of each page - click the icon directly under the article title (on the left) that looks like a Chinese character next to an A. I'd snip a screenshot, but I'm on mobile...

> I'd snip a screenshot, but I'm on mobile...

If you've got a smart phone chances are you can probably take a screenshot, only hassle then is uploading it somewhere.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#69
post #32
post #23

Earlier quoted context omitted.

> Uhm, no. It just means that people are hovering over links with their mouse. It does not imply any opinion about the previews. When you evaluate features using engagement metrics, there are only two possibilities. If the metric is high, users love the feature. If the metric is low, users don't know the feature exists, and more alerts or "unread" badges must be added to help them learn.

Alternatively, if it's a feature you don't like: If the metric is high, users are spending too much time on it. Also, it is responsible for every single frustration that users have with your product. If the metric is low, the feature can be safely removed (see: Windows Start menu).

> If the metric is low, the feature can be safely removed (see: Windows Start menu).

That turned out really well...

Whoever is responsible for putting the touch start screen on Windows Server is responsible for 90% of my frustrations...

Post reply on HN