Yak shaving?
Why it took a long time to build the tiny link preview on Wikipedia
221–230 of 255 posts
Re: Why it took a long time to build the tiny link preview on Wikipedia
#222Those summaries are a -big win-. Just hovering over an unfamiliar term gives you enough to get the gist. Greatly improves the value of the hyperlinks, without any injury to their utility if you want more. This should spread...
Re: Why it took a long time to build the tiny link preview on Wikipedia
#223I can't thank you enough for this feature. Great work peeps!
Re: Why it took a long time to build the tiny link preview on Wikipedia
#224Earlier quoted context omitted.
Also, is 5K hits/minute really that impressive? According to https://stats.wikimedia.org/EN/Sitemap.htm , English Wikipedia gets 88K views per minute.
Number was very very very wrong. Actually it's 0.5 million. I've corrected the post.
it's still shown as 5.5k in grafana. what am i missing? the unit is events per minute.
Re: Why it took a long time to build the tiny link preview on Wikipedia
#225Earlier quoted context omitted.
Number was very very very wrong. Actually it's 0.5 million. I've corrected the post.
it's still shown as 5.5k in grafana. what am i missing? the unit is events per minute.
The sample size for those graphs is 1% so you need to multiple that by 100. I personally checked the access logs to check they are consistent. That was where the confusion came from!
Re: Why it took a long time to build the tiny link preview on Wikipedia
#226Earlier quoted context omitted.
Try to read today's Featured Article in a text-to-speech program. The markup is so mangled it can't even read a complete sentence because it breaks anytime the text links out.
JAWS didn't have any issues for me, which reader are you using?
NVDA - aka the "2nd choice" - but I think it may have been a configuration issue on my end as the markup on the page, after looking into why NVDA was struggling with it, is just a
with some in it which NVDA should handle fine.
Re: Why it took a long time to build the tiny link preview on Wikipedia
#227Earlier quoted context omitted.
it's still shown as 5.5k in grafana. what am i missing? the unit is events per minute.
The sample size for those graphs is 1% so you need to multiple that by 100. I personally checked the access logs to check they are consistent. That was where the confusion came from!
Ah, thank you.
Re: Why it took a long time to build the tiny link preview on Wikipedia
#228I am continually impressed by the markup Wikipedia generates. They've managed to pull in pretty link previews, scientific notation and a grid layout, whilst building a highly nested markup structure? The remarkable part? Wikipedia works great inside a text browser like elinks. It works great in a modern browser. Without sacrificing the interactivity people have grown to expect.
Wikipedia's markup is just terrible for trying to do any sort of scraping or analysis. I once tried to write a script that pulled the latest version of macOS from the sidebar of this article[1] and I gave up because it was difficult and brittle in a way that made it nearly impossible. I'd probably have better results parsing the HTML with a regex. Likewise, I know a friend who literally had to scrap an entire project…
I think some of your issues are just inherent to the fact it's a wiki rather than the design of the markup. I mean I could edit the page just now from "Latest release" to "Latest version" or some such - it's just how wikis are.
Re: Why it took a long time to build the tiny link preview on Wikipedia
#229Earlier quoted context omitted.
> Even if you insist on scraping, in your case you're just looking for a whose immediately preceding contains the text "Latest Release", and that's something any XPath-based scraper can give you straight out of the box Sure, until it changes. Here it is in Jan 2016 when it was included in the opening paragraphs as the text "The latest version of OS X is ". https://en.wikipedia.org/w/index.php?title=MacOS&oldid=69769.…
I can't help feeling like there is a lot of tool blaming happening when the wrong tools were used in the first place. Wikipedia is pretty easy to scrape general blocks of text (I'm the author of an IRC bot which did link previewing, inc Wikipedia) but if you need specific, machine readable, passages which aren't going to change sentence structure over the years then you really should be getting that information from…
> I can't help feeling like there is a lot of tool blaming happening when the wrong tools were used in the first place.
Well, let's be fair: it's a bit surprising that a series of clear, readable key/value pairs in that Wikipedia "MacOS" infobox table can't be delivered by their API as JSON key/value pairs.
Using their API I can generate a JSON that has a big blob sandwiched in there. With the xmlfm format[1] that same blob has some nice-looking "key = value" pairs, too. Funny enough, those pairs for some reason exclude the "latest release" key.
Anyway, is there any case where a containing two columns in the Wikipedia infobox table doesn't hold a key/value pair? That just seems like such a valuable source of data to make available in simple JSON format.
[1] https://en.wikipedia.org/w/api.php?action=query&prop=revisio...
Re: Why it took a long time to build the tiny link preview on Wikipedia
#230They sunk four years of time into this, with developers, UI people, A/B testing... all of it. So they could have thumbnails when you hover over links. And they can't pay the people actually writing the encyclopedia.
I hate our current web sometimes. The only skill it seems to know how to reward is writing code. 99% of the value of Wikipedia has nothing to do with code at all. Yet nobody gets rewarded for that.