Live data from Hacker News

Why it took a long time to build the tiny link preview on Wikipedia

blog.wikimedia.org

211–220 of 255 posts

Re: Why it took a long time to build the tiny link preview on Wikipedia

#211
post #156

Earlier quoted context omitted.

As OP stated, that's tied to your cookies though and won't be a "permanent" solution.

I would expect the setting to persist if you're logged in but that's probably not something people do on Wikipedia.

I do, but that's only because I edit every once in a while.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#213

Earlier quoted context omitted.

> Even if you insist on scraping, in your case you're just looking for a whose immediately preceding contains the text "Latest Release", and that's something any XPath-based scraper can give you straight out of the box Sure, until it changes. Here it is in Jan 2016 when it was included in the opening paragraphs as the text "The latest version of OS X is ". https://en.wikipedia.org/w/index.php?title=MacOS&oldid=69769.…

I can't help feeling like there is a lot of tool blaming happening when the wrong tools were used in the first place. Wikipedia is pretty easy to scrape general blocks of text (I'm the author of an IRC bot which did link previewing, inc Wikipedia) but if you need specific, machine readable, passages which aren't going to change sentence structure over the years then you really should be getting that information from…

> Even if it means having to build your own backend process which polls the websites for the 20 respective OSs individually so you can compile your own API

One caveat there, a page like that for MacOS doesn't exist. Scraping Wikipedia may be insane, but it's often the best option. You can scrape macrumors or something, but then you're still just parsing a site meant to be read by humans. You also still risk those 20 OS websites changing as much as Wikipedia.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#215

Earlier quoted context omitted.

I can't help feeling like there is a lot of tool blaming happening when the wrong tools were used in the first place. Wikipedia is pretty easy to scrape general blocks of text (I'm the author of an IRC bot which did link previewing, inc Wikipedia) but if you need specific, machine readable, passages which aren't going to change sentence structure over the years then you really should be getting that information from…

Agreed. It's some mix of the XY problem plus the self-entitlement of "if I had the idea, then it should work." Yet the classic HNer confuses this for inherent weaknesses in the underlying platform that they then need to share lest someone has something good to say about the platform. And they'll often be using words like "terrible", "garbage", and "I hate..."

Please, no one said Wikipedia was terrible. You're taking statements out of context. The original comment said:

> Wikipedia's markup is just terrible for trying to do any sort of scraping or analysis.

I'd like to emphasize the "for trying to do any sort of scraping or analysis." Should we instead lie and say it's wonderful for scraping?

It's not an insult, it's the truth. If you want to build an app that automatically parses Wikipedia, it will not be easy.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#216

Earlier quoted context omitted.

> Even if you insist on scraping, in your case you're just looking for a whose immediately preceding contains the text "Latest Release", and that's something any XPath-based scraper can give you straight out of the box Sure, until it changes. Here it is in Jan 2016 when it was included in the opening paragraphs as the text "The latest version of OS X is ". https://en.wikipedia.org/w/index.php?title=MacOS&oldid=69769.…

I can't help feeling like there is a lot of tool blaming happening when the wrong tools were used in the first place. Wikipedia is pretty easy to scrape general blocks of text (I'm the author of an IRC bot which did link previewing, inc Wikipedia) but if you need specific, machine readable, passages which aren't going to change sentence structure over the years then you really should be getting that information from…

At the very least, parsing the Release History table seems way better than looking for a particular phrase in the text.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#217
post #169

Earlier quoted context omitted.

IIRC the delay value is set at 500ms before a preview is shown.

It seems like the timer starts as soon as the pointer enters the link area. I would be happier if it was 250ms that starts when the pointer stops moving. Either way I love this feature.

I'd imagine at the scale Wikipedia is operating, they'd have to be sensitive about accessibility features. Some might find it hard to keep the pointer at a compete stop and the 500ms timer seems to do the job pretty well.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#218

I feel like this is some dystopian alternate reality post. It took 4 years to release a hover popup! Take that in for a second. And the post seems extremely proud, and self-congratulatory about it. From the post: > Our initial version wasn’t good enough. Our community asked us not to go ahead with it. We answered by listening to them and making it better. This was 2 years ago, and read the comments on the 39 votes it…

>And the post seems extremely proud, and self-congratulatory about it. Well, yeah. Doing anything successfully at the scale of Wikipedia is worthy of some praise - and I say that as a US midwesterner - a culture not exactly known as the epitome of hubris. :) You might claim I have Stockholm syndrome or something since I worked with the team that developed this feature, but the discussion you highlight did have valid…

Thanks for engaging! To clarify my point, I do think that a the process was followed, and it did lead to some good points but if a process is taking 4+ years to launch something relatively simple (compared to what other companies with similar scale and teams might launch), then the process itself is flawed. I'm respectful of your work, but critical of the system that it operates under.

One could argue that Wikipedia has a broader responsibility to the readers than to just the editors, and such a process gives the editors undue weight in the process. The vocal minority cannot always represent the needs of the silent majority and that role would lie with the product team, which in my understanding hasn't been the case at Wikipedia (I say this, and having interviewed and turned down a Wikipedia PM offer and having a few friends worked in Design at Wikipedia and leaving disillusioned).

Re: Why it took a long time to build the tiny link preview on Wikipedia

#219
post #58

Earlier quoted context omitted.

I hit the same thing recently, but that's basically what Wikidata was founded for - and I'm sure it has the latest version of macOS. It's really easy to fetch Wikidata data using the Wikidata API (my example: https://gitlab.com/Flockademic/whereisscihub/blob/master/ind... )

Unfortunately that data isn't granular enough for what I need: I'm looking for the build number, which Wikipedia somehow keeps up to date.

https://en.wikipedia.org/wiki/MacOS has a footnote after the build number. That led me to https://developer.apple.com/news/releases/. I guess that doesn’t do semantic markup, and I didn’t look at the html at all, but it looks like it could provide you what you want fairly easily (likely not 100% reliably if automated, but chances are Wikipedia‘s somehow keeps up to date involves humans, too)

Alternatively, buy a Mac, set it to auto-update, have cron or launchd reboot it reboot it twice a day, and read the version info from the CLI after rebooting (https://coderwall.com/p/4yz8dq/determine-os-x-version-from-t...)

Re: Why it took a long time to build the tiny link preview on Wikipedia

#220

Earlier quoted context omitted.

>And the post seems extremely proud, and self-congratulatory about it. Well, yeah. Doing anything successfully at the scale of Wikipedia is worthy of some praise - and I say that as a US midwesterner - a culture not exactly known as the epitome of hubris. :) You might claim I have Stockholm syndrome or something since I worked with the team that developed this feature, but the discussion you highlight did have valid…

Thanks for engaging! To clarify my point, I do think that a the process was followed, and it did lead to some good points but if a process is taking 4+ years to launch something relatively simple (compared to what other companies with similar scale and teams might launch), then the process itself is flawed. I'm respectful of your work, but critical of the system that it operates under. One could argue that Wikipedia…

Wikipedia can exist without readers, but not without editors.
Post reply on HN