Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why it took a long time to build the tiny link preview on Wikipedia

blog.wikimedia.org

Post reply on HN
Print view View original

71–80 of 255 posts

Re: Why it took a long time to build the tiny link preview on Wikipedia

#71

Post by mmarx »

Vinnl wrote: post #58
saagarjha wrote: post #19

Earlier quoted context omitted.

Wikipedia's markup is just terrible for trying to do any sort of scraping or analysis. I once tried to write a script that pulled the latest version of macOS from the sidebar of this article[1] and I gave up because it was difficult and brittle in a way that made it nearly impossible. I'd probably have better results parsing the HTML with a regex. Likewise, I know a friend who literally had to scrap an entire project…

I hit the same thing recently, but that's basically what Wikidata was founded for - and I'm sure it has the latest version of macOS. It's really easy to fetch Wikidata data using the Wikidata API (my example: https://gitlab.com/Flockademic/whereisscihub/blob/master/ind... )

If you're just interested in a single value, using the SPARQL endpoint[0] is probably still more simple, since you don't have to filter out deprecated statements, for example.

[0] http://tinyurl.com/ya957wem

Re: Why it took a long time to build the tiny link preview on Wikipedia

#72

Post by blauditore »

saagarjha wrote: post #36
_delirium wrote: post #14

I had no idea what this was talking about, and it appears to be because they've defaulted it to off for existing logged-in users. Maybe that's a way of reducing pushback. In case you want to turn it on (or off), it's under Preferences->Appearance->Page previews. I think I'll probably leave it off personally. I like the previews that have already existed for a while in the mobile app, but on desktop not so sure.

> they've defaulted it to off for existing logged-in users I see these even when logged out.

Yeah, they're default-on for logged out users, but default-off for logged-in ones. Was the same for me.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#73

Post by lucideer »

omegote wrote: post #63

Not to mention another probable reason: Mediawiki's codebase is a mess and should be rebuilt from the ground up, if possible not in PHP. I once had to build an extension for it and it still gives me the creeps.

> if possible not in PHP

I'm not a fan of PHP as a language, but given the community has been working with PHP for 16 years, it would be odd to switch suddenly to an entirely new language and expect support and adoption.

The codebase is an absolute nightmare though, and a ground-up rebuild would be great. I wonder though about it having a similar affect to the Wordpress codebase: people who recognise the mess stay away completely, and people who don't contribute, leaving a contributor base who isn't really equipped (or extremely willing) to do a quality, maintainable rewrite.

I suspect any rewrite attempt would be doomed to end up being an unmaintainable behemoth.

A better approach would be to focus on secure integration tools and API entry-points, to make users less entrenched and solely dependent on the MW software.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#74

Post by subpixel »

majewsky wrote: post #8

> People seem to like it — we are seeing 5,000 hits to our API a minute to serve those cards that show when you hover over any link. Uhm, no. It just means that people are hovering over links with their mouse. It does not imply any opinion about the previews. > The original idea was conceived four years ago When I was active at Wikipedia/Wikibooks 12 years ago, there was a user script floating around that did the sam…

I do mean to express dislike, as I'm one of the millions of users giving Wikipedia a false metric and sense of satisfaction. I've seen the pop-up hundreds of times, and have considered it a hindrance to my process every single time.

Something that just happens accidentally is a bug, no matter how useful it might be if it were not happening accidentally.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#76

Post by lucideer »

Numberwang wrote: post #26

I personally always use the mobile version https://en.m.wikipedia.org/wiki/Main_Page for desktop these days. I much prefer it to the standard version.

If you have a Wikipedia account you can set this theme as your default theme in your account preferences without having to visit the m. domain.

Re: Why it took a long time to build the tiny link preview on Wikipedia

#77

Post by Numberwang »

lucideer wrote: post #76
Numberwang wrote: post #26

I personally always use the mobile version https://en.m.wikipedia.org/wiki/Main_Page for desktop these days. I much prefer it to the standard version.

If you have a Wikipedia account you can set this theme as your default theme in your account preferences without having to visit the m. domain.

Thank you!

Re: Why it took a long time to build the tiny link preview on Wikipedia

#78

Post by xab9 »

seba_dos1 wrote: post #55
xab9 wrote: post #42

Earlier quoted context omitted.

Using IE11 with the emulator switched to IE5 I can assure you, it works in IE5. At least the main layout, no popup though :D Still equal parts scary and astonishing.

Wikipedia is a content-heavy website first, webapp only second. If it's written correctly, then of course it's going to work in IE5 - maybe with some parts of the layout looking ugly, maybe with some margins being wrong, some media not being embedded, but generally should be usable. Nowadays it's fine for a webdev to not test their pages on IE5, but when the page is done right, there is no reason for it not to work i…

Nope, it's not "done right". It's a table layout from the nineties. But probably there's an api so noone would have to scrape a wiki page, I never needed to do that fortunately :)

Re: Why it took a long time to build the tiny link preview on Wikipedia

#80

Post by Artemis2 »

Learned from this article that they make their Grafana dashboards public: https://grafana.wikimedia.org/dashboard/file/varnish-http-er...

The variety of metrics and the sheer volume of those is awesome to explore! (e.g. frequent peaks to 20M requests per minute)

Post reply on HN