What's the story here and why is this newsworthy?
The Unreal Engine Wiki is now permanently offline
111–120 of 137 posts
Re: The Unreal Engine Wiki is now permanently offline
#112The lack of a static copy of the wiki really sucks but it's understandable that a mediawiki install would be pulled indefinitely. Mediawiki racks up a dozen CVEs in an average year and even a single one of those is an opportunity to perform watering hole attacks on every UE licensee. Getting RCE on a single UE customer's machine is an opportunity for million+-dollar industrial espionage - it's not uncommon for someone to get a copy of a game's source code and try to extort the developer for cash. We generally only know about the cases where the extortion fails...
It's possible that really aggressive security measures could mostly prevent that but even if you were to patch weekly that won't stop someone from pairing an undisclosed mediawiki attack with some other attack that isn't well-known. A game studio's machines are probably using LTS versions of Firefox or Chrome w/slower update cadence, which potentially means multiple days of vulnerability even after an exploit is patched.
Also, now that Epic processes credit card payments (Epic Store, etc) it's possible the mediawiki install would prevent them from passing PCI-DSS audits.
Re: The Unreal Engine Wiki is now permanently offline
#113Earlier quoted context omitted.
Looks like the post got updated with some reasoning: > Why can’t we put a read-only archive online currently? The Wiki, even in it’s read-only state, was presenting security risks, and it was deemed necessary to take it offline.
So crawl the Wiki pages, grind out static HTML copies of them, and make those available. Not many security risks associated with static HTML. What am I missing here?
I read it's very easy to that nowadays with Chrome's Puppeteer.
Re: The Unreal Engine Wiki is now permanently offline
#114Earlier quoted context omitted.
So crawl the Wiki pages, grind out static HTML copies of them, and make those available. Not many security risks associated with static HTML. What am I missing here?
I read it's very easy to that nowadays with Chrome's Puppeteer.
Easier still with wget perhaps? https://www.gnu.org/software/wget/manual/wget.html#Very-Adva...
Re: The Unreal Engine Wiki is now permanently offline
#115Earlier quoted context omitted.
im surprised at myself for shilling a service i dont even use and dont have any stake in, but https://pinboard.in/ lets you bookmark things and then also archive those things so they dont go away
That site could do the same thing as the wiki.
Not impossible, but it's a big part of how the site makes money.
Re: The Unreal Engine Wiki is now permanently offline
#116Earlier quoted context omitted.
im surprised at myself for shilling a service i dont even use and dont have any stake in, but https://pinboard.in/ lets you bookmark things and then also archive those things so they dont go away
That site could do the same thing as the wiki.
You can download a complete archive of all the bookmarks and pages: https://alexwlchan.net/2017/07/backing-up-pinboard-archives/
Re: The Unreal Engine Wiki is now permanently offline
#117Earlier quoted context omitted.
I think you're discounting the value of the tools Unity and UE offer to experienced developers/teams, and ignoring that outside a few exceptions, Indies haven't really been able to compete with AAA Devs since the PS2 era, when games more or less got programmed from scratch, or were adapted from the brittle code of previous titles. Nearly every major studio or publisher has a similar toolset they've either built from…
Excellent post, but in the context of this discussion, I disagree on one point: > a mature engine like UE, or Unity has a wealth of crowdsourced documentation, and it's almost impossible to trip over an issue that someone else hasn't already triaged before you C++ in UE4 is a nightmare to work with because you have to scour forums for a day and a half to find somebody who might have mentioned the name of the function…
Thanks, and quite right. An oversight on my part. Admittedly, programming for Unreal's well outside my realm of experience. I tried it some time ago, but mostly played with it as an art tool. Its coding conventions seemed more opinionated relative to Godot and Unity's, which turned me off, because I was only really looking for a fun toy in a domain outside my experience.
Re: The Unreal Engine Wiki is now permanently offline
#118Earlier quoted context omitted.
That site could do the same thing as the wiki.
You can download a complete archive of all the bookmarks and pages: https://alexwlchan.net/2017/07/backing-up-pinboard-archives/
I tried this in the past, but could not even get it to compile. Can you share your working setup, or have you just found this?
Re: The Unreal Engine Wiki is now permanently offline
#119I had to decom a major wiki in a very large company not long ago. I can give you four reasons why these things happen, possibly without notice. 1) The wiki software and database have been abandoned for years. There is no maintenance and no further release. 2) It will stop working shortly. Like, it doesn't start on Ubuntu > 14 or current MySql at all. 3) It already stopped working and/or the content is already lost. C…
It shouldn't be hard to save a static copy of the actual HTML and replace the dynamic site with that though. Or at least give others enough time to do the same.
It's not hard, except when it is and it takes weeks of work. From my own experience:
1) httrack wasn't able to crawl anything for some reasons. wget only with special flags available in the latest version.
2) All the links are broken. Wiki have their own linking system between pages that are processed on the fly. The archive with all links hardcoded to wikidump.example.com/wikidump/page really doesn't work in place.
3) More challenges with saving pictures and attachments.
4) Unicode characters in some pages, that broke the page and/or the crawler.
5) Infinite crawling and a ton of useless pages. Consider that every URL with a query string is a separate page. /page?version=50&comparewith=49
6) Crawling large amount of documents takes forever. Could be an hour wasted on each try. Consider tens or hundreds of thousands of unique URLs to save, see point five. Really wish the crawler could parallelize and run on the same host as the wiki.
Re: The Unreal Engine Wiki is now permanently offline
#120Earlier quoted context omitted.
Lesson I've long learned: never bookmark anything. Bookmarks are to temporarily remember a URL, not to archive content. If the content is important or meaningful to you, save the page.
Too bad modern single-page apps make it very hard to just scrape the site.
For some inexplicable reason some pages add script tags with javascript. So what happens is that you archive a page. The page is gone and when you visit it you notice all the broken links pointing to javascript files that are gone as well.
Archiving even just a single webpage is non-trivial. You're dependent on the internet archive if you want a reliable backup.