Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How can I back up an old vBulletin forum without admin access?

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 74 posts

Re: Ask HN: How can I back up an old vBulletin forum without admin access?

#11

Post by spike021 »

emj wrote: post #8
spike021 wrote: post #5

Earlier quoted context omitted.

Unfortunately the owner doesn't make themselves available to be contacted for anything at this point. They were more involved years ago. As far as who owns the data, everything mentioned above is community-owned at least. So if anything it'd be up to individual community members, I'd think, to be OK with it. Since the forum is fairly dead, many of those original members aren't even active either.

Depends on where the data is stored, if the owner of the domain lives in the EU or a similar jurisdiction the data will be under Database rights for 15 years. There is no such thing as community owned. Each post might be owned by the person who posted it but often that is not the case. EDIT: so of course you should dump the data, wget -r and dump it.

Ahhh. Sorry, hadn't thought of rights that way since that isn't my responsibility in my day to day work. Thanks for the explanation.

Re: Ask HN: How can I back up an old vBulletin forum without admin access?

#12

Post by russnewcomer »

quesera wrote: post #10

I would make a read-only archive first, using `wget --mirror`. This will fix relative paths, download assets, etc and can be published as-is on a new site. I'm ignoring copyright questions in the interest of archiving fragile data. Then I'd use an HTML parser against the local archive to extract the individual posts, if the additional work was justified.

An acquaintance did basically this a few years ago on a vBulletin.

My memory says there was a project to be a vB->Discourse converter that didn't rely on DB access, but I can't immediately find it.

Re: Ask HN: How can I back up an old vBulletin forum without admin access?

#13

Post by parasti »

We (community for a video game) lost over a decade of accumulated community content due to an unreachable owner. This happened as I was considering scraping, but did not get a chance to implement it. Internet Archive has been a godsend - a lot of public content that was served in a text-based format is available from there. Due to a PHP misconfiguration, even a bunch of binary files were archived because they were being served with PHP errors.

Re: Ask HN: How can I back up an old vBulletin forum without admin access?

#14

Post by ZoomerCretin »

russnewcomer wrote: post #12
quesera wrote: post #10

I would make a read-only archive first, using `wget --mirror`. This will fix relative paths, download assets, etc and can be published as-is on a new site. I'm ignoring copyright questions in the interest of archiving fragile data. Then I'd use an HTML parser against the local archive to extract the individual posts, if the additional work was justified.

An acquaintance did basically this a few years ago on a vBulletin. My memory says there was a project to be a vB->Discourse converter that didn't rely on DB access, but I can't immediately find it.

> there was a project to be a vB->Discourse converter

God, why? I can't be the only person to find Discourse inferior to vBulletin.

Re: Ask HN: How can I back up an old vBulletin forum without admin access?

#15

Post by ZoomerCretin »

toyg wrote: post #9

1. Scrape it. Plenty of options here. 2. set up a new forum. I think the current state of the art is Discourse but I could be wrong. 3. automate the recreation of posts and threads with some backend script (will depend on which sw you picked). On each post, add a link to the original. 4. Tell everyone about your superduper clone, move the old-timers over. 5. ... 6. Profit.

vBulletin just had a major release 5 months ago. It's not abandoned, and it's certainly superior to the UI of Discourse, especially to a community which is very familiar with vBulletin.

Re: Ask HN: How can I back up an old vBulletin forum without admin access?

#16

Post by fullspectrumdev »

Two options:

1. Scrape it with wget or httrack or similar tool.

2. If the owners not really around it’s probably behind on its security patches, and there’s some relatively recent-ish vB exploits that would let you gain code execution and take a backup the “extremely illegal way” of the entire database, site, etc.

I recommend 1, but 2 is amusing to ponder briefly over a coffee ;)

Re: Ask HN: How can I back up an old vBulletin forum without admin access?

#17

Post by toyg »

ZoomerCretin wrote: post #15
toyg wrote: post #9

1. Scrape it. Plenty of options here. 2. set up a new forum. I think the current state of the art is Discourse but I could be wrong. 3. automate the recreation of posts and threads with some backend script (will depend on which sw you picked). On each post, add a link to the original. 4. Tell everyone about your superduper clone, move the old-timers over. 5. ... 6. Profit.

vBulletin just had a major release 5 months ago. It's not abandoned, and it's certainly superior to the UI of Discourse, especially to a community which is very familiar with vBulletin.

Fair enough, I honestly don't know the field. The point is not to change software as much as domain/instance, really.

Re: Ask HN: How can I back up an old vBulletin forum without admin access?

#18

Post by duskwuff »

ZoomerCretin wrote: post #14
russnewcomer wrote: post #12

Earlier quoted context omitted.

An acquaintance did basically this a few years ago on a vBulletin. My memory says there was a project to be a vB->Discourse converter that didn't rely on DB access, but I can't immediately find it.

> there was a project to be a vB->Discourse converter God, why? I can't be the only person to find Discourse inferior to vBulletin.

vBulletin is commercial software; Discourse is open source.

While I'd agree that Discourse isn't ideal, it's one of the better options available in the open source world.

Re: Ask HN: How can I back up an old vBulletin forum without admin access?

#20

Post by CharlesW »

spike021 wrote: post #6
TedDoesntTalk wrote: post #4

The format you want is WARC. Even the Library of Congress uses it. There are many many WARC scrapers. I'd look at what the Internet Archive recommends. A quick search turned up this from the Archive Team and Jason Scott https://github.com/ArchiveTeam/grab-site ( https://wiki.archiveteam.org/index.php/Who_We_Are ) but I found that in less than 15 seconds of searching so do your own diligence.

Thanks for the links and info. Totally fair on my not sharing those in my original post. I was coming at it thinking in terms of still making it useful to others. WARC seems like a good first step. I'm just not sure what the intermediate steps would be to get something usable like a vBulletin -- basically with the intention of being able to continue sharing the archived stuff with users who may not be as technical an…

> I'm just not sure what the intermediate steps would be to get something usable like a vBulletin…

Once you have an archive, you can convert that unstructured data to structured data. For example, if I look at https://www.vbulletin.org/forum/showthread.php?t=326241, the thread title and hierarchy is in , posts are in , etc. I see an old project (https://github.com/IanLondon/detectorist-scraper) that may be a useful place to start, and I imagine there have been other similar efforts.

Once you have a structured representation (in a database, in JSON/XML files, etc.), you can decide whether to use it to build a static site, to import it into other forum software, etc.

Post reply on HN