Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How can I back up an old vBulletin forum without admin access?

news.ycombinator.com

Post reply on HN
Print view View original

1–10 of 74 posts

Ask HN: How can I back up an old vBulletin forum without admin access?

#1

Post by spike021 »

I'm part of a car community where our vBulletin forum contains a wealth of information.

What we've run into:

1. Admin/owner is nearly / absolutely unreachable, which causes a variety of issues. Mainly we cannot even request a traditional backup of the database underneath the forum software.

2. As with anything, the forums don't get much active engagement other than older forum regulars. However, Google searches easily find useful posts for things like DIY maintenance, modification installs, test data from driving with ECU tunes, track day experiences, etc.

3. It's easy to point people on other social networks at posts by their URL, but due to neglect the website constantly has problems making access increasingly complicated and inconsistent.

Ideally, it'd be nice to find a way to scrape everything as closely as possible into a manageable database.

Even more ideally, if we could convert said scraped data into a format that is easily publishable to a new platform, that would be handy. Even if the new platform is static and simply renders the old threads.

I can't imagine we are the only forum that is experiencing problems like this with most forums probably dying in the last decade.

Has anyone gone through this kind of archival process with vBulletin before?

Thanks.

Re: Ask HN: How can I back up an old vBulletin forum without admin access?

#4

Post by TedDoesntTalk »

The format you want is WARC. Even the Library of Congress uses it. There are many many WARC scrapers. I'd look at what the Internet Archive recommends. A quick search turned up this from the Archive Team and Jason Scott https://github.com/ArchiveTeam/grab-site (https://wiki.archiveteam.org/index.php/Who_We_Are) but I found that in less than 15 seconds of searching so do your own diligence.

Re: Ask HN: How can I back up an old vBulletin forum without admin access?

#5

Post by spike021 »

fexelein wrote: post #3

It seems to me like you don’t own this data. If you want to preserve this data, try again with the owner?

Unfortunately the owner doesn't make themselves available to be contacted for anything at this point. They were more involved years ago.

As far as who owns the data, everything mentioned above is community-owned at least. So if anything it'd be up to individual community members, I'd think, to be OK with it. Since the forum is fairly dead, many of those original members aren't even active either.

Re: Ask HN: How can I back up an old vBulletin forum without admin access?

#6

Post by spike021 »

TedDoesntTalk wrote: post #4

The format you want is WARC. Even the Library of Congress uses it. There are many many WARC scrapers. I'd look at what the Internet Archive recommends. A quick search turned up this from the Archive Team and Jason Scott https://github.com/ArchiveTeam/grab-site ( https://wiki.archiveteam.org/index.php/Who_We_Are ) but I found that in less than 15 seconds of searching so do your own diligence.

Thanks for the links and info. Totally fair on my not sharing those in my original post. I was coming at it thinking in terms of still making it useful to others. WARC seems like a good first step. I'm just not sure what the intermediate steps would be to get something usable like a vBulletin -- basically with the intention of being able to continue sharing the archived stuff with users who may not be as technical and only know how to consume from a forum format if that makes sense.

Thanks again.

Re: Ask HN: How can I back up an old vBulletin forum without admin access?

#8

Post by emj »

spike021 wrote: post #5
fexelein wrote: post #3

It seems to me like you don’t own this data. If you want to preserve this data, try again with the owner?

Unfortunately the owner doesn't make themselves available to be contacted for anything at this point. They were more involved years ago. As far as who owns the data, everything mentioned above is community-owned at least. So if anything it'd be up to individual community members, I'd think, to be OK with it. Since the forum is fairly dead, many of those original members aren't even active either.

Depends on where the data is stored, if the owner of the domain lives in the EU or a similar jurisdiction the data will be under Database rights for 15 years. There is no such thing as community owned. Each post might be owned by the person who posted it but often that is not the case.

EDIT: so of course you should dump the data, wget -r and dump it.

Re: Ask HN: How can I back up an old vBulletin forum without admin access?

#9

Post by toyg » 

   1. Scrape it. Plenty of options here.
   2. set up a new forum. 
      I think the current state of the art is Discourse but I could be wrong.
   3. automate the recreation of posts and threads with some backend script
      (will depend on which sw you picked). 
      On each post, add a link to the original.
   4. Tell everyone about your superduper clone, move the old-timers over.
   5. ...
   6. Profit.

Re: Ask HN: How can I back up an old vBulletin forum without admin access?

#10

Post by quesera »

I would make a read-only archive first, using `wget --mirror`.

This will fix relative paths, download assets, etc and can be published as-is on a new site. I'm ignoring copyright questions in the interest of archiving fragile data.

Then I'd use an HTML parser against the local archive to extract the individual posts, if the additional work was justified.

Post reply on HN