A friend of mine, technical but not a hacker, wanted to turn an old offline forum into a chatbot. I know there are many not so good RAGs to do this. What are the current best practices, as of this month?
Ask HN: What to do with text from old, unarchived, online forums?
41–50 of 112 posts
Re: Ask HN: What to do with text from old, unarchived, online forums?
#42Earlier quoted context omitted.
>A question from someone a bit younger: I mean.... likely more than "a bit" younger :) 1. Computer not always available. A lot of the time it could be a "Family Computer". Not "Personal Computer". 2. There were far more detailed discussions than today, but computer are not portable so you want to read it may be while you are in bed. 3. Access information has a cost. Not only does it take time on a dial up modem. Tran…
Well, I'm over thirty, so I do remember metered connections and life without a smartphone. The shared computer reason explains it completely, thanks :)
From your initial comment I assumed a 14 year old to be fair.
Re: Ask HN: What to do with text from old, unarchived, online forums?
#43You can scan and OCR (adobe) this yourself with not much more effort than boxing and shipping at FedEx Then repost on related public-facing online forums or a free wordpress site or upload to archive.org
I get this answer a lot when I ask similar question but how do we scan it? Taking a Photos of every print out from our Phone? That is 1000s of photos each needing to snap and focus. Not everyone has a scanner, and not every scanner has automated tray feed. And even if they do the quality of their scan are really low by default. I have similar question with Business Cards as well. I have hundreds if not thousands of t…
I use https://1dollarscan.com/ if I don't need the physical items back, and use https://www.scancafe.com/ if I do. Those digital files then get ingested into my workflow.
(no affiliation, just a customer)
Re: Ask HN: What to do with text from old, unarchived, online forums?
#44Earlier quoted context omitted.
? Saving locally to a desktop computer surely existed back then, I even did it myself sometimes.
But then you have to be seated at that desktop computer to read it
Indeed.
I was just pointing out that saving locally (the original question) has nothing to do with googling about laptops and smartphones (the condescending reply).
Re: Ask HN: What to do with text from old, unarchived, online forums?
#45Earlier quoted context omitted.
What does that have to do with anything? Disk drives did exist.
They did but were rather small and unreliable - for instance around 300-500 MB and could easily die. People printed stuff for the same reason books exist - storage and mobility, neither of which were easily available. I used to print source code just to review in silence. Might print backup emails to ensure durability.
> for instance around 300-500 MB
I mean, that's more than enough for text downloaded from a usenet group. Printing that much plaintext out would be something.
Re: Ask HN: What to do with text from old, unarchived, online forums?
#46Hey there, I do some work with the Archive folks as an ad hoc physical archive volunteer. I'll pay for you to ship them and facilitate getting them ingested into items and a collection if you'd like. Let me know how to get in touch. Once scanned, derive operations will take care of OCR and generating relevant metadata from the artifacts. If there are any issues (copyright, etc) preventing these items from being publi…
Do you take MYSQL backups of complete BBS sites?
Re: Ask HN: What to do with text from old, unarchived, online forums?
#47Earlier quoted context omitted.
Ok, but why print it out instead of saving it locally? (Other than to read it on the go? Did people do that?)
Very often you could be using a 'fast' internet connection (T1 at school, library), and you could easily download an entire floppy disk in a minute or two. And it was unlikely you had a CD-RW drive (maaaybe a Zip disk), so it'd be hard to take it home and you wouldn't be able to save it locally. Heck, even if you could save it locally, you might have only had a 500MB hard drive, it seems weird now because with TB dri…
> you might have only had a 500MB hard drive
I mean that was already a massive amount of storage for plain text.
> you could easily download an entire floppy disk in a minute or two
And how does printing it on paper help with that? If anything, switching out the floppies and carrying them home is faster and easier than with hundreds (or thousands) of pages.
But I get your overall point of transferring the content home.
Re: Ask HN: What to do with text from old, unarchived, online forums?
#48Hey there, I do some work with the Archive folks as an ad hoc physical archive volunteer. I'll pay for you to ship them and facilitate getting them ingested into items and a collection if you'd like. Let me know how to get in touch. Once scanned, derive operations will take care of OCR and generating relevant metadata from the artifacts. If there are any issues (copyright, etc) preventing these items from being publi…
Do you take MYSQL backups of complete BBS sites?
[deleted]
Re: Ask HN: What to do with text from old, unarchived, online forums?
#49A question from someone a bit younger: What was the purpose of printing out online discussions? Was it to read them while not at the computer? Was it to physically archive them?
Ah, memories of printing out the entire DooM FAQ because reading it on a small screen was annoying and not always available, other people needed to use the computer, too.
Re: Ask HN: What to do with text from old, unarchived, online forums?
#50Earlier quoted context omitted.
Ok, but why print it out instead of saving it locally? (Other than to read it on the go? Did people do that?)
I do wonder about the relative cost of paper to digital storage mediums of the time. It could be that would have something to do with it. Take for example the fact that (testing with a 150kb block of lorem ipsum text) you could easily squeeze the text capacity of a 3.5mm floppy into around 20 pages of physical text. 10 sheets of double-side printed paper and some ink might well have been cheaper than the floppy it wo…
Printing was mostly about the cost of paper (and the amortized cost of the printer). Dot matrix ribbons were very cheap. Without deeply diving into them vs. now printer costs I’m guessing that dot matrix then vs. B&W laser now were similar in inflation-adjusted dollars and inkjets today probably actually cost more.