Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Britannica11.org – a structured edition of the 1911 Encyclopædia Britannica

britannica11.org

Post reply on HN
Print view View original

41–50 of 138 posts

Re: Britannica11.org – a structured edition of the 1911 Encyclopædia Britannica

#41

Post by hallole »

ahaspel wrote: post #4
logicallee wrote: post #3

Earlier quoted context omitted.

Thanks so much for sharing this. It looks fantastic. A couple of questions, if you don't mind: what license are you releasing this under, if any? Is there any way to download it? The reason someone might want to download it is for use as training data.

Thanks! The underlying text (1911 edition) is public domain, but the structured version here — the parsing, reconstruction, and linking — is something I put together for this site. Right now there isn’t a bulk download available. I’m considering exposing structured access (API or dataset) in some form, but haven’t decided exactly how that will work yet. If you have a specific use case in mind (especially for training…

I've wanted to do something like this for The Encyclopédie, a hugely relevant text to the Enlightenment. If you ever get around to adding a rough "How I (generally) Made This" section, that'd be appreciated! Site looks great :)

Re: Britannica11.org – a structured edition of the 1911 Encyclopædia Britannica

#42

Post by ahaspel »

peterldowns wrote: post #40

I've been meaning to build ~exactly this experience, but for the 1952 Encyclopedia Brittanica Great Books of the World collection and its experimental index Syntopicon [0]. Would love to know more about how you OCR'd or otherwise ingested and parsed the raw material. I have a physical copy of the books, and I found some samizdat raw-image scans and started working on a custom OCR pipeline, but wondering if maybe I co…

I'm familiar with the Synopticon, which would be fun to structure.

I didn’t do OCR myself, except for the topic index and to fill in a few gaps. I started from existing Wikisource text and then built a pipeline around that: cleaning (headers, hyphenation, etc.), detecting article boundaries, reconstructing sections, and linking things back to the original page images. Most of the effort went into rendering the complex layouts, and handling the cross-linking, not the initial ingestion.

Glad to go into more detail if you’re interested, but that’s the gist of it.

Re: Britannica11.org – a structured edition of the 1911 Encyclopædia Britannica

#43

Post by zozbot234 »

logicallee wrote: post #3
ahaspel wrote: post #2

I rebuilt the 1911 Encyclopædia Britannica into a clean, structured, navigable site: https://britannica11.org/ What it does: – ~37k articles reconstructed from the original volumes – section-level structure (contents are clickable within articles) – cross-references extracted and linked – contributors indexed and searchable – original volume + page references preserved and shown while reading – links to the original…

Thanks so much for sharing this. It looks fantastic. A couple of questions, if you don't mind: what license are you releasing this under, if any? Is there any way to download it? The reason someone might want to download it is for use as training data.

Wikisource has the original scans available in the public domain, and their enriched text under CC-BY-SA: https://en.wikisource.org/wiki/EB1911

Re: Britannica11.org – a structured edition of the 1911 Encyclopædia Britannica

#44

Post by ahaspel »

ahmedfromtunis wrote: post #36

No entry on the Great War? Really?!!! Just kidding, of course. This is incredible and surprisingly nostalgic. Reading some of the entries took me right back to being a kid huddled in my room for hours pouring over an encyclopedia or even the dictionary. And I still vividly remember the rush of installing Encarta for the first time on the family PC. I couldn't believe that I, a mere kid, have now access to iconic hist…

I feel exactly the same way about encyclopedias and dictionaries. And Encarta really was amazing. You'd be surprised how much modern criticism of the 11th amounts to "no entry on the Great War", except in earnest.

Re: Britannica11.org – a structured edition of the 1911 Encyclopædia Britannica

#45

Post by zozbot234 »

peterldowns wrote: post #40

I've been meaning to build ~exactly this experience, but for the 1952 Encyclopedia Brittanica Great Books of the World collection and its experimental index Syntopicon [0]. Would love to know more about how you OCR'd or otherwise ingested and parsed the raw material. I have a physical copy of the books, and I found some samizdat raw-image scans and started working on a custom OCR pipeline, but wondering if maybe I co…

That collection is not in the public domain, AIUI? You might be able to do it for the Harvard Classics, which has a nice collection-wide index of terms. https://en.wikisource.org/wiki/The_Harvard_Classics has links to the scans.

Re: Britannica11.org – a structured edition of the 1911 Encyclopædia Britannica

#46

Post by doctor_blood »

Small world - I'm currently cleaning up scans of the EB 9th edition to put it online as a mediawiki site; I'm including all the illustrations and plates so I'm only a third of the way through.

I've been testing different OCR tools and so far I've been the most impressed with paddleOCR - it correctly split the text columns, labled the illustrations, and noted the maragin text.

Still, it's not perfect, so I'm having to hand-edit some tables. I plan to put the source pages online as well so you can switch between the scanned page and the electronic text.

Re: Britannica11.org – a structured edition of the 1911 Encyclopædia Britannica

#47

Post by zozbot234 »

neonscribe wrote: post #34

You can discover beliefs that are shocking today, such as this excerpt from the article "Adolescence": "In the case of girls, let them run, leap and climb with their brothers for the first twelve years or so of life. But as puberty approaches, with all the change, stress and strain dependent thereon, their lives should be appropriately modified. Rest should be enforced during the menstrual periods of these earlier ye…

You can nowadays paste the text from pretty much anything that's in the public domain into a near-SOTA LLM such as Kimi or GLM and it will give you a pretty nice summary of what it's about in modern language (Extremely useful: the LLM tendency to go overboard on formatting nicely balances out the wall-of-text format from historical publications, which was aimed at saving paper and minimizing manual layout effort), and then gladly tell you about all the things in the historical text that would be absolutely beyond the pale today. (Sometimes you have to nudge it by prompting "How would this text be received today?" or something like it after it has put its nice summary in context, but once you do that it tends to be quite thorough.)

Re: Britannica11.org – a structured edition of the 1911 Encyclopædia Britannica

#48

Post by quamserena »

zozbot234 wrote: post #47
neonscribe wrote: post #34

You can discover beliefs that are shocking today, such as this excerpt from the article "Adolescence": "In the case of girls, let them run, leap and climb with their brothers for the first twelve years or so of life. But as puberty approaches, with all the change, stress and strain dependent thereon, their lives should be appropriately modified. Rest should be enforced during the menstrual periods of these earlier ye…

You can nowadays paste the text from pretty much anything that's in the public domain into a near-SOTA LLM such as Kimi or GLM and it will give you a pretty nice summary of what it's about in modern language (Extremely useful: the LLM tendency to go overboard on formatting nicely balances out the wall-of-text format from historical publications, which was aimed at saving paper and minimizing manual layout effort), an…

You can also read the text yourself and draw your own conclusions...

Re: Britannica11.org – a structured edition of the 1911 Encyclopædia Britannica

#49

Post by smallerize »

zozbot234 wrote: post #47
neonscribe wrote: post #34

You can discover beliefs that are shocking today, such as this excerpt from the article "Adolescence": "In the case of girls, let them run, leap and climb with their brothers for the first twelve years or so of life. But as puberty approaches, with all the change, stress and strain dependent thereon, their lives should be appropriately modified. Rest should be enforced during the menstrual periods of these earlier ye…

You can nowadays paste the text from pretty much anything that's in the public domain into a near-SOTA LLM such as Kimi or GLM and it will give you a pretty nice summary of what it's about in modern language (Extremely useful: the LLM tendency to go overboard on formatting nicely balances out the wall-of-text format from historical publications, which was aimed at saving paper and minimizing manual layout effort), an…

You didn't really explain what that does for you. Why do you paste it into an LLM?

Re: Britannica11.org – a structured edition of the 1911 Encyclopædia Britannica

#50

Post by ahmedfromtunis »

ahaspel wrote: post #44
ahmedfromtunis wrote: post #36

No entry on the Great War? Really?!!! Just kidding, of course. This is incredible and surprisingly nostalgic. Reading some of the entries took me right back to being a kid huddled in my room for hours pouring over an encyclopedia or even the dictionary. And I still vividly remember the rush of installing Encarta for the first time on the family PC. I couldn't believe that I, a mere kid, have now access to iconic hist…

I feel exactly the same way about encyclopedias and dictionaries. And Encarta really was amazing. You'd be surprised how much modern criticism of the 11th amounts to "no entry on the Great War", except in earnest.

Thanks a lot for this incredible gem!

By the way, it looks like there's a bug where I can't search for articles when already inside one. To do so, I need to go back to home > articles and then search.

Post reply on HN