Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

github.com

Post reply on HN
Print view View original

1–10 of 142 posts

Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#1

Post by brig90 »

I built this project as a way to learn more about NLP by applying it to something weird and unsolved.

The Voynich Manuscript is a 15th-century book written in an unknown script. No one’s been able to translate it, and many think it’s a hoax, a cipher, or a constructed language. I wasn’t trying to decode it — I just wanted to see: does it behave like a structured language?

I stripped a handful of common suffix-like endings (aiin, dy, etc.) to isolate what looked like root forms. I know that’s a strong assumption — I call it out directly in the repo — but it helped clarify the clustering. From there, I used SBERT embeddings and KMeans to group similar roots, inferred POS-like roles based on position and frequency, and built a Markov transition matrix to visualize cluster-to-cluster flow.

It’s not translation. It’s not decryption. It’s structural modeling — and it revealed some surprisingly consistent syntax across the manuscript, especially when broken out by section (Botanical, Biological, etc.).

GitHub repo: https://github.com/brianmg/voynich-nlp-analysis Write-up: https://brig90.substack.com/p/modeling-the-voynich-manuscrip...

I’m new to the NLP space, so I’m sure there are things I got wrong — but I’d love feedback from people who’ve worked with structured language modeling or weird edge cases like this.

Show HN: I modeled the Voynich Manuscript with SBERT to test for structure
github.com

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#2

Post by nine_k »

In short, the manuscript looks like a genuine text, not like a random bunch of characters pretending to be a text.

Key Findings

* Cluster 8 exhibits high frequency, low diversity, and frequent line-starts — likely a function word group

* Cluster 3 has high diversity and flexible positioning — likely a root content class

* Transition matrix shows strong internal structure, far from random

* Cluster usage and POS patterns differ by manuscript section (e.g., Biological vs Botanical)

Hypothesis

The manuscript encodes a structured constructed or mnemonic language using syllabic padding and positional repetition. It exhibits syntax, function/content separation, and section-aware linguistic shifts — even in the absence of direct translation.

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#3

Post by andyjohnson0 »

This looks very interesting - nice work!

I have no background in NLP or linguistics, but I do have a question about this:

> I stripped a set of recurring suffix-like endings from each word — things like aiin, dy, chy, and similar variants

This seems to imply stripping the right-hand edges of words, with the assumption that the text was written left to right? Or did you try both possibilities?

Once again, nice work.

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#4

Post by brig90 »

nine_k wrote: post #2

In short, the manuscript looks like a genuine text, not like a random bunch of characters pretending to be a text. Key Findings * Cluster 8 exhibits high frequency, low diversity, and frequent line-starts — likely a function word group * Cluster 3 has high diversity and flexible positioning — likely a root content class * Transition matrix shows strong internal structure, far from random * Cluster usage and POS patte…

Yep, that was my takeaway too — the structure feels too consistent to be random, and it echoes known linguistic patterns.

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#5

Post by brig90 »

andyjohnson0 wrote: post #3

This looks very interesting - nice work! I have no background in NLP or linguistics, but I do have a question about this: > I stripped a set of recurring suffix-like endings from each word — things like aiin, dy, chy, and similar variants This seems to imply stripping the right-hand edges of words, with the assumption that the text was written left to right? Or did you try both possibilities? Once again, nice work.

Great question — and you’re right to catch the assumption there. I did assume left-to-right when stripping suffixes, mostly because that’s how the transliteration files were structured and how most Voynich analyses approach it. I didn’t test the reverse — though flipping the structure and checking clustering/syntax behavior would be a super interesting follow-up. Appreciate you calling it out!

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#8

Post by brig90 »

Avicebron wrote: post #6

Maybe I missed it in the README but how did you do the initial encoding for the "words"? so for example, if you have ""okeeodair" as a word, where do you map that back to original symbols?

Yep, that’s exactly right — the words like "okeeodair" come directly from the EVA transliteration files, which map the original Voynich glyphs to ASCII approximations. So I’m not working with the glyphs themselves, but rather the standardized transliterated words based on the EVA (European Voynich Alphabet) system. The transliterations I used can be found here: https://www.voynich.nu/

I didn’t re-map anything back to glyphs in this project — everything’s built off those EVA transliterations as a starting point. So if "okeeodair" exists in the dataset, that’s because someone much smarter than me saw a sequence of glyphs and agreed to call it that.

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#9

Post by [deleted] »

nine_k wrote: post #2

In short, the manuscript looks like a genuine text, not like a random bunch of characters pretending to be a text. Key Findings * Cluster 8 exhibits high frequency, low diversity, and frequent line-starts — likely a function word group * Cluster 3 has high diversity and flexible positioning — likely a root content class * Transition matrix shows strong internal structure, far from random * Cluster usage and POS patte…

[deleted]

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#10

Post by brig90 »

tetris11 wrote: post #7

UMAP or TSNE would be nice, even if PCA already shows nice separation. Reference mapping each cluster to all the others would be a nice way to indicate that there's no variability left in your analysis

Great points — thank you. PCA gave me surprisingly clean separation early on, so I stuck with it for the initial run. But you’re right — throwing UMAP or t-SNE at it would definitely give a nonlinear perspective that could catch subtler patterns (or failure cases).

And yes to the cross-cluster reference idea — I didn’t build a similarity matrix between clusters, but now that you’ve said it, it feels like an obvious next step to test how much signal is really being captured.

Might spin those up as a follow-up. Appreciate the thoughtful nudge.

Post reply on HN