Live data from Hacker News

Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

github.com

121–130 of 142 posts

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#121

Earlier quoted context omitted.

Even before we consider the cipher, there's a huge difference between a constructed language and a stochastic process to generate language-like text.

A stochastic pattern to generate language-like text in the early 1400s is a lot more interesting than gibberish.

I ching is similar conceptually to llms today (ancient chinese)

https://en.wikipedia.org/wiki/I_Ching

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#122
I had a look at the manuscript for a while and found it suspicious how tightly packed the writing was against the illustrations on some pages. In common language words and letters vary in width, so when you approach the end of the line when writing, you naturally insert a break to begin a new word and avoid overrun. The manuscript is missing these kinds of breaks - I saw many places where it looked like whatever letter might squeeze in had been written at the end of the line.

I wanted to do an analysis of what letters occur just before/after a line break to see if there is a difference from the rest of the text, but couldn't find a transcribed version.

My completely amateur take is that it's an elaborate piece of art or hoax.

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#123
Although I skimmed the methodology out of curiosity, what really drew my eye was the transcription in the repository of the manuscript. This led me down a rabbit hole leading here [1] about historic efforts to transcript or transliterate the manuscript.

[1] https://www.voynich.nu/transcr.html

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#125

Earlier quoted context omitted.

With how undecipherable the manuscript is, my personal theory is that it's the work of a naive artist and that there's no language behind it. Just someone aping language without knowing the rules about language: https://en.wikipedia.org/wiki/Naïve_art It's not a mental issue, it's just a rare thing that happens. Voynich fits the whole bill for the work of a naive artist.

You're not alone. Many have hypothesized this is just made up gibberish given the unusual distribution of glyphs. Not a recent hoax/scam, but an ancient one. It's not like there weren't a ton of fake documents in the middle age and renaissance, from the donation of Constantine to Preserve John's letter.

The way you describe it is why it’s not readily accepted. It’s misunderstood. You called it a hoax/scam and a fake. It’s not!

Whoever made the document was sincere in making up something that doesn’t exist. They had no intention to mislead. You wouldn’t call a D&D campaign a hoax because it features nonexistent things?

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#126

This looks very interesting - nice work! I have no background in NLP or linguistics, but I do have a question about this: > I stripped a set of recurring suffix-like endings from each word — things like aiin, dy, chy, and similar variants This seems to imply stripping the right-hand edges of words, with the assumption that the text was written left to right? Or did you try both possibilities? Once again, nice work.

It's generally agreed that the Voynich Manuscript was written (and should be read) left to right. For example, margins are aligned on the left and uneven on the right, indicating that the writer started from the left.

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#128

Confirm or deny my suspicion: your post and your comments in this thread are substantially written by ChatGPT?

I'm definitely not the most comfortable writing in public forums, so guilty as charged with throwing my comments through an LLM to make sure my point isn't being misconstrued.

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#129
post #120

Earlier quoted context omitted.

Other than war plans, religious text was pretty much the only thing in the 1500s that would have been encrypted. However war plans would be very unlikely to be disguised as a botany book, for all kinds of reasons. War plans are temporary, not something you'd dedicate that level of artistic effort and permanence to.

The art of war by Sun Tzu is pretty timeless tho

Right, because it's not a war plan. A war plan is about when, where, how, who, etc, for specific attack(s).

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#130
> Traditional analyses often fall into two camps: statistical entropy checks or wild guesswork.

I'd argue that these are just the camps that non-traditional, amateur analysis efforts fall into. I've only briefly skimmed Voynich work, but my impression is that, traditionally, more academic analyses rely on a combination of linguistic and cryptological analysis. This does happen to be informed by some statistical analysis, but goes way beyond that.

For example, as I recall the strongest argument that Voynichese probably isn't just an alternative alphabet for a well-known language relies on comparing Voynichese to the general patterns for how writing systems map symbols to sounds. That permits the development of more specific hypotheses about how it could possibly function, including how likely it is to be an alphabet or abjad, and, hypotheses about which characters could plausibly represent more than one sound, possible digraphs, etc. All of that work casts severe doubt on the likelihood of it representing a language from the area because it just can't plausibly represent a language with the kinds of phonological inventories we see in the language families that existed in that place and time.

There's also been some pretty interesting work on identifying individual scribes based on a confluence of factors including, but not limited to, analysis of the text itself. Some of the inferred scribes exclusively wrote in the A language (oh yeah, Voynichese seems to contain two distinct "languages"), some exclusively wrote in the B language, I think they've even hypothesized that there's one who actually used both languages.

There isn't a lot of popular awareness of this work because it's not terribly sexy to anyone but a linguistics nerd. But I'd guess that any attempt to poke at the Voynich manuscript that isn't informed by it is operating at a severe disadvantage. You want to be standing on the shoulders of the tallest giants, not the ones with the best social media presence.

Post reply on HN