Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

github.com

Post reply on HN
Print view View original

101–110 of 142 posts

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#101

Post by loxias »

patcon wrote: post #41

I see that you're looking for clusters within PCA projections -- You should look for deeper structure with hot new dimensional reduction algorithms, like PaCMAP or LocalMAP! I've been working on a project related to a sensemaking tool called Pol.is [1], but reprojecting its wiki survey data with these new algorithms instead of PCA, and it's amazing what new insight it uncovers with these new algorithms! https://patco…

Try TDA ("mapper", or really, anything based on kernel density computed connectivity), it's a whole new world.

This ain't your parents' "factor analysis".

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#103

Post by riffraff »

philistine wrote: post #94
us-merul wrote: post #13

I’ve found this to be one of the most interesting hypotheses: http://voynichproject.org/ The author made an assumption that Voynichese is a Germanic language, and it looks like he was able to make some progress with it. I’ve also come across accounts that it might be an Uralic or Finno-Ugric language. I think your approach is great, and I wonder if tweaking it for specific language families could go even further.

With how undecipherable the manuscript is, my personal theory is that it's the work of a naive artist and that there's no language behind it. Just someone aping language without knowing the rules about language: https://en.wikipedia.org/wiki/Naïve_art It's not a mental issue, it's just a rare thing that happens. Voynich fits the whole bill for the work of a naive artist.

You're not alone. Many have hypothesized this is just made up gibberish given the unusual distribution of glyphs.

Not a recent hoax/scam, but an ancient one.

It's not like there weren't a ton of fake documents in the middle age and renaissance, from the donation of Constantine to Preserve John's letter.

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#106

Post by cronopios »

philistine wrote: post #94
us-merul wrote: post #13

I’ve found this to be one of the most interesting hypotheses: http://voynichproject.org/ The author made an assumption that Voynichese is a Germanic language, and it looks like he was able to make some progress with it. I’ve also come across accounts that it might be an Uralic or Finno-Ugric language. I think your approach is great, and I wonder if tweaking it for specific language families could go even further.

With how undecipherable the manuscript is, my personal theory is that it's the work of a naive artist and that there's no language behind it. Just someone aping language without knowing the rules about language: https://en.wikipedia.org/wiki/Naïve_art It's not a mental issue, it's just a rare thing that happens. Voynich fits the whole bill for the work of a naive artist.

And that naïve artist somehow managed to create a work that follows Zipf's law, 4 centuries before it was discovered?

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#107

Post by mellow_observer »

marcodiego wrote: post #58

How expensive is a "brute force" approach to decode it? I mean, how about mapping each unknown word by a known word in a known language and improve this mapping until a 'high score' is reached?

I don't think that's likely possible. How would you determine the score? Where would you get your corpus of medieval words? How would you deal with the insane computational complexity?

Pecularities in Voynich also suggest that one to one word mappings are very unlikely to result in well described languages. For instance there's cases of repeated word sequences you don't really see in regular text. There's a lack of extremely common words that you would expect would be neccessary for a word based structured grammar, there's signs that there's at least two 'languages', character distributions within words don't match any known language, etc.

If there still is a real unencoded language in here, it's likely to be entirely different from any known language.

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#108

Post by mellow_observer »

krick wrote: post #62
Reubend wrote: post #39

Earlier quoted context omitted.

Most agree that this is not a real solution. Many of the pages translate to nonsense using that scheme, and some of the figures included in the paper don't actually come from the Voynich manuscript in the first place. For more info, see https://www.voynich.ninja/thread-3940-post-53738.html#pid537...

I'm not really following the research, so it's rather a lazy question (assuming you do): does any of it follow the path Derek Vogt was suggesting in his (kinda famous) videos (that he deleted for some reason)? I remember when I was watching them, it felt so convincing I thought "Alright, it looks like there must be a short leap to the actual solution now." Yet 10 years later I still hear that the consensus is that th…

If you spend some time working on Voynich yourself you'll find that it's actually fairly doable to come up with some translation where you can find a few words that seem to agree with each other. And when you allow yourself some permissions like unorthodox spellings or characters that can mean different things in different places, then it's not so hard to even be able to 'translate' a few seemingly reasonable sentences. This gives a lot of hope to the translator and any who follow them

So far none of these ideas have been shown to be applicable to the full text though. What you would expect with a real translation is that the further you get with your translation, the easier it becomes to translate more. But with the attempts so far is that we keep seeing that it becomes more and more difficult to pretend that other pages are just as translatable using the same scheme you came up initially. It eventually just dies a quiet death

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#109

Post by raverbashing »

munchler wrote: post #73
marcodiego wrote: post #58

How expensive is a "brute force" approach to decode it? I mean, how about mapping each unknown word by a known word in a known language and improve this mapping until a 'high score' is reached?

This seems to assume that a 1:1 mapping between words exists, but I don't think that's true for languages in general. Compound words, for example, won't map cleanly that way. Not to mention deeper semantic differences between languages due to differences in culture.

Correct

Mapping words 1:1 is not going to lead you anywhere (especially for a text that has stood undecoded for so long time)

It kiiiinda works for very close languages (think DutchGerman or FrenchSpanish) and even then.

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#110

Post by psychoslave »

PaulDavisThe1st wrote: post #90
bdbenton5255 wrote: post #84

Another great natural mystery that machine learning could tackle is earthquake prediction. Sure you could find some patterns modeling historical data.

What data would you feed it and why?

Most likely seismograph logs, I guess? Not sure gas emissions are something we can expect to provide a more ahead of time alert. That is my guess is that by the time some out of charts in gas emissions are showing, it's too late to make a whole city population move? I'm nothing like a volcanologist though, this is just very wild guess.

On the other hand, it's a bit wild to build a whole city next to volcanos that are definitely going to wake up in less than a few centuries, to begin with.

Post reply on HN