Live data from Hacker News

Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

github.com

141–142 of 142 posts

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#141
post #17

A point of note is that the text embeddings model used here is paraphrase-multilingual-MiniLM-L12-v2 ( https://huggingface.co/sentence-transformers/paraphrase-mult... ), which is about 4 years old. In the NLP world, that's effectively ancient, particularly as the robustness of even small embeddings models due to global LLM improvements has increased dramatically both in information representation and distinctiveness…

Totally fair — I defaulted to paraphrase-multilingual-MiniLM-L12-v2 mostly for speed and wide compatibility, but you’re right that it’s long in the tooth by today’s standards. I’d be really curious to see how something like all-mpnet-base-v2 or even text-embedding-ada-002 would behave, especially if we keep the suffixes in and lean into full contextual embeddings rather than reducing to root forms. Appreciate you cal…

Be careful: they have super short context length AND silently crop if the text is too long. To me there is really no reason to use them.

I recommend ollama to run the artic-embed-v2 model, it also is multimingual and you can use --quantize when loading the modelfile to get it even smaller.

Re: Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

#142
I feel like we are missing an important point...the transformer model that has been used here is trained on known languages. This means it cannot extract meaningful embeddings from a text in a unknown language...are the plots just noise then?
Post reply on HN