Live data from Hacker News

Translating Akkadian clay tablets with ChatGPT?

janromme.com

41–50 of 121 posts

Re: Translating Akkadian clay tablets with ChatGPT?

#43
post #30

Earlier quoted context omitted.

I can confirm that GPT 4 fails pretty badly on Middle Egyptian texts that are outside of its training set, using both MdC encoding and transliteration. I get translations which have at most a word or two in common with the original text. Almost pure hallucination.

From 'do robots dream' to 'how do we make them stop'

“Do Androids Dream of Electric Sheep?” is a must-read for fans of Blade Runner and the cyberpunk genre.

Also, it’s interesting that in 2023, we can reasonably definitively answer that question for any definition of “dream”.

Re: Translating Akkadian clay tablets with ChatGPT?

#45
post #13

The blog post mentions that you have to double check everything, but they did not check the translation. If you just put the same text through multiple times (in different chat sessions), you get completely different answers back. I think it's just making up the whole thing.

Not only double-check, but probably triple- or quadruple-. There's too much chance of hallucination to trust one or two instances; I think this could be done, but would require a number of disconnected instances doing the translations, with the resulting documents compared with each other to see a) which ones are more prone to making things up, b) if they hallucinate in similar ways, and c) where they agree, which wo…

Last week, a story in praise of ChatGPT ended with a haiku that has the wrong syllable count. Why check when it sounds good?

Re: Translating Akkadian clay tablets with ChatGPT?

#46

I tried with some other undeciphered texts and I just got random translations that changed drastically with every attempt. Once I became skeptical, I tried to see if it could do a simple replacement cipher on English text, and it could not. I don’t think ChatGPT is a good fit for this kind of problem.

Yes, I’ve also seen it struggle with ROT13. I’ve read that this is because the tokenizer breaks up words in a way that makes mapping to ROT13 hard. I don’t expect it to have much luck with a language with a small corpus.

Re: Translating Akkadian clay tablets with ChatGPT?

#47
As others have noticed, the translation is hilariously wrong. Even if your only method for gauging the correctness of the so-called "translation" were googling, you will easily find Leichty's [0] or Borger's [1] editions with translations. So to be very clear, saying "a translation is lacking" is thoroughly incorrect. It betrays a lack of familiarity with the subject and its reference works.

Moving back to when I first saw this post, I immediately gravitated towards checking the "translation" and so neglected to read anything else written. Now that I've lovingly wasted my time transcribing and translating from the supplied transliteration before checking Leichty [0], I can move on to wasting my time with the other issues.

The initial complaint about a "Karen" is obnoxious and unhelpful. You can always _write down_ the id number or title of an object in a museum if you have a specific interest. If the tablet one is interested in is Biblically relevant, then it is almost certainly referenced in literature (which you can take a shortcut to find said literature if you have that number you wrote down). All you have to do is some reading and/or contacting of relevant scholars. Tablets with references to Biblical events and persons are often exceptionally well documented due to Biblical scholarship's smothering embrace of Assyriology. If you think there might be multiple copies of this tablet, then you could ask Hobby Lobby's if they have a copy in their collection they obtained by funding ISIS. I'm sure they would love to talk about it.

Moving away from museum etiquette, there are some other issues. Searching through the CDLI requires some effort. It is a tool, not a search engine with results ranked for relevance and ad revenue. Knowing the language and the processes of transliteration, transcription, and translation will help. Spellings can vary within dialects and search will not conjugate or magically find alternative cases for you. The transliterations on the CDLI are less than perfect so don't think you can just train some LLM to do this for you (There is a lot of bad data. Feel free to make some more bad data by doing some transliterating yourself with the provided low dpi images. Transliterating without both the tablet and a movable light source in front of you can border on impossible. You might make a transliteration, but it will be bad if you lack the will spend weeks staring at a photograph.). I found at least 25 texts with decent mentions of some Manassehs on the CDLI. Keep searching and remember that ChatGPT doesn't really understand cases and conjugations in the Akkadian dialects :) (especially for loan words lol).

Oh and "Lugal" is Sumerian. Saying "the Akkadian word Lugal" is simply wrong. This is basic stuff. In Neo-Assyrian cuneiform (I hesitate to say all cuneiform writing systems but it probably is all (that includes non semitic languages)), LUGAL is a logogram for the "king". In this text, since it is written in the Neo-Assryian dialect of Akkadian, that would be šarru in whatever case (nominative, genitive, accusative) is appropriate.

Oh and finally, it is very clear that it is a King Manasseh _of Judah_. Not Assur. Manasseh is not an Assyrian name. If you actually look at the text and just read the goddamn Akkadian, this will be obvious. It clearly says "m^me-na-si-i LUGAL URU.ia-ú-di" not "m^me-na-si-i LUGAL aš-šur^ki". It's not hard to see how wrong the translation is if you just, you know, actually translate the Akkadian. Looking more closely, ChatGPT seems to be unable to associate the given names of kings with place names. Can it not apply its "understanding" of the genitive? Is it just stupid? In fact, many of the place names in the ChatGPT translation are not mentioned anywhere in the column, let alone in this specific section.

From these initial observations, GPT4/ChatGPT seems incapable of understanding rote definitions of Akkadian vocabulary, Akkadian grammatical constructs, determinatives, logograms, et al. The reading of col. V line 55 is incredibly simple. If the now GPT4 powered ChatGPT cannot translate _that_, I suspect it will be a long while before LLMs can even begin producing marginally correct translations of texts written in cuneiform. I think that doing this correctly will require an absolutely massive, tailor-made model. Since the age of giant models is now over [2] (lol) the funding just probably won't be there.

From fighting with this now essay length comment, it is my newly formed opinion that proofreading hallucinated translations is more difficult that just doing the translation yourself. Any work with untranslated cuneiform requires accuracy and a thorough comprehension of the text itself. Errors, even if small can completely ruin whatever point/thesis you were trying to make.

You have to remember that when dealing with cuneiform texts you aren't dealing with one language. You are dealing with whatever the cuneiform was written in, probably some sumerian phrases, maybe the language of the scribe, the languages of your reference works and dictionaries (English, French, German, etc), the idiosyncrasies of the scribe from 4000++ years ago, the idiosyncrasies of the transliterator, missing chunks of tablet, and a writing system that has no modern equivalent. And even just knowing all those things isn't enough, you personally have to be able to fill in all the gaps left by the human actions along the chain of translation. LLMs seem to struggle with the unsaid, barely hinted human influence that is a core part of translating texts written in cuneiform. Will an LLM ever be able to imagine itself as a scribe 3000 years ago, pushing a reed stylus into damp clay to make the signs of an already dead language? Will it understand why that is necessary for translation? Can it comprehend the nuance stacked on nuance that lead to it ingesting whatever text it is going to fail to translate? If it does not, then it is a poor translator. And when working with languages written in cuneiform, a poor translation is worse than useless.

[0] https://www.eisenbrauns.org/books/titles/978-1-57506-209-9.h... [1] AfO Bh. 9 [2] https://www.wired.com/story/openai-ceo-sam-altman-the-age-of...

Re: Translating Akkadian clay tablets with ChatGPT?

#48
post #3

[flagged]

Sam Harris did an episode on existential threat and nuclear war where he talks about certain knowledge potentially being deadly for humanity. The knowledge of how to build an atom bomb is an example. Knowledge of how to build an atom bomb with household items would almost certainly end humanity.

Super-intelligent AI with poor alignment could totally give us this knowledge. This information could already exist somewhere, or we might be only a few thoughts away from discovering it. At the very least, we probably already have some prerequisites to discover humanity-ending tech and we will probably continue to close the gap as we don’t know what the dangerous technology is.

Some argue this is how technological species would approach the great filter.

Re: Translating Akkadian clay tablets with ChatGPT?

#49

Could have gone without the Karen digression. It took up a third of the post, didn’t really have a point other than that he didn’t get a picture of the tablet, and wasn’t even really in the original spirit of the phrase anyway.

Yea - along with interpretation of an icon being the issue.

Re: Translating Akkadian clay tablets with ChatGPT?

#50
Some people have been "strengthening" Chat GPT by re-running the same query several times, then combining the various outputs. I asked GPT 4 to translate one of the linked snippets twice, pasted in the original plus the variants, and told it to merge all three into an improved translation. I also gave it metadata and context from the page. Without that, it complained that an isolated snippet of Sumerian or Akkadian is too difficult to translate:

    Esarhaddon, the great king, the powerful king,
    King of the Universe, King of Assyria, Governor of Babylon,
    King of Sumer and Akkad, King of the Four Corners of the Universe,
    Wisely nurtured by the great gods,
    From his earliest days, Assur, Shamash, Enlil, and Ishtar,
    The gods of Nineveh and the gods of Assur,
    For the kingship of Assyria, have determined his fate.
    The circumstances of my origins, my noble lineage, they caused me to understand.
    In the service of Assur, Sin, Shamash, Enlil, and Ishtar,
    The gods of Nineveh and the gods of Assur, they created me,
    Amidst my numerous heroic deeds, my valor has no equal,
    They said, "This is the one, let him rule!"
    Shamash and Adad revealed this to me through a favorable omen,
    They declared to me: 'You are the one who will shepherd our people',
    Their weighty destiny, they assigned, and the land of Assyria became great,
    My numerous offspring, the product of my loins, they caused to grow and multiply.
    Before Assur, Sin, Shamash, Ishtar, Marduk, the gods of Assyria,
    The gods, my helpers, who reside in heaven and earth, they have enlarged my reign.
    Their weighty command, they caused to be heard,
    In the days of abundance, from dawn to dusk, in their temples, they have decreed my fate.
This is from the first 20 lines of: https://cdli.mpiwg-berlin.mpg.de/artifacts/422293/reader/201...

I found a professional translation here: http://oracc.museum.upenn.edu/rinap/Q003230

    The palace of Esarhaddon, great king, mighty king, 
    king of the world, king of Assyria, governor of Babylon, 
    king of Sumer and Akkad, king of the four quarters, true shepherd, 
    favorite of the great gods,
    whom from his childhood the gods Aššur, Šamaš, Bēl, and Nabû, Ištar of Nineveh, 
    (and) Ištar of Arbela named for the kingship of Assyria
    I am my older brothers’ youngest brother (and) by the command of the 
    gods Aššur, Sîn, Šamaš, Bēl, and Nabû, Ištar of Nineveh, (and) Ištar of Arbela, (my) father, who engendered me, 
    elevated me firmly in the assembly of my brothers, 
    saying: ‘This is the son who will succeed me.’ 
    He questioned the gods Šamaš and Adad by divination, and they answered him with a firm ‘yes,’ 
    saying: ‘He is your replacement.’ He heeded their important word(s) and gathered together the people 
    of Assyria, young (and) old, (and) my brothers, the seed of the house of my father.     
    Before the gods Aššur, Sîn, Šamaš, Nabû, (and) Marduk, the gods of Assyria, 
    the gods who live in heaven and netherworld, he made them swear their solemn oath(s) concerning the safe-guarding of my succession.
    In a favorable month, on a propitious day, in accordance with their sublime command, 
    I joyfully entered the House of Succession, an awe-inspiring place within which the appointing to kingship (takes place).
The gist of it seems to match the GPT 4 translation pretty well, and if anything, I prefer the GPT 4 version, as it is more readable!
Post reply on HN