Live data from Hacker News

Translating Akkadian clay tablets with ChatGPT?

janromme.com

21–30 of 121 posts

Re: Translating Akkadian clay tablets with ChatGPT?

#22
post #13

The blog post mentions that you have to double check everything, but they did not check the translation. If you just put the same text through multiple times (in different chat sessions), you get completely different answers back. I think it's just making up the whole thing.

It is. Here's a professional translation for this [0]: ...they made bricks. I razed that small palace in its entirety, took a large area from the fields for an addition, and added (it) to it (the palace). I laid its foundations with limestone, strong stone from the mountains, and raised the terrace. I summoned the kings of Ḫatti and Across the River (Syria-Palestine) Baʾalu, king of Tyre, Manasseh, king of Judah, Qaʾ…

Right?

> 51. ki-ma a-tar-tim-ma ul-tu lib3-bi_a-sza3-mesz_ ab-tuq-ma

Is translated as:

> 51. Like the Apsû, it rose up from the heart of the earth.

But 'Apsu' isn't a reading in any of those words?

Re: Translating Akkadian clay tablets with ChatGPT?

#23
post #13

The blog post mentions that you have to double check everything, but they did not check the translation. If you just put the same text through multiple times (in different chat sessions), you get completely different answers back. I think it's just making up the whole thing.

Not only double-check, but probably triple- or quadruple-. There's too much chance of hallucination to trust one or two instances; I think this could be done, but would require a number of disconnected instances doing the translations, with the resulting documents compared with each other to see a) which ones are more prone to making things up, b) if they hallucinate in similar ways, and c) where they agree, which would allow further study to find out what may have been in their training that allowed similar results to be output.

Re: Translating Akkadian clay tablets with ChatGPT?

#24
post #13

The blog post mentions that you have to double check everything, but they did not check the translation. If you just put the same text through multiple times (in different chat sessions), you get completely different answers back. I think it's just making up the whole thing.

Not only double-check, but probably triple- or quadruple-. There's too much chance of hallucination to trust one or two instances; I think this could be done, but would require a number of disconnected instances doing the translations, with the resulting documents compared with each other to see a) which ones are more prone to making things up, b) if they hallucinate in similar ways, and c) where they agree, which wo…

I would try round-tripping it at least. But I would have to know a little more Akkadian to tell if it's coming up with synonyms or completely different words.

Re: Translating Akkadian clay tablets with ChatGPT?

#25
A google search for their first prompt (ignoring, of course, the prompt intended to get chatgpt into "the right state of mind") "Can you speculate on how the biblical name of king Manasseh of Judah would have been written in Babylonian Cuneiform?" actually turns up the https://armstronginstitute.org/160-esarhaddon-prism-proves-k... link describing in detail the artifact they are interested in as the 5th result (4th if you don't include the post itself). Seems a lot simpler than trying to prompt chatgpt into giving you search terms that you use in some other search anyway.

But, lets say you're still amazed by that initial part of the post. The british museum (where the author had seen the artifact) provides translations of it here: https://www.britishmuseum.org/collection/object/W_1929-1012-...

Let's compare them to chatgpt...

chatgpt:

> (49) They built the lofty temple. (50) As for the temple's foundation, they laid it solidly (51) Like the Apsu, it rose up from the heart of the earth (52) Its shrine was radiantly visible in the midst of the city (53) Its brilliance extended over the lands. (54) They established the kingship in Hatti and made the crown resplendent. (55) They made Menasi king in Assur, established him as ruler in Babylon. (56) Qausgabri ruled in Uruk, Musiri ruled in Ma'ab (57) Issenen, king of Hazitu, reigned over the people of Esqaluna.

true translation:

> (49) they made (unburnt) bricks. That little palace (50) throughout I destroyed and much land (51) as an addition from the fields I cut off and (52) added thereto : with limestone, the solid stone from the mountains, (53) I laid its foundation and filled a terrace : (54) I assembled the kings of the Hittites and across the river. (55) Ba'lu, king of Tyre, Menasi, king of Judah, (56) Kausgabri, king of Edom, Musuri, king of Moab, (57) Sil-bel, king of Gaza, Metinti, king of Ascalon,

And, in fact the thing that the author seemed to care about (whether it was about Menasi of Judah), chatgpt, unsurprisingly, got wrong. Though again, just using google would've answered that immediately with the detailed description of the prism that I noted it returned as a top result above.

Re: Translating Akkadian clay tablets with ChatGPT?

#26
I tried with some other undeciphered texts and I just got random translations that changed drastically with every attempt. Once I became skeptical, I tried to see if it could do a simple replacement cipher on English text, and it could not. I don’t think ChatGPT is a good fit for this kind of problem.

Re: Translating Akkadian clay tablets with ChatGPT?

#27

A google search for their first prompt (ignoring, of course, the prompt intended to get chatgpt into "the right state of mind") "Can you speculate on how the biblical name of king Manasseh of Judah would have been written in Babylonian Cuneiform?" actually turns up the https://armstronginstitute.org/160-esarhaddon-prism-proves-k... link describing in detail the artifact they are interested in as the 5th result (4th i…

Here is GPT-4's attempt; 49-51 appear to be far off while the rest at least have some relation to the British Museum translation. I imagine there is some room for ambiguity.

    (49) They brought the (il-bi-nu) sig4 e2-gal-tur-ra to his (szu-a-tu2) place.

    (50) To the market, they went (aq-qur-ma) quickly (qaq-qa-ru), very much (ma-a'-du).

    (51) Like (ki-ma) the (a-tar-tim-ma) from the heart (lib3-bi) of the (a-sza3-mesz) they were taken away (ab-tuq-ma).

    (52) His (ugu-szu2) foundation was laid (usz-rad-di) in the (na4) stone (pi-i-li) of the (na4 kur) powerful mountain (dan-ni).

    (53) They (usz-szi-szu2) built (ad-di-ma) it complete (tam-la-a) and perfect (u2-mal-li).

    (54) And thus (ad-ke-e-ma) the kings (lugal-mesz) of Hatti and the other side (e-ber) of the river (i7).

    (55) King (disz lugal) Balu of (iri) Surri, King (disz lugal) Menasi of (iri) Yaudi.

    (56) King (disz lugal) Qaus-gabri of (iri) Udume, King (disz lugal) Musur of (iri) Maab.

    (57) King (disz s,il-en lugal) of (iri) Haziti, Metinti (disz me-ti-in-ti) the prince (man) of (iri) Isqaluna.

Re: Translating Akkadian clay tablets with ChatGPT?

#30

There is indeed plenty of translated Akkadian in open-access papers that could have become part of ChatGPT's training set, so it's completely plausible that GPT might have learnt a thing or two about the Babylonian language. However, the questionable 'translation' number 55 casts doubt in my mind as to its accuracy: > They made Menasî king in Assur, established him as ruler in Babylon. Just from the history, surely t…

I can confirm that GPT 4 fails pretty badly on Middle Egyptian texts that are outside of its training set, using both MdC encoding and transliteration.

I get translations which have at most a word or two in common with the original text. Almost pure hallucination.

Post reply on HN