First experience with llama3 (via ollama)
1) It keeps unnecessarily hash-tagging sentences, but listens better than mistral at removing them if I specify
2) It failed my pop culture "Arthas test":
Here's llama3:
>>> who's arthas
A great question!
Arathos, also known as Arathas or Artaxerxes (Persian: اردشیر), was a king of the
Achaemenid Empire, who ruled from 465 to 424 BCE. He was the
son of Xerxes I and Queen Amestris.
Arthas is best known for his military campaigns against
Greece during the Persian Wars.
In particular, he led a successful campaign against
Athens and Eretria in 459-449 BCE, which resulted in the
destruction of these cities
and the imposition of heavy reparations.
However, his later years were marked by a series of
defeats at the hands of the Greeks,
including the Battle of Plataea (479 BCE) and the Battle
of Mycale (479 BCE). This ultimately led to the decline of
the Achaemenid Empire and
its eventual defeat by Alexander the Great in 334 BCE.
So, there you have it! Arthas, a significant figure in
ancient Persian history.
It thought I said "Arathos" yet says "Arthas" at the end lol. Here's mistral:
>>> who's arthas
Arthas Menethil is a character from the real-time strategy
game "Warcraft III: Reign of Chaos" and its expansion "The
Frozen Throne," developed
by Blizzard Entertainment. He is also known as the Lich
King.
Arthas was once the prince of Stratholme, in the kingdom
of Lordaeron, and was a promising young knight who was
deeply devoted to his kingdom
and people. However, after a series of tragic events,
including the loss of his mother and the corruption of his
mentor Uther the Lightbringer,
Arthas became consumed by madness and betrayal. He turned
against his own people and eventually became the Lich
King, the leader of the Scourge,
an undead horde that plagued the land of Azeroth.
mistral seems more on-point and has more consistent outputs. but llama3 is better than mistral at following output instructions like "output the same list but dont use numbers" (mistral insists on either bullets or numbers, have to say "in naturally flowing sentences" to get it to work). This might mean llama3 is better to build with (esp. RAG), even if it's less knowledgeable.
It's not gpt-4 level that's for sure. I think mistral is still closest to gpt-4 that I've tried.