AMÁLIA and the future of European Portuguese LLMs
duarteocarmo.com
AMÁLIA and the future of European Portuguese LLMs
1–10 of 90 posts
Re: AMÁLIA and the future of European Portuguese LLMs
#2What a waste of time and money.
Trying to force a LLM into a specific language makes you missed out on most of the world knowledge.
Re: AMÁLIA and the future of European Portuguese LLMs
#3What a waste of time and money. Trying to force a LLM into a specific language makes you missed out on most of the world knowledge.
> makes you missed out on most of the world knowledge
and, who knows what will happen to grammar ?
Re: AMÁLIA and the future of European Portuguese LLMs
#4What a waste of time and money. Trying to force a LLM into a specific language makes you missed out on most of the world knowledge.
What LLM isn't forced into a specific language? That'd be a weird language model no one could understand, you need to chose at least one language, ideally the same as the creators speak.
Besides, there is knowledge that is locked behind languages, there are things known in Portuguese that aren't known in other languages, and the same for other languages too. More accessibility to those ideas wouldn't hurt.
Re: AMÁLIA and the future of European Portuguese LLMs
#5What a waste of time and money. Trying to force a LLM into a specific language makes you missed out on most of the world knowledge.
[dead]
Re: AMÁLIA and the future of European Portuguese LLMs
#6It is definitely an interesting problem, because Portugal is a small enough country that the actual total corpus of available texts in (non-Brazilian) Portuguese is potentially problematic.
Re: AMÁLIA and the future of European Portuguese LLMs
#7I'm not sure the direction should be to finetune a small local model for each country or language. These models are already not particularly great at information retrieval, so I doubt anyone would use them for questions like the author suggests (ie who was the president between X and Y). Similarly, they are a little too lightweight to be used for translations too.
If the budget is indeed so modest (5.5 million euros!), I would focus completely on preparing datasets and making sure all open cultural artifacts that we can find are well documented in them. That way every model, private or open, that gets trained in the future could better represent the culture and language of your country.
Re: AMÁLIA and the future of European Portuguese LLMs
#8What a waste of time and money. Trying to force a LLM into a specific language makes you missed out on most of the world knowledge.
What LLM isn't forced into a specific language? That'd be a weird language model no one could understand, you need to chose at least one language, ideally the same as the creators speak. Besides, there is knowledge that is locked behind languages, there are things known in Portuguese that aren't known in other languages, and the same for other languages too. More accessibility to those ideas wouldn't hurt.
To my knowledge, all major LLMs are multilingual. This article could really have used an evaluation of existing models' European Portuguese capabilities.
Re: AMÁLIA and the future of European Portuguese LLMs
#9What a waste of time and money. Trying to force a LLM into a specific language makes you missed out on most of the world knowledge.
What LLM isn't forced into a specific language? That'd be a weird language model no one could understand, you need to chose at least one language, ideally the same as the creators speak. Besides, there is knowledge that is locked behind languages, there are things known in Portuguese that aren't known in other languages, and the same for other languages too. More accessibility to those ideas wouldn't hurt.
E.g. gemma3:4b can fake simple conversations in several european languages, including portuguese, swedish and finnish.
It's just a database. If you push text in one language into it, it'll likely crap out stuff in that same language, unless the system prompt that also goes in with your query causes it not to.
Re: AMÁLIA and the future of European Portuguese LLMs
#10It is definitely an interesting problem, because Portugal is a small enough country that the actual total corpus of available texts in (non-Brazilian) Portuguese is potentially problematic.
I don't think so, Portugal the country might be small, with a small population, but there is ~250 million "Lusophones" (native Portuguese speakers), making it the fifth-most spoken native language in the world, I'd hardly call that small :) And before everyone screams; yes, European Portuguese is different from Brazilian Portuguese, but they're still both Portuguese and understand each other, so it's not like the text from one cannot be used to train a model for the other, or vice-versa.
All in all, I don't think that's a major issue here.