The Amália model is not yet publicly available. Until it's ready, one can fool around with Anália at https://analia.pt
AMÁLIA and the future of European Portuguese LLMs
51–60 of 90 posts
Re: AMÁLIA and the future of European Portuguese LLMs
#52This model is a waste of Public Funds. There is no public website to use it, be it free or paid, the dataset is not public, the code is not public (The github URL in the article returns 404 ), the claimed model intelligence is so low that is pretty much useless at 32K context and massively inferior to GPT‑4o. As per tradition in Portugal, some people managed to get 5.5 Million to produce nothing and no one is asking…
I'm not arguing with the rest of your points, but... > Just fine tune the open source Kimi 2.6 with an open source Portuguese dataset I think that tokenizers of all popular models are heavily biased towards English or English and Mandarin. And I don't think that it is possibple to replace the tokenizer without full retraining.
You are right about most tokenizers being heavily biased towards English, but the situation is not so bad for Portuguese. Here are some results on the Goldfish corpus [1] with a few different tokenizers. This measures #characters in corpus / #subwords in tokenized corpus.
```
Llama3
english, 0.216
portuguese, 0.285
italian, 0.287
greek, 0.592
```
```
Gemma4
english, 0.219
portuguese, 0.246
italian, 0.249
greek, 0.537
```
```
Kimi2.6
english, 0.214
portuguese, 0.310
italian, 0.308
greek, 0.716
```
Portuguese is worse than English certainly, but it is on par with Italian (which I think has more overlap with English) and much better than Greek (since it doesn't use the Latin script and is definitely not prioritized in the tokenizer construction).
On your second point, tokenizer transfer allows for extending/modifying a tokenizer without retraining the model from scratch. The simplest version of this is tokenizer extension + continual pretraining, where you just add a bunch more tokens to the vocab for the language/domain that you want to improve and train a little more. It's been done for Japanese [2] and Indic languages, but afaik not Portuguese.
So I think that continual pretraining for a large base model would have probably been fine for this case with huge cost savings. But it is good to have the ability to train your own base models, so I don't think this is such a bad idea.
-----------------------
[1]: https://huggingface.co/datasets/goldfish-models/fish-food
Re: AMÁLIA and the future of European Portuguese LLMs
#53Re: AMÁLIA and the future of European Portuguese LLMs
#54I’ve noticed that ChatGPT is noticeably dumber in languages other than English. It even will confidently repeat common but wrong superstitions from the target language as if they were fact.
Re: AMÁLIA and the future of European Portuguese LLMs
#55Earlier quoted context omitted.
We have the same thing happening, on multiple levels, here too. First some Spanish parents are afraid the children aren't listening and watching enough Spanish media. Then additionally, some Catalan parents are afraid the children don't get to use Catalan in school so they don't become proficient enough to use it in society.
The Catalan situation is completely different and unrelated, being a completely different language and not endangered (with or without scary quotes, as you prefer) by an ex-colony that became independent. Actually many Catalans would like to be such ex-colony.
> The Catalan situation is completely different and unrelated
I'm not saying it's the same, but there is definitively similarities in that parents are worrying about what language their children use. And yeah, unrelated, wasn't trying to claim it's the same or better/worse or anything, just another similar situation other (curious) people might want to learn more about, regardless of what you think Catalan wants or not.
Re: AMÁLIA and the future of European Portuguese LLMs
#56Wouldnt it be easier to fine tune a model to convert the Brazilian Portuguese corpus into European Portuguese and then use that corpus?
That idea is different than what most are talking here in other comments.
The grammar and vocabularies don't match, but I think the worst are the expressions. Both sides have *a lot* of expressions that vary per context and location.
Re: AMÁLIA and the future of European Portuguese LLMs
#57Earlier quoted context omitted.
Europe always has a thing for their languages. They think many languages make them stronger while spending billions in system loss due to communication barriers. It is obvious they will try to do the same with LLMs and call it the next best thing since bread and butter. I went to JCON EUROPE this year. The amount of "Europe this" "Europe that" "sovereign this, sovereign that" is mind boggling and just a waste of time…
whats your suggestion? we just eradicate all of our culture and languages and go full on english ? whats wrong with exploring ways to keep national languages alive in the LLM area
>and go full on english ?
Nobody is saying you have to swap your culture for English. You can have English as the mandatory language for tech and business across the EU, while still keeping your language and culture for your education, leisure, festivities, art, media, etc. This way everyone is happy. But countries like France would rather detonate its entire nuclear arsenal rather than accepting official use of English on its own soil.
As long as resources are spent across the EU to account for every language and bureaucracy, we'll keep falling behind internationally, and the only winners will be the bureaucrats, notaries, lawyers, consultants, translators, etc. which would be fine if this were preserving culture like you said in the beginning, but it isn't, it's just preserving friction, segmentation and bureaucracy.
We need another Concord moment. What's wild is that Concord was made via international cooperation, before the EU was even a thing. So whatever the EU is doing to improve things, it's either not good, not enough, or not working. I hope this improves but knowing how petty some EU states are about things being done their way, I doubt it.
Re: AMÁLIA and the future of European Portuguese LLMs
#58Earlier quoted context omitted.
[flagged]
everyone on this project probably learned a lot doing it, dont you think!
I'd also want to get paid to work on stuff not meant to bring any financial returns to my employer, just to learn and pad my resume. Sounds like a sweet gig. Where do I sign up?
Re: AMÁLIA and the future of European Portuguese LLMs
#59It is definitely an interesting problem, because Portugal is a small enough country that the actual total corpus of available texts in (non-Brazilian) Portuguese is potentially problematic.
I would have to imagine this might not actually be as bad as it seems, at the very least there should be a giant corpus of translated EU texts.
Re: AMÁLIA and the future of European Portuguese LLMs
#60This model is a waste of Public Funds. There is no public website to use it, be it free or paid, the dataset is not public, the code is not public (The github URL in the article returns 404 ), the claimed model intelligence is so low that is pretty much useless at 32K context and massively inferior to GPT‑4o. As per tradition in Portugal, some people managed to get 5.5 Million to produce nothing and no one is asking…
As a pt-BR speaker from across the pond: https://soberania.ai/ Similar waste.
Why?