Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT‑NL: a sovereign language model for the Netherlands

tno.nl

Post reply on HN
Print view View original

81–90 of 325 posts

Re: GPT‑NL: a sovereign language model for the Netherlands

#81

Post by zozbot234 »

thevinter wrote: post #73
armcat wrote: post #70

I keep seeing these "sovereign" LMs time and time again. In Sweden we had GPT-SW3 ( https://www.ai.se/en/project/gpt-sw3 ) and same story there. Instead of burning money on "sovereign" claims, national research labs should instead focus on building on top of solid baselines (like Qwen/Kimi) and finetuning frontier models with real agentic utility that can be applied across actual use cases and can be widely used by i…

And what happens once the "solid baselines" become unavailable for a reason or the other?

You keep building on the last available version? Fine tuning is a whole lot cheaper, easier and more useful than pretraining a model from scratch. It's a complete no brainer.

Re: GPT‑NL: a sovereign language model for the Netherlands

#83

Post by zozbot234 »

numpad0 wrote: post #54
dwa3592 wrote: post #25

Earlier quoted context omitted.

Understood, but they could fine tune base models on their own cultural context and language. Why reinventing the wheel?

I thought finetuning data can't contradict foundation models, and anything that are inconsistent with the standard LLM American-Chinese split personality would be rejected?

Fine tuning happens on top of pretraining, so of course it can "forget" pretrained defaults when warranted by the new data it's being fine tuned on.

Re: GPT‑NL: a sovereign language model for the Netherlands

#84

Post by joe_mamba »

hdaz0017 wrote: post #58
joe_mamba wrote: post #32

Earlier quoted context omitted.

Please don't move the goalposts. What computer parts does ASML or NXP make? ASML only makes the lithography machines, 85% of which go outside the EU (let that sink in). And then fabs in Taiwan, Korea or the US use those ASML machines to etch US IP for computer chips. EU doesn't make any computer parts domestically. And NXP mostly makes various microcontrollers and small chips, not high margin IP decenter centric part…

The World's Most Important Machine ;) https://www.youtube.com/watch?v=MiUHjLxm3V0

Most important machine ... built on US IP, subject to US export restrictions, used to manufacture high value US IP, in factories outside the EU, so profits of those chips goes to US. A point I have addressed over two times already.

Also ASML even threatened to leave the NL if the Dutch government doesn't do what they want on taxes and labor policies. So having only a single card to play that EU can loose at any time, it's not putting EU tech sovereignty argument in a good light.

The "wahabout ASML" that keeps being spammed by people here, isn't proof of EU compute and AI sovereignty. It's the exception which is why it's the only thing people can mention on EU tech and they DDoS you with it as if that changes anything.

Are people here that petty that they can't stay on topic and argue in good faith and instead need to hijack your argument to go on offtopic whataboutism for a cheap gotcha spamming "whatabout ASML" on unrelated arguments?

Re: GPT‑NL: a sovereign language model for the Netherlands

#85

Post by vrganj »

dwa3592 wrote: post #16

I don't understand countries (especially governments) wanting to have their own models when there are already pretty solid open source (weights) models out there. Countries should want control over _where_ the compute is happening rather than _what code_ is running. What's wrong with a country hosting a Kimi, Qwen or GPT-Oss on their hardware for their government work purpose?

An LLM is an encoding of a culture, a way of viewing the world.

They are not neutral technology, they are a direct representation of the training set that has been chosen and how they are aligned.

In many ways, they are ideology made code.

If we leave building them to the US and China, only their way of seeing things will be digitized.

I don't like the idea of that.

Re: GPT‑NL: a sovereign language model for the Netherlands

#86

Post by jurschreuder »

What are they going to train with 13.5M really? We're a tiny company in Amsterdam in Holland and we've got "only 64x B300 to train on" so we could never make an LLM I thought, since we've got only 4M in compute.

And they're going to train an LLM with all kinds of extra difficulties compared to OpenAI for just 13.5M?

The very first Llama was 16M for one training.

Re: GPT‑NL: a sovereign language model for the Netherlands

#87

Post by wolvoleo »

vrganj wrote: post #85
dwa3592 wrote: post #16

I don't understand countries (especially governments) wanting to have their own models when there are already pretty solid open source (weights) models out there. Countries should want control over _where_ the compute is happening rather than _what code_ is running. What's wrong with a country hosting a Kimi, Qwen or GPT-Oss on their hardware for their government work purpose?

An LLM is an encoding of a culture, a way of viewing the world. They are not neutral technology, they are a direct representation of the training set that has been chosen and how they are aligned. In many ways, they are ideology made code. If we leave building them to the US and China, only their way of seeing things will be digitized. I don't like the idea of that.

Yes and also, US and Chinese models are censored in different ways. US models are way too prudish for personal use in Europe because they're afraid to piss off religious investors. Chinese models are too censored on history and current affairs, eg the tiananmen massacre never happened stuff like that.

Re: GPT‑NL: a sovereign language model for the Netherlands

#88

Post by ozim »

thevinter wrote: post #73
armcat wrote: post #70

I keep seeing these "sovereign" LMs time and time again. In Sweden we had GPT-SW3 ( https://www.ai.se/en/project/gpt-sw3 ) and same story there. Instead of burning money on "sovereign" claims, national research labs should instead focus on building on top of solid baselines (like Qwen/Kimi) and finetuning frontier models with real agentic utility that can be applied across actual use cases and can be widely used by i…

And what happens once the "solid baselines" become unavailable for a reason or the other?

Seems like you don’t understand.

You take current version and build on top of it. You have the weights.

You might not get some n+1 version at some point but the n version you will have will be still most likely much better than whatever you come up with burning good will money of people believing in „sovereignty”.

You are not getting ahead in this game by being „true to your local values” capital expenditure is insane in this game.

Re: GPT‑NL: a sovereign language model for the Netherlands

#89

Post by LaurensBER »

jurschreuder wrote: post #86

What are they going to train with 13.5M really? We're a tiny company in Amsterdam in Holland and we've got "only 64x B300 to train on" so we could never make an LLM I thought, since we've got only 4M in compute. And they're going to train an LLM with all kinds of extra difficulties compared to OpenAI for just 13.5M? The very first Llama was 16M for one training.

This is too little, too late. Europe really need to start focussing.

All these tiny niche models are perhaps fun as an academic exercise or great for the researchers resume but I highly doubt that they'll add any value or will be used for anything serious.

Even if this becomes a somewhat decent model with a fantastic understanding of "gezellig", "kring verjaardag" or "pannenkoeken", how many people will interact with it before the limits of it will drive them back to a frontier model?

Even if the purpose of this is government & other regulated industries, do we really want our government to use a poor model? Either do it right or don't do it at all.

Re: GPT‑NL: a sovereign language model for the Netherlands

#90

Post by dvdkon »

tgv wrote: post #29
Achterlangs wrote: post #18

Earlier quoted context omitted.

It is not about the country but the language. Most llms have poor or no support for Dutch.

Idk which models you refer to, but I tested a bunch recently, and they performed well on Dutch. Only the smallest, such as qwen 3.6 27B, made up words and switched languages.

There would be a bunch of value in having, say, a good 30B-class model that used my local language as well as it does English. There's lots of cases, especially in the government sphere, where local processing is a requirement and frontier-level capabilities aren't required. Making those cheap to run seems like a fine goal.
Post reply on HN