Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT‑NL: a sovereign language model for the Netherlands

tno.nl

Post reply on HN
Print view View original

181–190 of 325 posts

Re: GPT‑NL: a sovereign language model for the Netherlands

#181

Post by hrmon »

bradhe wrote: post #167
appplication wrote: post #153

Earlier quoted context omitted.

Disagree, it’s in the country’s best interest to facilitate internal expertise on the full stack and own their “supply chain” so to speak and fight brain drain. The outcome isn’t just the model, it’s the expertise. Otherwise all their smartest folks will depart for countries where LLM development is strongest.

Got it, so every country should focus on having a mediocre-at-best AI strategy by refusing to work together? Surely this will create a better future instead of pooling resources. This vaguely-nationalist world view around tech that’s emerging in Europe is dangerous, man. On the brain drain problem in particular, one way to ensure talent sticks around is to create a good environment for people to do their best work. I…

> This vaguely-nationalist world view around tech that’s emerging in Europe is dangerous, man.

Guess which country blocked access to a SOTA model based on national security bullshit.

Re: GPT‑NL: a sovereign language model for the Netherlands

#182

Post by bigfudge »

mholm wrote: post #150
sublimefire wrote: post #69

It is crazy that anything Europe gets so much hate. IMO it is important to build models within the boundaries of smaller nations, using their own language. Research has to continue even if it is outside of US and China.

If a teenager on your street said he was going to spend $1,000 to customize his Honda Civic for his needs, you'd believe him. If he says he's going to build a brand new car, better than a Honda civic, for $10,000, you'd laugh and say good luck.

But if a teenager says he’s going to spend 50k going to university to study engineering you might support then.

I agree there is likely some hubris in this sort of announcement, but investing in European expertise and industrial base in this area is important.

Re: GPT‑NL: a sovereign language model for the Netherlands

#183

Post by bigfudge »

throw310822 wrote: post #133
dvdkon wrote: post #90

Earlier quoted context omitted.

There would be a bunch of value in having, say, a good 30B-class model that used my local language as well as it does English. There's lots of cases, especially in the government sphere, where local processing is a requirement and frontier-level capabilities aren't required. Making those cheap to run seems like a fine goal.

Can you provide some examples of these use cases?

Support bots and question answering with access to sensitive pii?

Re: GPT‑NL: a sovereign language model for the Netherlands

#184

Post by vintermann »

mschuster91 wrote: post #95
armcat wrote: post #70

I keep seeing these "sovereign" LMs time and time again. In Sweden we had GPT-SW3 ( https://www.ai.se/en/project/gpt-sw3 ) and same story there. Instead of burning money on "sovereign" claims, national research labs should instead focus on building on top of solid baselines (like Qwen/Kimi) and finetuning frontier models with real agentic utility that can be applied across actual use cases and can be widely used by i…

Kimi and Qwen come out of China, which means that their training material may be biased e.g. relating to Taiwan [1]. In addition, there is no way to determine what input went into the training, if it was properly licensed, if it was legal (e.g. not contaminated by CSAM), or how the human component of RLHF was sourced - in US models, for example, stories about exploitation like [2] have been floating for years. Assumi…

The Chinese models are almost certainly taught to comply with "Chinese values" in the RLHF step, not from filtering the training data. There may be a few things which are too radioactive to be allowed even in the training material - but that's more likely to be things like child abuse images for a visual model, things non-Chinese values also have an issue with.

I'm pretty sure no county taking a stab at making their own model for sovereignty purposes will let "proper licensing" stand in their way.

Re: GPT‑NL: a sovereign language model for the Netherlands

#185

Post by bigfudge »

joe_mamba wrote: post #32
davedx wrote: post #30

Earlier quoted context omitted.

Have you heard of ASML? NXP? Ignorant comment

Please don't move the goalposts. What computer parts does ASML or NXP make? ASML only makes the lithography machines, 85% of which go outside the EU (let that sink in). And then fabs in Taiwan, Korea or the US use those ASML machines to etch US IP for computer chips. EU doesn't make any computer parts domestically. And NXP mostly makes various microcontrollers and small chips, not high margin IP decenter centric part…

Europe is currently hosed because we made the mistake of trying to develop economies complementary to the US and china.

That was a big strategic mistake. In the US case it was borne of the mistaken belief that we shared values and were partners.

But don’t mistake the situation for lack of innovation of capability. Europe is currently adapting, but I think the success of Ukraine is one reason to be optimistic that current adversity might actually leave us better off in the long run.

Corrupt countries with broken legal systems tend not to fare that well in the longer run.

Re: GPT‑NL: a sovereign language model for the Netherlands

#186

Post by bradhe »

hrmon wrote: post #181
bradhe wrote: post #167

Earlier quoted context omitted.

Got it, so every country should focus on having a mediocre-at-best AI strategy by refusing to work together? Surely this will create a better future instead of pooling resources. This vaguely-nationalist world view around tech that’s emerging in Europe is dangerous, man. On the brain drain problem in particular, one way to ensure talent sticks around is to create a good environment for people to do their best work. I…

> This vaguely-nationalist world view around tech that’s emerging in Europe is dangerous, man. Guess which country blocked access to a SOTA model based on national security bullshit.

Okay fair point

Re: GPT‑NL: a sovereign language model for the Netherlands

#187

Post by jampekka »

mschuster91 wrote: post #95
armcat wrote: post #70

I keep seeing these "sovereign" LMs time and time again. In Sweden we had GPT-SW3 ( https://www.ai.se/en/project/gpt-sw3 ) and same story there. Instead of burning money on "sovereign" claims, national research labs should instead focus on building on top of solid baselines (like Qwen/Kimi) and finetuning frontier models with real agentic utility that can be applied across actual use cases and can be widely used by i…

Kimi and Qwen come out of China, which means that their training material may be biased e.g. relating to Taiwan [1]. In addition, there is no way to determine what input went into the training, if it was properly licensed, if it was legal (e.g. not contaminated by CSAM), or how the human component of RLHF was sourced - in US models, for example, stories about exploitation like [2] have been floating for years. Assumi…

> Most LLMs focus on the English, German, French and Chinese languages, but everything else is... left behind at best

Current frontier models (closed and open) are already really good at small languages too. I use them in Finnish sometimes, and the language is immaculate. They underestand even somewhat obscure dialects. Multilinguality seems to be a mostly solved problem.

Re: GPT‑NL: a sovereign language model for the Netherlands

#188

Post by yread »

appplication wrote: post #153
armcat wrote: post #70

I keep seeing these "sovereign" LMs time and time again. In Sweden we had GPT-SW3 ( https://www.ai.se/en/project/gpt-sw3 ) and same story there. Instead of burning money on "sovereign" claims, national research labs should instead focus on building on top of solid baselines (like Qwen/Kimi) and finetuning frontier models with real agentic utility that can be applied across actual use cases and can be widely used by i…

Disagree, it’s in the country’s best interest to facilitate internal expertise on the full stack and own their “supply chain” so to speak and fight brain drain. The outcome isn’t just the model, it’s the expertise. Otherwise all their smartest folks will depart for countries where LLM development is strongest.

Great, now they will get expertise and then get hired by openai and move

Re: GPT‑NL: a sovereign language model for the Netherlands

#189

Post by teekert »

armcat wrote: post #70

I keep seeing these "sovereign" LMs time and time again. In Sweden we had GPT-SW3 ( https://www.ai.se/en/project/gpt-sw3 ) and same story there. Instead of burning money on "sovereign" claims, national research labs should instead focus on building on top of solid baselines (like Qwen/Kimi) and finetuning frontier models with real agentic utility that can be applied across actual use cases and can be widely used by i…

There is something to be said for this "most cheapest" approach, there is also something to be said for making models that are entirely ethically sourced:

1. Free of controversy like unlicensed training materials

2. Free of exploitative rlfh loops by people in low-wages countries

3. The leasons learned (and published) from going through the entire training process on "European" hardware: "AI factories" (the term for Slurm HPC/HTC systems with lots of heavy GPU nodes, heavily subsidized by our government [0])

1 and 2 are strong counter-LLM arguments at the moment, and hold back some groups of potential users. Another is energy/water use, so going for maximum green energy would be a nice boon as well. 3 is something I consider to be highly useful for our European identity and "way of the ninja" (for you Naruto fans out there).

[0 https://hpc-portal.eu/funding-opportunities]

Re: GPT‑NL: a sovereign language model for the Netherlands

#190

Post by jampekka »

sublimefire wrote: post #69

It is crazy that anything Europe gets so much hate. IMO it is important to build models within the boundaries of smaller nations, using their own language. Research has to continue even if it is outside of US and China.

I was somewhat excited about these "sovereign" open models in the beginning, but it became soon apparent that they're not gonna be anything but toys compared to SOTA.

The problem is that there are a lot, at least 30, of these small projects scattered around, funded for a few years as some ad-hoc temporary coalition of universities and businesses. Those simply cannot compete with businesses spending tens of billions on developing these. Especially when you have to bring a spoon to a gunfight restricting to "clean" data.

Multilinguality is essentially a solved problem, and restricting too much on one language with more limited resources is gonna make the model worse in that language too.

Post reply on HN