Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT‑NL: a sovereign language model for the Netherlands

tno.nl

Post reply on HN
Print view View original

251–260 of 325 posts

Re: GPT‑NL: a sovereign language model for the Netherlands

#251

Post by torginus »

bradhe wrote: post #167
appplication wrote: post #153

Earlier quoted context omitted.

Disagree, it’s in the country’s best interest to facilitate internal expertise on the full stack and own their “supply chain” so to speak and fight brain drain. The outcome isn’t just the model, it’s the expertise. Otherwise all their smartest folks will depart for countries where LLM development is strongest.

Got it, so every country should focus on having a mediocre-at-best AI strategy by refusing to work together? Surely this will create a better future instead of pooling resources. This vaguely-nationalist world view around tech that’s emerging in Europe is dangerous, man. On the brain drain problem in particular, one way to ensure talent sticks around is to create a good environment for people to do their best work. I…

wdym? it says 13.5 million euros has been allocated to the project - you aint building a frontier LLM with that kind of money, but its enough to give research opportunities and real-world experience to a handful of researchers, and build up local competence.

These people can then fund for-profit companies once they have a promising approach and bring in private investment.

As for nationalism, like it or not, this is a govt sponsored effort, and governments and universities are funded by the public of specific nations

Re: GPT‑NL: a sovereign language model for the Netherlands

#252

Post by Gud »

trollbridge wrote: post #245
Gud wrote: post #239

Earlier quoted context omitted.

Yes, the current administration has been in power for less than two years so obviously the “day to day life” hasn’t changed much for the average American (except everything has gotten more expensive). The US is no longer looked at as a superpower, it just surrendered to a regional power and will pay $300B USD in tribute. But for Trump and his cronies life certainly has improved. But the damage is done and we are witn…

Nitpick: it’s 6 years.

The Republican Party looked very different during Trumps first term

Re: GPT‑NL: a sovereign language model for the Netherlands

#253

Post by GTP »

rapidfl wrote: post #135
zozbot234 wrote: post #81

Earlier quoted context omitted.

You keep building on the last available version? Fine tuning is a whole lot cheaper, easier and more useful than pretraining a model from scratch. It's a complete no brainer.

> You keep building on the last available version? yes but a sovereign can allocate some resources and a few people to stay in the loop from a first principles level. No need to wait for a rug pull. Of course, it can not compete with the frontier labs. But good to have researchers and professors "in-house". LLMs are here for the long-term.

> But good to have researchers and professors "in-house".

I'm not in this field, but I think we already have them. Probably the main difference is that we have most or all of them in academia and next to none insode private companies. But we do have them, and they could start working for private companies if the market moves in that direction in the EU as well

Re: GPT‑NL: a sovereign language model for the Netherlands

#254

Post by TalkingCodeMonk »

user43928 wrote: post #224
TalkingCodeMonk wrote: post #212

Earlier quoted context omitted.

This "greed is good, and should be rewarded" philosophy is one I see all too often on HN, and the entire reason why Americas political, regulatory, and business leadership have been overrun by the countries most criminally corrupt narcissists and psychopaths; why its democracy has collapsed. When you reward the most selfish, corrupt, and antisocial behaviours with wealth and power, you're guaranteed to create a selfi…

If tax incentives to attract companies and at-will employment are already viewed as a destructive collapse of democracy, it's no wonder we are not getting anything done here in Europe.

I suggest you take the time to read independent analysis on the benefits of corporate subsidies in America.

I know I've read of many cases where they cost cities and communities far more than the economic activity they promise to generate (eg Amazon warehouses and sportsball stadiums).

Subsidies could make strategic sense in the case of AI, but it's terrible economics to encourage businesses that take more from communities than they give back, especially those that don't employ many people.

Re: GPT‑NL: a sovereign language model for the Netherlands

#256

Post by andy12_ »

Zambyte wrote: post #247
andy12_ wrote: post #232

Earlier quoted context omitted.

I'm from Spain and I also hate these projects with passion. Creating models that speak multiple languages is a solved problem. Having each European Nation train its own useless "sovereign model" in its own language is a total waste of time and resources when we could pool resources and give it a try to training SOTA models that speak in all European languages. I'd rather have smaller european labs try to give it a go…

Excuse my ignorance if by "distributed training" you mean a specific process, but couldn't this be considered a step toward distributed training? If nations train models independently and then later distill them into a single model, all the work (both the compute and the research processes) are distributed for the initial training phase.

I mean it as in, train a model across different clusters instead of a centralized cluster. It's been shown that it's possible to train 10B models this way. If more research effort was put into this, that would be great

I don't think your approach would work because you can't create a strong model from distilling several weak models.

https://www.primeintellect.ai/blog/intellect-1

https://www.primeintellect.ai/blog/intellect-2-release

Re: GPT‑NL: a sovereign language model for the Netherlands

#257

Post by saidnooneever »

armcat wrote: post #70

I keep seeing these "sovereign" LMs time and time again. In Sweden we had GPT-SW3 ( https://www.ai.se/en/project/gpt-sw3 ) and same story there. Instead of burning money on "sovereign" claims, national research labs should instead focus on building on top of solid baselines (like Qwen/Kimi) and finetuning frontier models with real agentic utility that can be applied across actual use cases and can be widely used by i…

TNO doesn't serve nor represent global interest and hence does not care about global progress. It exists to enhance knowledge on things within Dutch society primarily with some ripple effect outward to EU because they have interests within the EU.

Its purpose is not to become some kind of OpenAI or global foundation offering services/tools on that scale.

There is a lot of critisism on this project, not invalid, but mostly based in lack of understanding of what the goals are of the organization as well as the people building the thing.

The people building it, are well aware of how it will be less capable than other LLMs on a general reasoning aspect, not only due to having actually purchased _all_ licensed data that has been used as inputs. Not being a multi billion dollar corporation, this means having very little data and should be an obvious signal to observers that it has not the goal to outdo other models.

In my opinion (personal) its a project that has a learning and demonstration value that is not 'look how well our model performs against others', but still offers value.

Re: GPT‑NL: a sovereign language model for the Netherlands

#258

Post by saidnooneever »

trhway wrote: post #233
Certhas wrote: post #221

Earlier quoted context omitted.

This "vaguely nationalistic world view around tech" is a direct consequence of the US government weaponizing its leading tech firms reach into the EU for nationalistic purposes. The EU was build on the principles of collaboration, to overcome the nationalistic impulses. Free trade and free movement of people, no need for everyone to replicate what everyone else is doing. Preventing individual nation states from favor…

>This "vaguely nationalistic world view around tech" is a direct consequence of the US government weaponizing its leading tech firms reach into the EU for nationalistic purposes. And in response NL should weaponize ASML for example. Then both sides would naturally back down. Specialization is the most efficient way of developing tech civilization. Whereis everybody building their own mediocre versions would be repeat…

Of NL weaponizes ASML they will just take their IP and leave NL and then they have nothing. They have already pushed a lot of things out of NL and should invest in leveraging the position without weaponizing it.

Re: GPT‑NL: a sovereign language model for the Netherlands

#259

Post by saidnooneever »

torginus wrote: post #251
bradhe wrote: post #167

Earlier quoted context omitted.

Got it, so every country should focus on having a mediocre-at-best AI strategy by refusing to work together? Surely this will create a better future instead of pooling resources. This vaguely-nationalist world view around tech that’s emerging in Europe is dangerous, man. On the brain drain problem in particular, one way to ensure talent sticks around is to create a good environment for people to do their best work. I…

wdym? it says 13.5 million euros has been allocated to the project - you aint building a frontier LLM with that kind of money, but its enough to give research opportunities and real-world experience to a handful of researchers, and build up local competence. These people can then fund for-profit companies once they have a promising approach and bring in private investment. As for nationalism, like it or not, this is…

this exactly. its not nationalistic at all to support your local development. this project has different goals than people perceive and their comments are all missing the plank ;)

Re: GPT‑NL: a sovereign language model for the Netherlands

#260

Post by joe_mamba »

bigfudge wrote: post #185
joe_mamba wrote: post #32

Earlier quoted context omitted.

Please don't move the goalposts. What computer parts does ASML or NXP make? ASML only makes the lithography machines, 85% of which go outside the EU (let that sink in). And then fabs in Taiwan, Korea or the US use those ASML machines to etch US IP for computer chips. EU doesn't make any computer parts domestically. And NXP mostly makes various microcontrollers and small chips, not high margin IP decenter centric part…

Europe is currently hosed because we made the mistake of trying to develop economies complementary to the US and china. That was a big strategic mistake. In the US case it was borne of the mistaken belief that we shared values and were partners. But don’t mistake the situation for lack of innovation of capability. Europe is currently adapting, but I think the success of Ukraine is one reason to be optimistic that cur…

>Europe is currently hosed because we made the mistake of trying to develop economies complementary to the US and china.

NO, it's hosed because it's not competitive and slept on the wheel at several key digital economical revolutions plus sleeping at the wheel at preventing obvious geopolitical issues (gas dependence to Russia, losing auto industry to China, losing semiconductor industry, losing SW industry, etc).

You can't be an economic leader if you keep losing on all fronts and only be a leader at how much welfare your spending.

> the mistaken belief we shared values and were partners.

We do share. US and a lot of latin america is mostly European immigrants and European culture, making our cultures are much more similar than the african and middle eastern ones the EU has been importing and adopting. Where we differ is that US still has free speech and isn't devolving into a stasi police state that arrests people for Tweets that the political establishment find uncomfortable.

> might actually leave us better off in the long run.

How? EU's economy has been pretty much stagnant since 2019 when you account for inflation loss.

> But don’t mistake the situation for lack of innovation of capability. Europe is currently adapting,

How? Where is Europe's Nvidia and AMD? Where is Europe's TSMC? ASML can't feed an entire continent.

Post reply on HN