Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Norway's 2 petabytes of Huawei flash storage and LLM training

blocksandfiles.com

Post reply on HN
Print view View original

181–190 of 227 posts

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#181

Post by internet_points »

KeplerBoy wrote: post #54

How true is this statement: "He asserted that any country with its own language that did not have a sovereign LLM trained in that language was at a disadvantage as a globally trained, English-speaking LLM would not know about that country’s history, news and culture that was described in the local language." I thought all big players already train on basically everything remotely available to them no matter the langu…

Maybe it can at least write like a Norwegian instead of just English-translated-into-Norwegian. It would be interesting to see if they try something like the experiments in https://arxiv.org/pdf/2507.22445 on it.

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#182

Post by rldjbpin »

may not be the most efficient way to go about things, but there remains a seemingly obvious use case for non-latin languages to do things from scratch.

see sarvam.ai and their tokenisation improvements on local languages [1]. not every llm needs to help with coding, nor it needs to already become Babel fish.

language is culture, so i can see the motivation behind their initiative. it must be nice to afford to do this yourself.

[1] https://www.sarvam.ai/blogs/sarvam-30b-105b

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#185

Post by kgeist »

rldjbpin wrote: post #182

may not be the most efficient way to go about things, but there remains a seemingly obvious use case for non-latin languages to do things from scratch. see sarvam.ai and their tokenisation improvements on local languages [1]. not every llm needs to help with coding, nor it needs to already become Babel fish. language is culture, so i can see the motivation behind their initiative. it must be nice to afford to do this…

>but there remains a seemingly obvious use case for non-latin languages to do things from scratch

>see sarvam.ai and their tokenisation improvements on local languages

You don't need to build from scratch to improve tokenization, though.

Russia's T-Bank was able to increase generation speeds by 1.5-3x by changing a stock Qwen's tokenizer to include 5 times more Cyrillic tokens (+ post-training on a Russian corpus).

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#186

Post by pjc50 »

embedding-shape wrote: post #65
WatchDog wrote: post #57

Earlier quoted context omitted.

If you want LLMs to have knowledge of the Norwegian language, wouldn't the most obvious thing to do be to build a good training dataset and make the dataset widely available? Why go to the expense of training your own model, especially when it will be inferior to state of the art models.

Yeah, was about to comment that too, instead of training a new model and new weights exclusively for Norwegian (and expecting/wanting every other small/medium-sized country to do the same) which seems infinity harder, they could have made high quality transcriptions and translations of the stories currently described only in Norwegian into English, and making it all public. I guess there still would be a worry that i…

Oddly enough, my wife was recently involved in a project to translate historical crime novels from Norwegian; since all the available late 20th century Scandinavian crime novels have already been translated and turned into popular TV series, the plan was to go further back. Into the 1930s. The first cut was done with LLMs, but encountered the problem that (a) Norwegian itself has changed noticeably since then, in both major dialects, and (b) the machine translation deteriorated on large sections, resulting in entirely missing paragraphs and pages in a few places. Not to mention the usual translation issues (what police role does lensman map to?) and localisation (to what extent should the casual antisemitism be left in or removed?)

Translation is never a bijective process. It's never quite the same experience in translation as it is in the original, due to the cultural differences between reader and writer. Larger in this case because 1930s Norway is very different even from 2020s Norway.

Ultimately this was not a success due to marketing difficulties; it is very difficult to get a book noticed.

( https://www.amazon.co.uk/Iron-Chariot-Nordic-Crime-Library/d... )

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#187

Post by pjc50 »

anal_reactor wrote: post #144
chvid wrote: post #101

Earlier quoted context omitted.

When I am chatting with ChatGPT - it is fairly obvious that it is American - its native language, its style, its attitude is American - even if we chat in Danish. Just as we cannot rely on Netflix and HBO to produce Scandinavian TV-shows even though they might do at the moment, we need to make our own stuff in this area too. And over time, the technology to do this will become cheap and readily available for us to do…

> And over time, the technology to do this will become cheap and readily available for us to do so. But then the English models will be even better and you'll be back to square one. My guess is that things are going to become more and more American. If you assume that "culture" is a resource like "microchips", then from economic point of view it makes sense to have one country specialize in producing it, and the rest…

> then from economic point of view it makes sense to have one country specialize in producing it, and the rest just consume

And, for exactly the same reasons as Europeans need to have sovereign compute to protect against economic imperialism, it is also essential to maintain local culture in order to avoid the great replacement of everything with Americanisms.

Yes, it requires pushing against the economics. But you have to do that if you believe that culture has any value per se at all.

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#188

Post by vidarh »

mettamage wrote: post #133
vidarh wrote: post #52

Earlier quoted context omitted.

It's really fantastic. I just wished there were fewer restrictions on the content that is accessible. (a lot is only accessible from Norwegian IP addresses, so it's one of the main reasons I maintain a VPN as I'm Norwegian but live in the UK; a second set is only available from the IP addresses of libraries or research institutions - still huge amounts that are generally available, though)

Silly question but can a non-Norwegian also access it? Willing to pick up some Norwegian along the way ;-)

You can access quite a bit directly. Check out nb.no (or https://www.nb.no/en/ for an English version of the page, but of course most of the works are in Norwegian)

There are escalating series of restrictions, basically:

* Available for everyone.

* Available from a Norwegian IP -> just requires a VPN.

* Available from Norwegian libraries

* Availble under "special conditions". This would mean from a participating research institution or university, or similar.

Pretty much everything that is out of copyright falls in the first category. The second and third categories has a bunch of copyrighted material where the copyright holders have granted limited usage rights. A bunch of newspaper archive material that is still under copyright (but sadly not the biggest ones) are available from Norwegian IPs for example.

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#189

Post by vidarh »

thatcat wrote: post #91
vidarh wrote: post #66

Earlier quoted context omitted.

You'd think so. It seems like there are a lot of odd gaps like that. I also have a favourite English language PhD thesis I ask every new model about that they still struggle to find even though there's a Wikipedia article about it that links a blog post I wrote about it. Anyone who thinks they've exhausted even publicly crawlable resources should ask them about some obscure stuff.

the models don't retain their full training data set

No, but they do retain enough that it is interesting what they fail to retain.

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#190

Post by embedding-shape »

pjc50 wrote: post #186
embedding-shape wrote: post #65

Earlier quoted context omitted.

Yeah, was about to comment that too, instead of training a new model and new weights exclusively for Norwegian (and expecting/wanting every other small/medium-sized country to do the same) which seems infinity harder, they could have made high quality transcriptions and translations of the stories currently described only in Norwegian into English, and making it all public. I guess there still would be a worry that i…

Oddly enough, my wife was recently involved in a project to translate historical crime novels from Norwegian; since all the available late 20th century Scandinavian crime novels have already been translated and turned into popular TV series, the plan was to go further back. Into the 1930s. The first cut was done with LLMs, but encountered the problem that (a) Norwegian itself has changed noticeably since then, in bot…

Sorry if I was unclear, I didn't want to give the impression I think translations or even transcriptions in some cases is easy, or without problems, or not painstakingly time-consuming, it very much is.

I just think building a LLM from scratch is ever harder, with more potential problems that are harder to solve, more time-consuming and even more resource-intensive.

Post reply on HN