Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Norway's 2 petabytes of Huawei flash storage and LLM training

blocksandfiles.com

Post reply on HN
Print view View original

221–227 of 227 posts

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#221

Post by fastasucan »

anal_reactor wrote: post #144
chvid wrote: post #101

Earlier quoted context omitted.

When I am chatting with ChatGPT - it is fairly obvious that it is American - its native language, its style, its attitude is American - even if we chat in Danish. Just as we cannot rely on Netflix and HBO to produce Scandinavian TV-shows even though they might do at the moment, we need to make our own stuff in this area too. And over time, the technology to do this will become cheap and readily available for us to do…

> And over time, the technology to do this will become cheap and readily available for us to do so. But then the English models will be even better and you'll be back to square one. My guess is that things are going to become more and more American. If you assume that "culture" is a resource like "microchips", then from economic point of view it makes sense to have one country specialize in producing it, and the rest…

>makes sense to have one country specialize in producing it, and the rest just consume

Oh ok then, just finnish culture for you then.

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#222

Post by fastasucan »

spwa4 wrote: post #17
kvam wrote: post #13

As a Norwegian this sounds like a mistake. Who will use this LLM? Where? For what? The underlying data could be made more easily searchable and digestible for agents in general if the goal is better knowledge of Norwegian culture.

Exactly, if there's one thing transformers are good at it's translation. One I've found particularly nice: any question ChatGPT can answer in English it can answer in French. I'm assuming Norwegian too. So there's no point.

There is a lot more to it than literal translations. Even if an american can talk Norwegian it doesn't mean that they get the cultural context right.

"Oh yeah after you drive 2 hours home from work your wife and kid will greet you with some delicious pie" doesnt work so well even if its in Norwegian.

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#223

Post by fastasucan »

kvam wrote: post #140
sisve wrote: post #26

Earlier quoted context omitted.

The point is that norway willl have its own LLM. And will not have dependencies to another state or private company. The goal is not to be the best model. But to have a model that include more Norwegian data then other LLM and that it's not screwed against other sources.

But what does that give you? If the model is far less capable? What will it do for you with that Norwegian data, that a better model could not do with better search or context?

Other better models cant use the training data this model have...

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#224

Post by fastasucan »

teknologist wrote: post #212

Modern frontier LLMs know Norwegian. Wouldn't this simply be the job of RAG to do lookups as the user requests for data? Like how Gemini integrates with Google Search. Seems like they should be building an MCP service rather than training an entirely new LLM...

Culture is a lot more than translated language.

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#225

Post by xadhominemx »

weregiraffe wrote: post #107
xadhominemx wrote: post #93

Earlier quoted context omitted.

This model is going to start miles behind the frontier and the gap will only grow.

Why would the gap grow? There is no more training data to acquire, frontier model are training on the entire internet. Everything from now on is just fine-tuning.

Orders of magnitude less compute for pre- and post-training than the the frontier labs.

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#226

Post by black_puppydog »

bakugo wrote: post #166
black_puppydog wrote: post #72

Earlier quoted context omitted.

I task GPT/Claude with researching stuff that pertains to very specific cultural or legal aspects in French politics, on a daily basis. Even though French is a way more common language globally than Norwegian, these models still haven't figured out that, no matter the language I myself speak to them (German or English depending on my mood) their web searches need to be done in French to return reasonable results. I h…

> their web searches need to be done in French to return reasonable results. I wonder how much of this is also just the search engine's region setting. It's a big problem I regularly have with Google. I almost always want English language, US-centric results, so I have my region set to the US. But occasionally I want results relevant to my actual country, and even searching in my native language usually yields much w…

I gave up on Google's language and region settings a long time ago, years before giving up on google as a product.

To this day they still think I'm in Sweden sometimes, in Paris other times, or in Germany, while I haven't lived in any of those places for years.

Post reply on HN