Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Norway's 2 petabytes of Huawei flash storage and LLM training

blocksandfiles.com

Post reply on HN
Print view View original

21–30 of 227 posts

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#21

Post by sgt »

spwa4 wrote: post #17
kvam wrote: post #13

As a Norwegian this sounds like a mistake. Who will use this LLM? Where? For what? The underlying data could be made more easily searchable and digestible for agents in general if the goal is better knowledge of Norwegian culture.

Exactly, if there's one thing transformers are good at it's translation. One I've found particularly nice: any question ChatGPT can answer in English it can answer in French. I'm assuming Norwegian too. So there's no point.

There's quite a bit more to culture and language than just being able to have transformers come up with believable language and/or dialect.

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#22

Post by otabdeveloper4 »

solenoid0937 wrote: post #11

> The Olivia system is an HPE Cray Supercomputing EX system, with 448 GPUs and 64,512 CPU cores. Training a sovereign LLM with this meager hardware as opposed to a LORA on some open source model seems like a huge mistake and a potential red flag. There is no way these people have the resources to train a fully fledged LLM, so claiming that is their goal makes me think they don't intend for the LLM to be useful. Which…

> meager hardware

Qwen was made on a cluster about that size.

And this is before anybody ever thought about optimizing the training process. (Currently it's just pytorch analyst-as-coder slop, with extremely overprovisioned quantizations, etc.)

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#23

Post by otabdeveloper4 »

spwa4 wrote: post #17
kvam wrote: post #13

As a Norwegian this sounds like a mistake. Who will use this LLM? Where? For what? The underlying data could be made more easily searchable and digestible for agents in general if the goal is better knowledge of Norwegian culture.

Exactly, if there's one thing transformers are good at it's translation. One I've found particularly nice: any question ChatGPT can answer in English it can answer in French. I'm assuming Norwegian too. So there's no point.

They're only good at it because they were trained on massive amounts of English and French data.

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#26

Post by sisve »

spwa4 wrote: post #17
kvam wrote: post #13

As a Norwegian this sounds like a mistake. Who will use this LLM? Where? For what? The underlying data could be made more easily searchable and digestible for agents in general if the goal is better knowledge of Norwegian culture.

Exactly, if there's one thing transformers are good at it's translation. One I've found particularly nice: any question ChatGPT can answer in English it can answer in French. I'm assuming Norwegian too. So there's no point.

The point is that norway willl have its own LLM. And will not have dependencies to another state or private company. The goal is not to be the best model. But to have a model that include more Norwegian data then other LLM and that it's not screwed against other sources.

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#27

Post by 100ms »

jauntywundrkind wrote: post #3

384 core cpu cluster? 2 petabytes? Dell just launched a 2U that fits almost 10 petabytes in it. It's probably not 384 core capable but that is very doable right now, Epyc chips are 192 cores each! https://www.techradar.com/pro/dell-launches-record-shatterin...

5x 400gbit running to a 2U box whoa, the PCI lanes must have heat shielding.

More seriously there is a sensibility limit on extreme density where it's not needed. The idea that you're just going to magically get 2 TBit/s out of those ports seems unlikely even with tweaked software, and you're stuck with a power and comms hotspot that's liable to dictate the remainder of your network design.

At max utilisation that 2U would take 12 hours to drain, and only 12 hours assuming peak and likely unachievable throughput and the box otherwise being completely out of service. Not a great start

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#28

Post by devttyeu »

arjie wrote: post #24

This can’t be right. 2 PB of flash is like $200k. It’s within reach of many individuals. Then again I guess you don’t need that much storage so maybe it is.

More like $1M at current prices at this scale / level of performance.

If you go with HDD arrays probably $50k

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#29

Post by kristjansson »

solenoid0937 wrote: post #11

> The Olivia system is an HPE Cray Supercomputing EX system, with 448 GPUs and 64,512 CPU cores. Training a sovereign LLM with this meager hardware as opposed to a LORA on some open source model seems like a huge mistake and a potential red flag. There is no way these people have the resources to train a fully fledged LLM, so claiming that is their goal makes me think they don't intend for the LLM to be useful. Which…

DeepSeek claims to have trained on something like 2k H800, this is ~0.5k GH200 … it’s not nothing. Sure they’re not going to _serve_ it at scale, but that’s not the point?

Also the line between “finetuning a base model” and “man this is a real good initialization” gets pretty blurry at scale.

Altogether a pretty presumptuous take.

Re: Norway's 2 petabytes of Huawei flash storage and LLM training

#30

Post by vslira »

solenoid0937 wrote: post #11

> The Olivia system is an HPE Cray Supercomputing EX system, with 448 GPUs and 64,512 CPU cores. Training a sovereign LLM with this meager hardware as opposed to a LORA on some open source model seems like a huge mistake and a potential red flag. There is no way these people have the resources to train a fully fledged LLM, so claiming that is their goal makes me think they don't intend for the LLM to be useful. Which…

It may not be useful to anyone outside, but it's possible that one of the goals is institutional learning (that is, embedding the knowledge in how to build LLMs in an organization).

Even though it's nominally the national library behind this, they were probably chosen (as per the article) because they legally own and can use all NO material for this end. I'd guess researchers from related entities like unis will be involved in the process.

Post reply on HN