Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LongCat-2.0, a large-scale MoE model with 1.6T total and 48B Active

longcat.chat

Post reply on HN
Print view View original

31–40 of 98 posts

Re: LongCat-2.0, a large-scale MoE model with 1.6T total and 48B Active

#31

Post by chvid »

BoorishBears wrote: post #17
gardnr wrote: post #4

> The training and deployment of LongCat-2.0 are built on large-scale clusters of tens of thousands of AI ASIC superpods. Compared to the mature Nvidia GPU ecosystem, the supporting software community is still less developed. We have therefore put significant effort into building a stable, secure, and scalable infrastructure. This is the real news story. It looks like they may have used Huawei Ascend 910C chips: http…

If they really managed this from pre-training a 1.6 T parameter model through to post-training without NVIDIA, Dwarkesh Patel got what he wanted.

It is interesting how much people doubt Huawei’s capabilities in this area - Jensen does not (in the dp interview) - of course you can dismiss this as him talking his own book.

Re: LongCat-2.0, a large-scale MoE model with 1.6T total and 48B Active

#34

Post by IshKebab »

3eb7988a1663 wrote: post #24
zythyx wrote: post #12

Earlier quoted context omitted.

Nah, n=1 is enough to give evidence that something is entirely broken, of course. /s

Well, when we had deterministic tools, it would only take a single example of a calculator claiming 1+1=4 for me to throw it in the trash.

And if you can come up with a deterministic tool that can do everything LLMs can then that would be amazing! Until then, we have to accept the non-determinism.

Re: LongCat-2.0, a large-scale MoE model with 1.6T total and 48B Active

#35

Post by throwa356262 »

1024 Huawei Ascend superpods = 50K 910C chips.

That is a tiny tiny system. OpenAI uses _milions_ of GPUs for training

On the other hand, this probably reuses the existing deepseek v4 architecture and weights. Maybe didn't need that much compute.

Re: LongCat-2.0, a large-scale MoE model with 1.6T total and 48B Active

#37

Post by cyberax »

3eb7988a1663 wrote: post #9
credit_guy wrote: post #6

I just tested it with a slightly tricky question > If you could run a nuclear reactor with U-235 as fuel or Pu-241 (both mixed with 95% U-238), which one would you choose and why? For a human this would not be tricky at all. For an LLM it could be, because this question certainly does not exist in any sort of training, because Pu-241 does not exist in pure form, it only exist as a minor component of reactor-grade plu…

I am not a physicist but perhaps your question was leading more than you expected? I would take the question to pre-suppose I have an abundance of the stated material, ignoring practical realities of refinement. If I did have fully pure Pu-241, would that be a better fuel than U-235? Or stated another way, "If you could run a generator on gasoline or jet fuel, which one would you choose and why?" I would answer jet f…

A higher delayed fraction of neutrons makes it easier to control the reactor. Without delayed neutrons you can only make a bomb.

Re: LongCat-2.0, a large-scale MoE model with 1.6T total and 48B Active

#38

Post by try-working »

ValentineC wrote: post #21
skybrian wrote: post #11

Apparently this comes from Meituan which is a Chinese food delivery company.

The thing that stood out for me about Meituan was that their power bank rental gizmos were everywhere in China, and people would rather rent a power bank than own and carry one around because of how convenient it is.

People buy millions of powerbanks in China.

Re: LongCat-2.0, a large-scale MoE model with 1.6T total and 48B Active

#39

Post by tryagainian »

3eb7988a1663 wrote: post #9
credit_guy wrote: post #6

I just tested it with a slightly tricky question > If you could run a nuclear reactor with U-235 as fuel or Pu-241 (both mixed with 95% U-238), which one would you choose and why? For a human this would not be tricky at all. For an LLM it could be, because this question certainly does not exist in any sort of training, because Pu-241 does not exist in pure form, it only exist as a minor component of reactor-grade plu…

I am not a physicist but perhaps your question was leading more than you expected? I would take the question to pre-suppose I have an abundance of the stated material, ignoring practical realities of refinement. If I did have fully pure Pu-241, would that be a better fuel than U-235? Or stated another way, "If you could run a generator on gasoline or jet fuel, which one would you choose and why?" I would answer jet f…

> Which would ignore that jet fuel is going to be a multiple of the gasoline price.

That doesn’t sound right. If my Duck Fu is any good, jet fuel is currently going due US$3.00 per gallon, avgas (leaded petrol) at $3.30, and gasoline at $2.88 gallon.

There’s nothing much special about jet fuel, it’s just kerosene, same as RP1 (Rocket Propellant), heater fuel, and lamp oil you can buy from the hardware store, with a touch of something to stop it gelling at low temperature if I understand correctly, but also jet fuel tanks are heated if I recall correctly.

I believe standard diesel fuel will also works in jet engines, but kerosene is cheaper.

I’m not in the US, and if I understand correctly their gasoline (petrol) price can vary greatly from state to state, California being the worst? Is that right?

Re: LongCat-2.0, a large-scale MoE model with 1.6T total and 48B Active

#40

Post by teaearlgraycold »

3eb7988a1663 wrote: post #9
credit_guy wrote: post #6

I just tested it with a slightly tricky question > If you could run a nuclear reactor with U-235 as fuel or Pu-241 (both mixed with 95% U-238), which one would you choose and why? For a human this would not be tricky at all. For an LLM it could be, because this question certainly does not exist in any sort of training, because Pu-241 does not exist in pure form, it only exist as a minor component of reactor-grade plu…

I am not a physicist but perhaps your question was leading more than you expected? I would take the question to pre-suppose I have an abundance of the stated material, ignoring practical realities of refinement. If I did have fully pure Pu-241, would that be a better fuel than U-235? Or stated another way, "If you could run a generator on gasoline or jet fuel, which one would you choose and why?" I would answer jet f…

It’s tough to write good questions for LLM evaluations. They’re so good at picking up subtleties they can pass a multiple choice test when given only the answers and not the questions.
Post reply on HN