Apparently this comes from Meituan which is a Chinese food delivery company.
LongCat-2.0, a large-scale MoE model with 1.6T total and 48B Active
11–20 of 98 posts
Re: LongCat-2.0, a large-scale MoE model with 1.6T total and 48B Active
#12I just tested it with a slightly tricky question > If you could run a nuclear reactor with U-235 as fuel or Pu-241 (both mixed with 95% U-238), which one would you choose and why? For a human this would not be tricky at all. For an LLM it could be, because this question certainly does not exist in any sort of training, because Pu-241 does not exist in pure form, it only exist as a minor component of reactor-grade plu…
Did you ask the question several times in fresh chat contexts to see if it sometimes gives the right answer ?
Nah, n=1 is enough to give evidence that something is entirely broken, of course.
/s
Re: LongCat-2.0, a large-scale MoE model with 1.6T total and 48B Active
#13[deleted]
Re: LongCat-2.0, a large-scale MoE model with 1.6T total and 48B Active
#14I wish they would release the requirements to run on llama.cpp with any announcements of open models. A bonus would be tok/s on common hardware.
I don't think llama.cpp supports any of the LongCat models, actually. They haven't posted weights/inference solutions for LongCat-2.0 [1], but LongCat-Next had transformers support, which I assume means it works with vLLM/SGLang. Given it's 1.6T, "common hardware" is probably out of the question; even 2bpw is going to measure out at 400GB, even before considering the bandwidth requirements for 48B active. I haven't r…
Yeah, for me it seems like a if you have to ask you can't run it" type question.
In general the TL;DR is that anything above 35B needs hardware you buy basically only to run large LLMs, and if you have that hardware you don't need to ask the question.
Re: LongCat-2.0, a large-scale MoE model with 1.6T total and 48B Active
#15[flagged]
Re: LongCat-2.0, a large-scale MoE model with 1.6T total and 48B Active
#16I just tested it with a slightly tricky question > If you could run a nuclear reactor with U-235 as fuel or Pu-241 (both mixed with 95% U-238), which one would you choose and why? For a human this would not be tricky at all. For an LLM it could be, because this question certainly does not exist in any sort of training, because Pu-241 does not exist in pure form, it only exist as a minor component of reactor-grade plu…
I am not a physicist but perhaps your question was leading more than you expected? I would take the question to pre-suppose I have an abundance of the stated material, ignoring practical realities of refinement. If I did have fully pure Pu-241, would that be a better fuel than U-235? Or stated another way, "If you could run a generator on gasoline or jet fuel, which one would you choose and why?" I would answer jet f…
If I did have fully pure Pu-241, would that be a better fuel than U-235?
Also not a physicist, but I assume from the fact that the OP is asking the LLM this question to trip it up, the point is that U-235 is better even if you have an abundance of both. It's scarcity of Pu-241 leads to the lack of data in training, not that it's actually better.
Re: LongCat-2.0, a large-scale MoE model with 1.6T total and 48B Active
#17> The training and deployment of LongCat-2.0 are built on large-scale clusters of tens of thousands of AI ASIC superpods. Compared to the mature Nvidia GPU ecosystem, the supporting software community is still less developed. We have therefore put significant effort into building a stable, secure, and scalable infrastructure. This is the real news story. It looks like they may have used Huawei Ascend 910C chips: http…
If they really managed this from pre-training a 1.6 T parameter model through to post-training without NVIDIA, Dwarkesh Patel got what he wanted.
Re: LongCat-2.0, a large-scale MoE model with 1.6T total and 48B Active
#18> The training and deployment of LongCat-2.0 are built on large-scale clusters of tens of thousands of AI ASIC superpods. Compared to the mature Nvidia GPU ecosystem, the supporting software community is still less developed. We have therefore put significant effort into building a stable, secure, and scalable infrastructure. This is the real news story. It looks like they may have used Huawei Ascend 910C chips: http…
If they really managed this from pre-training a 1.6 T parameter model through to post-training without NVIDIA, Dwarkesh Patel got what he wanted.
Who? What did he want?
Re: LongCat-2.0, a large-scale MoE model with 1.6T total and 48B Active
#19I asked a question with "Search" enabled, with the app set to English, and got results back in Chinese. Interesting view into how the LLM responds to its context.
Re: LongCat-2.0, a large-scale MoE model with 1.6T total and 48B Active
#20Apparently this comes from Meituan which is a Chinese food delivery company.
I don't think this is where you were going with your comment, but I'll mention this just because you're somewhat adjacent to a routine mistake in business:
Uber is a people delivery company, but they've had a lot of bright engineers working for them on their infrastructure and software over the years, and that work has rippled out through the industry.
Amazon (in VMWare's words) is "a company that sells books", and their leadership couldn't accept they were losing to them ("I look at this audience, and I look at VMware and the brand reputation we have in the enterprise, and I find it really hard to believe that we cannot collectively beat a company that sells books.").