Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Trinity large: An open 400B sparse MoE model

arcee.ai

Post reply on HN
Print view View original

61–70 of 83 posts

Re: Trinity large: An open 400B sparse MoE model

#61

Post by anon373839 »

nl wrote: post #38
zamadatix wrote: post #28

Earlier quoted context omitted.

It'd certainly be odd if people were recommending old LLMs which score worse, even if marginally. That said, 4o is really a lot more usable than you're making it out to be. The particular benchmark in the example is fungible but you have to pick something to make a representative example. No matter which you pick someone always has a reason "oh, it's not THAT benchmark you should look at". The benchmarks from the cha…

Progress has not become linear. We've just hit the limits of what we can measure and explain easily. One year ago coding agents could barely do decent auto-complete. Now they can write whole applications. That's much more difficult to show than an ELO score based on how people like emjois and bold text in their chat responses. Don't forget Llama4 led Lmarena and turned out to be very weak.

Much of these gains can be attributed to better tooling and harnesses around the models. Yes, the models also had to be retrained to work with the new tooling, but that doesn’t mean there was a step change in their general “intelligence” or capabilities. And sure enough, I’m seeing the same old flaws as always: frontier models fabricating info not present in the context, having blindness to what is present, getting into loops, failing to follow simple instructions…

Re: Trinity large: An open 400B sparse MoE model

#62

Post by freakynit »

cyanydeez wrote: post #22
linolevan wrote: post #15

Earlier quoted context omitted.

This model is sort of interesting since it seems to be using a lot of synthetic training data – but your point stands

So it's a rip off of a rip off, is that whats interesting?

reminds of this recent news https://www.medianama.com/2026/01/223-nvidia-high-speed-acce...

Re: Trinity large: An open 400B sparse MoE model

#63

Post by querez »

YetAnotherNick wrote: post #54
querez wrote: post #44

Earlier quoted context omitted.

> No Gemini was not "entirely trained on TPUs". They did hundreds of experiments on GPUs to get to the final training run done entirely on TPUs. GCP literally has millions of GPUs and you bet your ass that the gemini team has access to them and uses them daily. You are wrong. Gemini was definitely trained entirely on TPU. Of course your point of "you need to count failed experiments, too". Is correct. But you seem to…

Using TPU has the same opportunity cost as GPU. Just because they built something doesn't mean it's cheaper. If it is they can rent it cheaper to save money on paying billions of dollars to Nvidia. A big segment of the market just uses GPU/TPU to train LLMs, so they don't exactly need flexibility if some tool is well supported.

I assume TPU TCO is significantly cheaper than GPU TCO. At the same time, I also assume that market demand for GPUs is higher than TPUs (external tooling is just more suited to GPU -- e.g. I'm not sure what the Pytorch-on-TPU story is these days, but I'd be astounded if it's on par with their GPU support). So moving all your internal teams to TPUs means that all the GPUs can be allocated to GCP.

Re: Trinity large: An open 400B sparse MoE model

#64

Post by yorwba »

radarsat1 wrote: post #60
timschmidt wrote: post #10

Earlier quoted context omitted.

Even with MoE, holding the model in RAM while individual experts are evaluated in VRAM is a bit of a compromise. Experts can be swapped in and out of VRAM for each token. So RAM VRAM bandwidth becomes important. With a model larger than RAM, that bandwidth bottleneck gets pushed to the SSD interface. At least it's read-only, and not read-write, but even the fastest of SSDs will be significantly slower than RAM. That…

> Experts can be swapped in and out of VRAM for each token. I've often wondered how much it happens in practice. What does the per-token distribution of expert selection actually look like during inference? For example does it act like uniform random variable, or does it stick with the same 2 or 3 experts for 10 tokens in a row? I haven't been able to find much info on this. Obviously it depends on what model you are…

The model is explicitly trained to produce as uniform a distribution as possible, because it's designed for batched inference with a batch size much larger than the expert count, so that all experts are constantly activated and latency is determined by the highest-loaded expert, so you want to distribute the load evenly to maximize utilization.

Prompt ingestion is still fairly similar to that setting, so you can first compute the expert routing for all tokens, load the first set of expert weights and process only those tokens that selected the first expert, then load the second expert and so on.

But if you want to optimize for single-stream token generation, you need a completely different model design. E.g. PowerInfer's SmallThinker moved expert routing to a previous layer, so that the expert weights can be prefetched asynchronously while another layer is still executing: https://arxiv.org/abs/2507.20984

Re: Trinity large: An open 400B sparse MoE model

#65

Post by DoctorOetker »

gwern wrote: post #20
zamadatix wrote: post #5

Earlier quoted context omitted.

The price/scaling of training another same class model always seems to be dropping through the floor but training models which score much better seems to be hitting a brick wall. E.g. gemini-3-pro tops the lmarena text chart today at 1488 vs 1346 for gpt-4o-2024-05-13. That's a win rate of 70% (where 50% is equal chance of winning) over 1.5 years. Meanwhile, even the open weights stuff OpenAI gave away last summer sc…

> E.g. gemini-3-pro tops the lmarena text chart today at 1488 vs 1346 for gpt-4o-2024-05-13. That's a win rate of 70% (where 50% is equal chance of winning) over 1.5 years. Meanwhile, even the open weights stuff OpenAI gave away last summer scores between the two. Why do you care about LM Arena? It has so many problems, and the fact that no one would suggest using GPT-4o for doing math or coding right now, or much of…

It very sad there is so much gaming of metrics with LLMs.

If we wish to avoid everyone creating benchmarks for themselves, then instead of predetermined benchmarks (public ones allow gaming, while publicly scored private ones require blind trust) we could use gradient descent on sentences to find disagreements between models, and then present them to human domain experts.

At least it could be public without possibility of leaking (since the model creators don't yet know of all possible disagreements between LLM's, which ones will be selected for review by human experts)

Re: Trinity large: An open 400B sparse MoE model

#66

Post by lumost »

zamadatix wrote: post #5
mynti wrote: post #3

They trained it in 33 days for ~20m (that includes apparently not only the infrastructure but also the salaries over a 6 month period). And the model is coming close to QWEN and Deepseek. Pretty impressive

The price/scaling of training another same class model always seems to be dropping through the floor but training models which score much better seems to be hitting a brick wall. E.g. gemini-3-pro tops the lmarena text chart today at 1488 vs 1346 for gpt-4o-2024-05-13. That's a win rate of 70% (where 50% is equal chance of winning) over 1.5 years. Meanwhile, even the open weights stuff OpenAI gave away last summer sc…

It’s becoming clear that training a frontier model is a capex/infra problem. This problem involves data acquisition, compute, and salaries for the researchers familiar with the little nuances of training at this scale.

For the same class model, you can train on more or less the same commodity datasets. Over time these datasets become more efficient to train on as errata are removed and the data is cleaner. The cost of dataset acquisition can be amortized and sometimes drops to 0 as the dataset is open sourced.

Frontier models mean acquiring fresh datasets at unknown costs.

Re: Trinity large: An open 400B sparse MoE model

#67

Post by YetAnotherNick »

querez wrote: post #63
YetAnotherNick wrote: post #54

Earlier quoted context omitted.

Using TPU has the same opportunity cost as GPU. Just because they built something doesn't mean it's cheaper. If it is they can rent it cheaper to save money on paying billions of dollars to Nvidia. A big segment of the market just uses GPU/TPU to train LLMs, so they don't exactly need flexibility if some tool is well supported.

I assume TPU TCO is significantly cheaper than GPU TCO. At the same time, I also assume that market demand for GPUs is higher than TPUs (external tooling is just more suited to GPU -- e.g. I'm not sure what the Pytorch-on-TPU story is these days, but I'd be astounded if it's on par with their GPU support). So moving all your internal teams to TPUs means that all the GPUs can be allocated to GCP.

Just doesn't make sense. If you make significantly more money renting TPU, why not rent them cheaper to shift the customers(and save billions that you are giving to Nvidia). TPU right now isn't significantly more cheaper to external customer.

Again I am talking about LLM training/inference which if I were to guess is more than half of the workload currently for which the switching cost is close to 0.

Re: Trinity large: An open 400B sparse MoE model

#68

Post by trilogic »

Testing it now in HugstonOne. Running smooth at 5.8 T/S : Loaded Trinity-Large-Preview-UD-Q4_K_XL-00001-of-00005.gguf.

The T/S speed is acceptable, also stable 60 degrees celcius for the gpu temperature. Accuracy and precision in math problems. So far so good. Results: https://www.reddit.com/r/Hugston/comments/1qq9d5i/testing_tr...

Re: Trinity large: An open 400B sparse MoE model

#69

Post by esskay »

zamadatix wrote: post #5
mynti wrote: post #3

They trained it in 33 days for ~20m (that includes apparently not only the infrastructure but also the salaries over a 6 month period). And the model is coming close to QWEN and Deepseek. Pretty impressive

The price/scaling of training another same class model always seems to be dropping through the floor but training models which score much better seems to be hitting a brick wall. E.g. gemini-3-pro tops the lmarena text chart today at 1488 vs 1346 for gpt-4o-2024-05-13. That's a win rate of 70% (where 50% is equal chance of winning) over 1.5 years. Meanwhile, even the open weights stuff OpenAI gave away last summer sc…

Training costs might be coming down but costs for hardware that can run these models is still obscenely high and rising. We're still nowhere near a point where its realistically feasible to run a home LLM that doesn't feel like it's suffering with severe brain damage.

Re: Trinity large: An open 400B sparse MoE model

#70

Post by LoganDark »

NitpickLawyer wrote: post #56
LoganDark wrote: post #36

According to the article, nearly 50% of the dataset is synthetic (8T out of 17T tokens). I don't know what constitutes "a breadth of state-of-the-art rephrasing approaches", but I lack some confidence in models trained on LLM output, so I hope it wasn't that.

> but I lack some confidence in models trained on LLM output, so I hope it wasn't that. That's misguided. Models have been trained on synthetic data for ~2+ years already. The "model collapse" myth is based on a very poor paper that got waaaay more attention than it deserved (because negativity sells, I guess). In practice every lab out there is doing this, because it works.

When ChatGPT first released and jailbreaks were pretty easy, I was able to easily get some extremely good/detailed output from it, with very little errors or weirdness. Now even when I can get jailbreaks to work with their newer models, it's just not the same, and no open-source model or even commercial model has seem to come close to the quality of that very first release. They're all just weird, dumb, random or incoherent. I keep trying even the very large open-source or open-weights models, and new versions of OpenAI's models and Claude and Gemini and so on, but it just all sucks. It all feels like slop!

I'm convinced it's because that first ChatGPT release was probably trained on data almost entirely untainted by other LLMs, and it may no longer ever be possible to obtain such a dataset again. Every model feels so artificial and synthetic. I do not know for sure why this is, but I bet it has something to do with people thinking it's possible to programmatically generate almost half the dataset?! I feel like OpenAI's moat could have been the quality and authenticity of their dataset, since they scraped practically most of the internet before LLMs became widespread, but even they've probably lost it by now.

I haven't really internalized anything about "model collapse", other than that if you train an LLM on outputs from other LLMs, you will be training to emulate an imprecise version of an imprecise version of writing, which will be measurably and perceptibly worse than merely one layer of imprecise version of actual writing.

Post reply on HN