Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

U.S. Department of Energy Launches the Genesis Open Models Initiative

genesisopenmodels.anl.gov

Post reply on HN
Print view View original

71–80 of 161 posts

Re: U.S. Department of Energy Launches the Genesis Open Models Initiative

#71

Post by boc »

MangoCoffee wrote: post #40
dmix wrote: post #34

Earlier quoted context omitted.

Mostly because it's generally a bad idea for government to try to compete with a brand new tech industry with hundreds of billions in private capital developing commercial models. If the American private industry does actually wash out vs Chinese open models there might be talent available for them to put money into, so maybe they are just preparing for that scenario in the meantime.

The American attitude is generally to let private companies build up a new industry so it can create jobs and pay taxes. However, in the LLM race, the Chinese open weight playbook pretty much killed that. China has basically commoditized LLMs. Chinese models are good enough, so the race has come down to who can offer the cheapest tokens.

Chinese open weight models are great for this turn, but American private models generate orders of magnitude more cashflow. This cashflow = investment in training future models. It's unclear how Chinese open weight companies are going to compete in future rounds if they can't raise the same capital for training runs.

The American business model is exceedingly efficient at building large businesses from zero. I wouldn't dismiss it as just a jobs creation thing.

Re: U.S. Department of Energy Launches the Genesis Open Models Initiative

#72

Post by cududa »

lithobraking wrote: post #66

I'm interested to see where they want to land performance-wise (i.e. which point they choose on the scaling curve) and the niche they want to carve. They have a decent ways to scale beyond trinity large, in paticular on posttrain/RL before they are competitive with open-weights, especially internationally. Deepseek is explicitly banned [1] at LLNL and I wouldn't be suprised if there's a blanket ban on all Chinese mod…

I'd actually suggest a great starting point would be a local command reviewer LLM. Could ostensibly be a modern AV type thing. Particularly seeing this lately has driven the need home deeper to me: https://x.com/chrisbanes/status/2085341561609425230?s=20

An open weight tool call auto-reviewer, has all sorts of achievable scaling curve milestones.

Re: U.S. Department of Energy Launches the Genesis Open Models Initiative

#73

Post by buildbot »

coder543 wrote: post #22
firasd wrote: post #16

Earlier quoted context omitted.

Just looked into some Nemotron stats Looks like on https://arena.ai > agent arena (grouped by lab) Nvidia is 15/15 (much worse than Thinky and Mistral) and on text arena it's 18/27 On https://openrouter.ai/models?order=most-popular > I definitely see usage though (probably mostly cause Nemotron 3 Ultra is free) the grouped order is DeepSeek, Tencent, Xiaomi, OpenAI, Z.ai, Nvidia

I think glancing at a random snapshot from today misses all the context. Nemotron 3 is far more significant than you're giving it credit for. At this point, Nemotron 3 is really an 8 month old model series. That's when Nemotron 3 Nano was released, and the Nemotron 3 Super/Ultra models this year are obviously based on that recipe, mostly just bigger with a few tweaks here and there. Against today's models, no, not th…

Nemotron 3 also introduced LatentMoE, which was adopted by Kimi K3 :)

Re: U.S. Department of Energy Launches the Genesis Open Models Initiative

#74

Post by Tepix »

kadoban wrote: post #48
behnamoh wrote: post #37

Earlier quoted context omitted.

No it doesn't follow instructions and is substantially slower than ds4.

It's a lot smaller, and runs (quantized) on a 3090 quite well. Ds4 flash 0731 you're talking about? It's great but it's much harder to run locally.

Are you talking about S or XS? S is too large for a 3090 at 118b parameters.

Re: U.S. Department of Energy Launches the Genesis Open Models Initiative

#75

Post by strictnein »

firasd wrote: post #16
ipsum2 wrote: post #12

Earlier quoted context omitted.

There's a bunch of American open models. Inkling, Nemotron, Trinity come to mind, but I'm sure there's others.

Just looked into some Nemotron stats Looks like on https://arena.ai > agent arena (grouped by lab) Nvidia is 15/15 (much worse than Thinky and Mistral) and on text arena it's 18/27 On https://openrouter.ai/models?order=most-popular > I definitely see usage though (probably mostly cause Nemotron 3 Ultra is free) the grouped order is DeepSeek, Tencent, Xiaomi, OpenAI, Z.ai, Nvidia

Nemotron is a very nice model with an excellent license as well.

Re: U.S. Department of Energy Launches the Genesis Open Models Initiative

#76

Post by ericd »

kadoban wrote: post #48
behnamoh wrote: post #37

Earlier quoted context omitted.

No it doesn't follow instructions and is substantially slower than ds4.

It's a lot smaller, and runs (quantized) on a 3090 quite well. Ds4 flash 0731 you're talking about? It's great but it's much harder to run locally.

I found Laguna S to be pretty good at coding, pretty fast, but pretty bad as an agent - not proactive, would frequently stubbornly argue things that weren't true, and pretty bad general knowledge.

But as a pure coding model, pretty good.

Deepseek v4 Flash 0731 is so much better if you can run it, though.

Grain of salt, I think I grabbed Laguna after they fixed the initial looping issues, didn't notice those, but there might've been other fixes since.

Re: U.S. Department of Energy Launches the Genesis Open Models Initiative

#77

Post by unethical_ban »

lithobraking wrote: post #66

I'm interested to see where they want to land performance-wise (i.e. which point they choose on the scaling curve) and the niche they want to carve. They have a decent ways to scale beyond trinity large, in paticular on posttrain/RL before they are competitive with open-weights, especially internationally. Deepseek is explicitly banned [1] at LLNL and I wouldn't be suprised if there's a blanket ban on all Chinese mod…

That's interesting a locally hosted LLM would be banned. I'm assuming locally hosted is included. Do they think it's been trained to sabotage equipment?

Re: U.S. Department of Energy Launches the Genesis Open Models Initiative

#78

Post by johnecheck »

loeg wrote: post #51
solomatov wrote: post #20

Earlier quoted context omitted.

Which remarks? Could you share a link?

He said something to the effect of "I love open source and open models and we'll do open models when it makes sense and closed models when it makes sense" in a recent Q&A.

Given that his company has already released open models, I find it funny that, as you described it, his remark communicates absolutely nothing whatsoever. Not sure what the question was, but this was an artful non-answer.

Re: U.S. Department of Energy Launches the Genesis Open Models Initiative

#79

Post by CMay »

firasd wrote: post #8

Just realized that there are basically no American open models right now ever since the Llama series was abandoned. Basically Gemma and GPT-OSS I guess? Ah but Mira Murati's new Inkling is Apache 2.0 But it makes sense that if you're a university researcher you are thinking about what's a model that will be open weight and developed over the long term and doesn't raise 'Chyna' concerns in Washington DC

LiquidAI LFM models are amazing, but very situational. IBM Granite series are also unique and interesting for trying to reduce liability and extend local context size. Nvidia ships some and there was also that Inkling model recently. Poolside just released theirs.

Meta might release something this year. X AI's Grok is still due to release a model, if Elon keeps to his word even if they only release a distilled version. Reflection AI has been quiet, but their access to compute is ramping up. Microsoft's MAI is considering releasing some open weight models which would be great to see!

Ilya's SSI is unlikely to release an open model since he's aiming for radical safety. That bet could pay off if the existing approach produces so much chaos within the next 10-20 years that some global ban is achieved and a super safe model is promoted as the compliant route.

We don't get many huge model releases though. I think it's harder and more expensive to safety align them. Even if you do, people will work around the safety and abuse the models. Plus it makes it even easier for Chinese companies to distill things that aren't as easy over filtered APIs.

There is a lot of internet propaganda to the effect that the US is simply unable to release open weight models or that China has so many more AI companies that the US is drowning in Chinese open weight models, but it's more like we're being careful and China doesn't care. If you host a model in China, it has to be censored and downloading any models requires you to provide your identity. Huggingface is banned there. When they release their open models in the west, they don't have to care whether the models are aligned in any way.

Re: U.S. Department of Energy Launches the Genesis Open Models Initiative

#80

Post by jauntywundrkind »

CMay wrote: post #79
firasd wrote: post #8

Just realized that there are basically no American open models right now ever since the Llama series was abandoned. Basically Gemma and GPT-OSS I guess? Ah but Mira Murati's new Inkling is Apache 2.0 But it makes sense that if you're a university researcher you are thinking about what's a model that will be open weight and developed over the long term and doesn't raise 'Chyna' concerns in Washington DC

LiquidAI LFM models are amazing, but very situational. IBM Granite series are also unique and interesting for trying to reduce liability and extend local context size. Nvidia ships some and there was also that Inkling model recently. Poolside just released theirs. Meta might release something this year. X AI's Grok is still due to release a model, if Elon keeps to his word even if they only release a distilled versio…

Allen Institute for AI has quite a range of very interesting very competent more specialized models, for earth sensing, embedded robots, for others. Their SERA model shows a remarkably capable model for such a deliberately small investment effort, with documentation on how you can train such a model yourself or refine it easily at little cost. Their EMO pioneered a better MoE with great numbers (at least at the time). https://allenai.org/
Post reply on HN