Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open models by OpenAI

openai.com

Post reply on HN
Print view View original

211–220 of 909 posts

Re: Open models by OpenAI

#211

Post by golergka »

podnami wrote: post #48
sadiq wrote: post #41

Looks like Groq (at 1k+ tokens/second) and Fireworks are already live on openrouter: https://openrouter.ai/openai/gpt-oss-120b $0.15M in / $0.6-0.75M out edit: Now Cerebras too at 3,815 tps for $0.25M / $0.69M out.

Wow this was actually blazing fast. I prompted "how can the 45th and 47th presidents of america share the same parents?" On ChatGPT.com o3 thought for for 13 seconds, on OpenRouter GPT OSS 120B thought for 0.7 seconds - and they both had the correct answer.

When I pay attention to o3 CoT, I notice it spends a few passes thinking about my system prompt. Hard to imagine this question is hard enough to spend 13 seconds on.

Re: Open models by OpenAI

#213

Post by chromaton »

This has been available (20b version, I'm guessing) for the past couple of days as "Horizon Alpha" on Openrouter. My benchmarking runs with TianshuBench for coding and fluid intelligence were rate limited, but the initial results show worse results that DeepSeek R1 and Kimi K2.

Re: Open models by OpenAI

#214

Post by dns_snek »

sdenton4 wrote: post #49
snewman wrote: post #5

Earlier quoted context omitted.

No, because there are lots of things people can do that it still can't do.

"If it is still possible to put a goalpost somewhere - and we don't care where - then it's not AGI."

LLMs are what they are, calling them "AGI" won't make them any more useful or exciting than they are, it's just going to devalue the term "AGI" which has revolutionary, disease-curing, humanity-saving connotations. What are you looking for us to say exactly?

1. We aren't even close to AGI and it's unclear that we'll ever get there, but it would change the course of humanity in a significant way if we ever do.

2. Wow we've reached AGI but now I'm realizing that AGI is lame, we need a new term for the humanity-saving sales pitch that we were promised!

Re: Open models by OpenAI

#215

Post by redox99 »

BoorishBears wrote: post #54
johntiger1 wrote: post #10

Wow, this will eat Meta's lunch

Maverick and Scout were not great, even with post-training in my experience, and then several Chinese models at multiple sizes made them kind of irrelevant (dots, Qwen, MiniMax) If anything this helps Meta: another model to inspect/learn from/tweak etc. generally helps anyone making models

There's nothing new here in terms of architecture. Whatever secret sauce is in the training.

Re: Open models by OpenAI

#216

Post by Davidzheng »

n42 wrote: post #135
bogtog wrote: post #95

Earlier quoted context omitted.

When people talk about running a (quantized) medium-sized model on a Mac Mini, what types of latency and throughput times are they talking about? Do they mean like 5 tokens per second or at an actually usable speed?

here's a quick recording from the 20b model on my 128GB M4 Max MBP: https://asciinema.org/a/AiLDq7qPvgdAR1JuQhvZScMNr and the 120b: https://asciinema.org/a/B0q8tBl7IcgUorZsphQbbZsMM I am, um, floored

the active param count is low so it should be fast.

Re: Open models by OpenAI

#217

Post by 4b6442477b1280b »

a_wild_dandan wrote: post #175
4b6442477b1280b wrote: post #99

Earlier quoted context omitted.

in that era, OpenAI and Anthropic were still deluding themselves into thinking they would be the "stewards" of generative AI, and the last US administration was very keen on regoolating everything under the sun, so "safety" was just an angle for regulatory capture. God bless China.

Oh absolutely, AI labs certainly talk their books, including any safety angles. The controversy/outrage extended far beyond those incentivized companies too. Many people had good faith worries about Llama. Open-weight models are now vastly more powerful than Llama-1, yet the sky hasn't fallen. It's just fascinating to me how apocalyptic people are. I just feel lucky to be around in what's likely the most important de…

>Many people had good faith worries about Llama.

ah, but that begs the question: did those people develop their worries organically, or did they simply consume the narrative heavily pushed by virtually every mainstream publication?

the journos are heavily incentivized to spread FUD about it. they saw the writing on the wall that the days of making a living by producing clickbait slop were coming to an end and deluded themselves into thinking that if they kvetch enough, the genie will crawl back into the bottle. scaremongering about sci-fi skynet bullshit didn't work, so now they kvetch about joules and milliliters consumed by chatbots, as if data centers did not exist until two years ago.

likewise, the bulk of other "concerned citizens" are creatives who use their influence to sway their followers, still hoping against hope to kvetch this technology out of existence.

honest-to-God yuddites are as few and as retarded as honest-to-God flat earthers.

Re: Open models by OpenAI

#218

Post by n42 »

siliconc0w wrote: post #210

It seems like OSS will win, I can't see people willing to pay like 10x the price for what seems like 10% more performance. Especially once we get better at routing the hardest questions to the better models and then using that response to augment/fine-tune the OSS ones.

to me it seems like the market is breaking into an 80/20 of B2C/B2B; the B2C use case becoming OSS models (the market shifts to devices that can support them), and the B2B market being priced appropriately for businesses that require that last 20% of absolute cutting edge performance as the cloud offering

Re: Open models by OpenAI

#219

Post by lukax »

Inference in Python uses harmony [1] (for request and response format) which is written in Rust with Python bindings. Another OpenAI's Rust library is tiktoken [2], used for all tokenization and detokenization. OpenAI Codex [3] is also written in Rust. It looks like OpenAI is increasingly adopting Rust (at least for inference).

[1] https://github.com/openai/harmony

[2] https://github.com/openai/tiktoken

[3] https://github.com/openai/codex

Re: Open models by OpenAI

#220

Post by Rhubarrbb »

n42 wrote: post #135
bogtog wrote: post #95

Earlier quoted context omitted.

When people talk about running a (quantized) medium-sized model on a Mac Mini, what types of latency and throughput times are they talking about? Do they mean like 5 tokens per second or at an actually usable speed?

here's a quick recording from the 20b model on my 128GB M4 Max MBP: https://asciinema.org/a/AiLDq7qPvgdAR1JuQhvZScMNr and the 120b: https://asciinema.org/a/B0q8tBl7IcgUorZsphQbbZsMM I am, um, floored

Generation is usually fast, but prompt processing is the main limitation with local agents. I also have a 128 GB M4 Max. How is the prompt processing on long prompts? processing the system prompt for Goose always takes quite a while for me. I haven't been able to download the 120B yet, but I'm looking to switch to either that or the GLM-4.5-Air for my main driver.
Post reply on HN