Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open models by OpenAI

openai.com

Post reply on HN
Print view View original

531–540 of 909 posts

Re: Open models by OpenAI

#531

Post by andai »

modeless wrote: post #231
sadiq wrote: post #41

Looks like Groq (at 1k+ tokens/second) and Fireworks are already live on openrouter: https://openrouter.ai/openai/gpt-oss-120b $0.15M in / $0.6-0.75M out edit: Now Cerebras too at 3,815 tps for $0.25M / $0.69M out.

I really want to try coding with this at 2600 tokens/s (from Cerebras). Imagine generating thousands of lines of code as fast as you can prompt. If it doesn't work who cares, generate another thousand and try again! And at $.69/M tokens it would only cost $6.50 an hour.

I tried this (gpt-oss-120b with Cerebras) with Roo Code. It repeatedly failed to use the tools correctly, and then I got 429 too many requests. So much for the "as fast as I can think" idea!

I'll have to try again later but it was a bit underwhelming.

The latency also seemed pretty high, not sure why. I think with the latency the throughout ends up not making much difference.

Btw Groq has the 20b model at 4000 TPS but I haven't tried that one.

Re: Open models by OpenAI

#532

Post by [deleted] »

M4R5H4LL wrote: post #437
wizee wrote: post #417

Earlier quoted context omitted.

Privacy, both personal and for corporate data protection is a major reason. Unlimited usage, allowing offline use, supporting open source, not worrying about a good model being taken down/discontinued or changed, and the freedom to use uncensored models or model fine tunes are other benefits (though this OpenAI model is super-censored - “safe”). I don’t have much experience with local vision models, but for text ques…

+1 - I work in finance, and there's no way we're sending our data and code outside the organization. We have our own H100s.

[deleted]

Re: Open models by OpenAI

#533

Post by nojito »

pxc wrote: post #506
nojito wrote: post #494

Earlier quoted context omitted.

Why does it need knowledge when it can just call tools to get it?

Right... knowledge is one of the things (the one thing?) that LLMs are really horrible at, and that goes double for models small enough to run on normal-ish consumer hardware. Shouldn't we prefer to have LLMs just search and summarize more reliable sources?

Yup which is why these models are so exciting!

They are specifically training on webbrowsing and python calling.

Re: Open models by OpenAI

#534

Post by iJohnDoe »

pxc wrote: post #510
teitoklien wrote: post #408

Earlier quoted context omitted.

for now, i wouldnt rank any model from openai in coding benchmarks, despite all the false messaging they are giving, almost every single model openai has launched even the high end o3 expensive models are absolutely monumentally horrible at coding tasks. So this is expected. If its decent in other tasks, which i do find openai often being better than others at, then i think its a win, especially a win for the open so…

NVIDIA will probably give us nice, coding-focused fine-tunes of these models at some point, and those might compare more favorably against the smaller Qwen3 Coder.

What is the best local coder model that that can be used with ollama?

Maybe a too opened ended question? I can run the deepseek model locally really nicely.

Re: Open models by OpenAI

#535

Post by nojito »

canvascritic wrote: post #404
captainregex wrote: post #397

Earlier quoted context omitted.

I’m still trying to understand what is the biggest group of people that uses local AI (or will)? Students who don’t want to pay but somehow have the hardware? Devs who are price conscious and want free agentic coding? Local, in my experience, can’t even pull data from an image without hallucinating (Qwen 2.5 VI in that example). Hopefully local/small models keep getting better and devices get better at running bigger…

Healthcare organizations that can't (easily) send data over the wire while remaining in compliance Organizations operating in high stakes environments Organizations with restrictive IT policies To name just a few -- well, the first two are special cases of the last one RE your hallucination concerns: the issue is overly broad ambitions. Local LLMs are not general purpose -- if what you want is local ChatGPT, you will…

Pretty much all the large players in healthcare (provider and payer) have model access (OpenAI, Gemini, Anthropic)

Re: Open models by OpenAI

#536

Post by teleforce »

Kudos OpenAI on releasing their open models, is now moving in the direction if only based on their prefix "Open" name alone.

For those who're wondering what are the real benefits, it's the main fact that you can run your LLM locally is awesome without resorting to expensive and inefficient cloud based superpower.

Run the model against your very own documents with RAG, it can provide excellent context engineering for your LLM prompts with reliable citations and much less hallucinations especially for self learning purposes [1].

Beyond Intel - NVIDIA desktop/laptop duopoly 96 GB of (V)RAM MacBook with UMA and the new high end AMD Strix laptop with similar setup of 96 GB of (V)RAM from the 128 GB RAM [2]. The osd-gpt-120b is made for this particular setup.

[1] AI-driven chat assistant for ECE 120 course at UIUC:

https://uiuc.chat/ece120/chat

[2] HP ZBook Ultra G1a Review: Strix Halo Power in a Sleek Workstation:

https://www.bestlaptop.deals/articles/hp-zbook-ultra-g1a-rev...

Re: Open models by OpenAI

#537

Post by deadbabe »

captainregex wrote: post #397
cco wrote: post #357

The lede is being missed imo. gpt-oss:20b is a top ten model (on MMLU (right behind Gemini-2.5-Pro) and I just ran it locally on my Macbook Air M3 from last year. I've been experimenting with a lot of local models, both on my laptop and on my phone (Pixel 9 Pro), and I figured we'd be here in a year or two. But no, we're here today. A basically frontier model, running for the cost of electricity (free with a rounding…

I’m still trying to understand what is the biggest group of people that uses local AI (or will)? Students who don’t want to pay but somehow have the hardware? Devs who are price conscious and want free agentic coding? Local, in my experience, can’t even pull data from an image without hallucinating (Qwen 2.5 VI in that example). Hopefully local/small models keep getting better and devices get better at running bigger…

We use it locally for deep packet inspection.

Re: Open models by OpenAI

#538

Post by cyanydeez »

captainregex wrote: post #397
cco wrote: post #357

The lede is being missed imo. gpt-oss:20b is a top ten model (on MMLU (right behind Gemini-2.5-Pro) and I just ran it locally on my Macbook Air M3 from last year. I've been experimenting with a lot of local models, both on my laptop and on my phone (Pixel 9 Pro), and I figured we'd be here in a year or two. But no, we're here today. A basically frontier model, running for the cost of electricity (free with a rounding…

I’m still trying to understand what is the biggest group of people that uses local AI (or will)? Students who don’t want to pay but somehow have the hardware? Devs who are price conscious and want free agentic coding? Local, in my experience, can’t even pull data from an image without hallucinating (Qwen 2.5 VI in that example). Hopefully local/small models keep getting better and devices get better at running bigger…

People who want programmatic solutions that wont be rug pulled

Re: Open models by OpenAI

#540

Post by littlestymaar »

mathiaspoint wrote: post #389
cco wrote: post #357

The lede is being missed imo. gpt-oss:20b is a top ten model (on MMLU (right behind Gemini-2.5-Pro) and I just ran it locally on my Macbook Air M3 from last year. I've been experimenting with a lot of local models, both on my laptop and on my phone (Pixel 9 Pro), and I figured we'd be here in a year or two. But no, we're here today. A basically frontier model, running for the cost of electricity (free with a rounding…

It's really training not inference that drains the lakes.

Training cost has increased a ton exactly because inference cost is the biggest problem: models are now trained on almost three orders of magnitude more data then what is compute-optimal to do (from the Chinchilla paper), because saving compute on inference makes it valuable to overtrain a smaller model to achieve similar performance for a bigger amount of training compute.
Post reply on HN