Open models by OpenAI • Hacker News Board

Re: Open models by OpenAI

#111

Post by jona777than » Tue, Aug 05, 2025, 5:38 PM UTC

thimabi wrote: post #15

coreyh14444 wrote: post #4

Earlier quoted context omitted.

Because GPT-5 comes out later this week?

It could be, but there’s so much hype surrounding the GPT-5 release that I’m not sure whether their internal models will live up to it. For GPT-5 to dwarf these just-released models in importance, it would have to be a huge step forward, and I’m still doubting about OpenAI’s capabilities and infrastructure to handle demand at the moment.

As a sidebar, I’m still not sure if GPT-5 will be transformative due to its capabilities as much as its accessibility. All it really needs to do to be highly impactful is lower the barrier of entry for the more powerful models. I could see that contributing to it being worth the hype. Surely it will be better, but if more people are capable of leveraging it, that’s just as revolutionary, if not more.

Re: Open models by OpenAI

#112

Post by timmg » Tue, Aug 05, 2025, 5:38 PM UTC

Orthogonal, but I just wanted to say how awesome Ollama is. It took 2 seconds to find the model and a minute to download and now I'm using it.

Kudos to that team.

Re: Open models by OpenAI

#113

Post by a_wild_dandan » Tue, Aug 05, 2025, 5:38 PM UTC

asdev wrote: post #67

johntiger1 wrote: post #10

Wow, this will eat Meta's lunch

Meta is so cooked, I think most enterprises will opt for OpenAI or Anthropic and others will host OSS models themselves or on AWS/infra providers.

I'll accept Meta's frontier AI demise if they're in their current position a year from now. People killed Google prematurely too (remember Bard?), because we severely underestimate the catch-up power bought with ungodly piles of cash.

Re: Open models by OpenAI

#114

Post by lcnPylGDnU4H9OF » Tue, Aug 05, 2025, 5:38 PM UTC

Leary wrote: post #28

GPQA Diamond: gpt-oss-120b: 80.1%, Qwen3-235B-A22B-Thinking-2507: 81.1% Humanity’s Last Exam: gpt-oss-120b (tools): 19.0%, gpt-oss-120b (no tools): 14.9%, Qwen3-235B-A22B-Thinking-2507: 18.2%

Was the Qwen model using tools for Humanity's Last Exam?

Re: Open models by OpenAI

#115

Post by dragonwriter » Tue, Aug 05, 2025, 5:38 PM UTC

mythz wrote: post #19

artembugara wrote: post #14

Disclamer: probably dumb questions so, the 20b model. Can someone explain to me what I would need to do in terms of resources (GPU, I assume) if I want to run 20 concurrent processes, assuming I need 1k tokens/second throughput (on each, so 20 x 1k) Also, is this model better/comparable for information extraction compared to gpt-4.1-nano, and would it be cheaper to host myself 20b?

gpt-oss:20b is ~14GB on disk [1] so fits nicely within a 16GB VRAM card. [1] https://ollama.com/library/gpt-oss

You also need space in VRAM for what is required to support the context window; you might be able to do a model that is 14GB in parameters with a small (~8k maybe?) context window on a 16GB card.

Re: Open models by OpenAI

#116

Post by artembugara » Tue, Aug 05, 2025, 5:39 PM UTC

Tostino wrote: post #73

artembugara wrote: post #29

Earlier quoted context omitted.

thanks, this part is clear to me. but I need to understand 20 x 1k token throughput I assume it just might be too early to know the answer

I legitimately cannot think of any hardware that will get you to that throughput over that many streams with any of the hardware I know of (I don't work in the server space so there may be some new stuff I am unaware of).

oh, I totally understand that I'd need multiple GPUs. I'd just want to know what GPU specifically and how many

Re: Open models by OpenAI

#117

Post by PeterStuer » Tue, Aug 05, 2025, 5:39 PM UTC

I love how they frame High-end desktops and laptops as having "a single H100 GPU".

Re: Open models by OpenAI

#118

Post by thimabi » Tue, Aug 05, 2025, 5:40 PM UTC

pamelafox wrote: post #96

Anyone tried running on a Mac M1 with 16GB RAM yet? I've never run higher than an 8GB model, but apparently this one is specifically designed to work well with 16 GB of RAM.

It works fine, although with a bit more latency than non-local models. However, swap usage goes way beyond what I’m comfortable with, so I’ll continue to use smaller models for the foreseeable future.

Hopefully other quantizations of these OpenAI models will be available soon.

Re: Open models by OpenAI

#119

Post by Workaccount2 » Tue, Aug 05, 2025, 5:41 PM UTC

swores wrote: post #100

podnami wrote: post #48

Earlier quoted context omitted.

Wow this was actually blazing fast. I prompted "how can the 45th and 47th presidents of america share the same parents?" On ChatGPT.com o3 thought for for 13 seconds, on OpenRouter GPT OSS 120B thought for 0.7 seconds - and they both had the correct answer.

I'm not sure that's a particularly good question for concluding something positive about the "thought for 0.7 seconds" - it's such a simple answer, ChatGPT 4o (with no thinking time) immediately answered correctly. The only surprising thing in your test is that o3 wasted 13 seconds thinking about it.

A current major outstanding problem with thinking models is how to get them to think an appropriate amount.

Re: Open models by OpenAI

#120

Post by kgwgk » Tue, Aug 05, 2025, 5:41 PM UTC

It may be useless for many use cases given that its policy prevents it for example from providing "advice or instructions about how to buy something."

(I included details about its refusal to answer even after using tools for web searching but hopefully shorter comment means fewer downvotes.)