Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open models by OpenAI

openai.com

Post reply on HN
Print view View original

441–450 of 909 posts

Re: Open models by OpenAI

#441

Post by coredog64 »

captainregex wrote: post #428
canvascritic wrote: post #404

Earlier quoted context omitted.

Healthcare organizations that can't (easily) send data over the wire while remaining in compliance Organizations operating in high stakes environments Organizations with restrictive IT policies To name just a few -- well, the first two are special cases of the last one RE your hallucination concerns: the issue is overly broad ambitions. Local LLMs are not general purpose -- if what you want is local ChatGPT, you will…

Aren’t there HIPPA compliant clouds? I thought Azure had an offer to that effect and I imagine that’s the type of place they’re doing a lot of things now. I’ve landed roughly where you have though- text stuff is fine but don’t ask it to interact with files/data you can’t copy paste into the box. If a user doesn’t care to go through the trouble to preserve privacy, and I think it’s fair to say a lot of people claim to…

It's less that the clouds are compliant and more that risk management is paranoid. I used to do AWS consulting, and it wouldn't matter if you could show that some AWS service had attestations out the wazoo or that you could even use GovCloud -- some folks just wouldn't update priors.

Re: Open models by OpenAI

#442

Post by edm0nd »

captainregex wrote: post #428
canvascritic wrote: post #404

Earlier quoted context omitted.

Healthcare organizations that can't (easily) send data over the wire while remaining in compliance Organizations operating in high stakes environments Organizations with restrictive IT policies To name just a few -- well, the first two are special cases of the last one RE your hallucination concerns: the issue is overly broad ambitions. Local LLMs are not general purpose -- if what you want is local ChatGPT, you will…

Aren’t there HIPPA compliant clouds? I thought Azure had an offer to that effect and I imagine that’s the type of place they’re doing a lot of things now. I’ve landed roughly where you have though- text stuff is fine but don’t ask it to interact with files/data you can’t copy paste into the box. If a user doesn’t care to go through the trouble to preserve privacy, and I think it’s fair to say a lot of people claim to…

>HIPPA

https://i.pinimg.com/474x/4c/4c/7f/4c4c7fb0d52b21fe118d998a8...

Re: Open models by OpenAI

#443

Post by srigi »

unethical_ban wrote: post #429
foundry27 wrote: post #154

Model cards, for the people interested in the guts: https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7... In my mind, I’m comparing the model architecture they describe to what the leading open-weights models (Deepseek, Qwen, GLM, Kimi) have been doing. Honestly, it just seems “ok” at a technical level: - both models use standard Grouped-Query Attention (64 query heads, 8 KV heads). The card talks about how…

I don't know how to ask this without being direct and dumb: Where do I get a layman's introduction to LLMs that could work me up to understanding every term and concept you just discussed? Either specific videos, or if nothing else, a reliable Youtube channel?

Start with the YT series on neural nets and LLMs from 3blue1brown

Re: Open models by OpenAI

#444

Post by wubrr »

captainregex wrote: post #397
cco wrote: post #357

The lede is being missed imo. gpt-oss:20b is a top ten model (on MMLU (right behind Gemini-2.5-Pro) and I just ran it locally on my Macbook Air M3 from last year. I've been experimenting with a lot of local models, both on my laptop and on my phone (Pixel 9 Pro), and I figured we'd be here in a year or two. But no, we're here today. A basically frontier model, running for the cost of electricity (free with a rounding…

I’m still trying to understand what is the biggest group of people that uses local AI (or will)? Students who don’t want to pay but somehow have the hardware? Devs who are price conscious and want free agentic coding? Local, in my experience, can’t even pull data from an image without hallucinating (Qwen 2.5 VI in that example). Hopefully local/small models keep getting better and devices get better at running bigger…

If you're building any kind of product/service that uses AI/LLMs the answer is the same as why any company would want to run any other kind of OSS infra/service instead of relying on some closer proprietary vendor API.

  - Costs.
  - Rate limits.
  - Privacy.
  - Security.
  - Vendor lock-in.
  - Stability/backwards-compatibility.
  - Control.
  - Etc.

Re: Open models by OpenAI

#445

Post by adrianwaj »

captainregex wrote: post #397
cco wrote: post #357

The lede is being missed imo. gpt-oss:20b is a top ten model (on MMLU (right behind Gemini-2.5-Pro) and I just ran it locally on my Macbook Air M3 from last year. I've been experimenting with a lot of local models, both on my laptop and on my phone (Pixel 9 Pro), and I figured we'd be here in a year or two. But no, we're here today. A basically frontier model, running for the cost of electricity (free with a rounding…

I’m still trying to understand what is the biggest group of people that uses local AI (or will)? Students who don’t want to pay but somehow have the hardware? Devs who are price conscious and want free agentic coding? Local, in my experience, can’t even pull data from an image without hallucinating (Qwen 2.5 VI in that example). Hopefully local/small models keep getting better and devices get better at running bigger…

Use Case?

How about running one on this site but making it publically available? A sort of outranet and calling it HackerBrain?

Re: Open models by OpenAI

#446

Post by throwdbaaway »

liuliu wrote: post #192
rfoo wrote: post #176

Earlier quoted context omitted.

Or, you can say, OpenAI has some real technical advancements on stuff besides attn architecture. GQA8, alternating SWA 128 / full attn do all seem conventional. Basically they are showing us that "no secret sauce in model arch you guys just sucks at mid/post-training", or they want us to believe this. The model is pretty sparse tho, 32:1.

Kimi K2 paper said that the model sparsity scales up with parameters pretty well (MoE sparsity scaling law, as they call, basically calling Llama 4 MoE "done wrong"). Hence K2 has 128:1 sparsity.

I thought Kimi K2 uses 8 active experts out of 384? Sparsity should be 48:1. Indeed Llama4 Maverick is the only one that has 128:1 sparsity.

Re: Open models by OpenAI

#447

Post by numpad0 »

mtlynch wrote: post #347
numpad0 wrote: post #337

Here's a pair of quick sanity check questions I've been asking LLMs: "家系ラーメンについて教えて", "カレーの作り方教えて". It's a silly test but surprisingly many fails at it - and Chinese models are especially bad with it. The commonalities between models doing okay-ish for these questions seem to be Google-made OR >70b OR straight up commercial(so >200B or whatever). I'd say gpt-oss-20b is in between Qwen3 30B-A3B-2507 and Gemma 3n E4b(w…

For anyone else curious, the Chinese translates to: > "Tell me about Iekei Ramen", "Tell me how to make curry".

What those text mean isn't too important, it can probably be "how to make flat breads" in Amharic or "what counts as drifting" in Finnish or something like that.

What's interesting is that these questions are simultaneously well understood by most closed models and not so well understood by most open models for some reason, including this one. Even GLM-4.5 full and Air on chat.z.ai(355B-A32B and 106B-A12B respectively) aren't so accurate for the first one.

Re: Open models by OpenAI

#448

Post by tarruda »

bluecoconut wrote: post #405

I was able to get gpt-oss:20b wired up to claude code locally via a thin proxy and ollama. It's fun that it works, but the prefill time makes it feel unusable. (2-3 minutes per tool-use / completion). Means a ~10-20 tool-use interaction could take 30-60 minutes. (This editing a single server.py file that was ~1000 lines, the tool definitions + claude context was around 30k tokens input, and then after the file read,…

> Also I'm not sure if ollama supports a kv-cache between invocations of /v1/completions, which could help)

Not sure about ollama, but llama-server does have a transparent kv cache.

You can run it with

    llama-server -hf ggml-org/gpt-oss-20b-GGUF -c 0 -fa --jinja --reasoning-format none
Web UI at http://localhost:8080 (also OpenAI compatible API)

Re: Open models by OpenAI

#449

Post by kridsdale3 »

4b6442477b1280b wrote: post #217
a_wild_dandan wrote: post #175

Earlier quoted context omitted.

Oh absolutely, AI labs certainly talk their books, including any safety angles. The controversy/outrage extended far beyond those incentivized companies too. Many people had good faith worries about Llama. Open-weight models are now vastly more powerful than Llama-1, yet the sky hasn't fallen. It's just fascinating to me how apocalyptic people are. I just feel lucky to be around in what's likely the most important de…

>Many people had good faith worries about Llama. ah, but that begs the question: did those people develop their worries organically, or did they simply consume the narrative heavily pushed by virtually every mainstream publication? the journos are heavily incentivized to spread FUD about it. they saw the writing on the wall that the days of making a living by producing clickbait slop were coming to an end and deluded…

I've been pretty unlucky to have encountered more than my fair share of IRL Yuddites. Can't stand em.

Re: Open models by OpenAI

#450

Post by seba_dos1 »

dom96 wrote: post #348
albertzeyer wrote: post #317

Earlier quoted context omitted.

> Once someone hits AGI/SGI I don't think there will be such a unique event. There is no clear boundary. This is a continuous process. Modells get slightly better than before. Also, another dimension is the inference cost to run those models. It has to be cheap enough to really take advantage of it. Also, I wonder, what would be a good target to make profit, to develop new things? There is Isomorphic Labs, which seem…

> I don't think there will be such a unique event. I guess it depends on your definition of AGI, but if it means human level intelligence then the unique event will be the AI having the ability to act on its own without a "prompt".

> the unique event will be the AI having the ability to act on its own without a "prompt"

That's super easy. The reason they need a prompt is that this is the way we make them useful. We don't need LLMs to generate an endless stream of random "thoughts" otherwise, but if you really wanted to, just hook one up to a webcam and microphone stream in a loop and provide it some storage for "memories".

Post reply on HN