Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Providing ChatGPT to the U.S. federal workforce

openai.com

Post reply on HN
Print view View original

51–60 of 187 posts

Re: Providing ChatGPT to the U.S. federal workforce

#51

Post by amluto »

janice1999 wrote: post #38
siva7 wrote: post #34

Earlier quoted context omitted.

> access to AI is way cheaper than it will be in the next 5-10 years. That evidently won't be the case as you can see with the recent open model announcements...

Do these model releases really matter to cost if the hardware is still so very expensive and Nvidia still has a defacto monopoly? I can't buy x8 H100s to run a model and whatever company I buy AI access from has to pay for them somehow.

I find it unlikely that the margins on inference hardware will remain anywhere near as high as they are right now.

Inference at scale can be complex, but the complexity is manageable. You can do fancy batched inference, or you can make a single pass over the relevant weights for each inference step. With more models using MoE, the latter is more tractable, and the actual tensor/FMA units that do the bulk of the math are simple enough that any respectable silicon vendor can make them.

Re: Providing ChatGPT to the U.S. federal workforce

#52

Post by willy_k »

janice1999 wrote: post #38
siva7 wrote: post #34

Earlier quoted context omitted.

> access to AI is way cheaper than it will be in the next 5-10 years. That evidently won't be the case as you can see with the recent open model announcements...

Do these model releases really matter to cost if the hardware is still so very expensive and Nvidia still has a defacto monopoly? I can't buy x8 H100s to run a model and whatever company I buy AI access from has to pay for them somehow.

Yes they do, if the model size / vram requirement keeps shrinking for a given performance target, like has been happening, then it gets cheaper to run X level of model.

Re: Providing ChatGPT to the U.S. federal workforce

#53

Post by schmidtleonard »

siva7 wrote: post #37
FergusArgyll wrote: post #8

Ten minutes before Anthropic was gonna do it :) https://www.axios.com/pro/tech-policy/2025/08/05/ai-anthropi...

What's up with these ai companies? Lab A announces major news, B and C follow about one hour later. This is only possible if those follow the same bizarre marketing strategy to wrap up news and advancements in a secure safe until they need to pack it out after competitor made first move.

No, they just pay attention to each other (some combination of reading the lines, reading between the lines, listening to loose lips, maybe even a spy or two) and copycat + frontrun.

The fast follower didn't have the release sitting in a safe so much as they rushed it out the door when prompted, and during the whole development they knew this was a possibility so they kept it able to be rushed out the door. Whatever compromise bullet they bit to make it happen still exists, though.

Re: Providing ChatGPT to the U.S. federal workforce

#55

Post by siva7 »

janice1999 wrote: post #38
siva7 wrote: post #34

Earlier quoted context omitted.

> access to AI is way cheaper than it will be in the next 5-10 years. That evidently won't be the case as you can see with the recent open model announcements...

Do these model releases really matter to cost if the hardware is still so very expensive and Nvidia still has a defacto monopoly? I can't buy x8 H100s to run a model and whatever company I buy AI access from has to pay for them somehow.

The news is that this won't be necessarily for the majority of private and workforce. They run on your own machine.

Re: Providing ChatGPT to the U.S. federal workforce

#56

Post by claytonjy »

fzzzy wrote: post #49
janice1999 wrote: post #38

Earlier quoted context omitted.

Do these model releases really matter to cost if the hardware is still so very expensive and Nvidia still has a defacto monopoly? I can't buy x8 H100s to run a model and whatever company I buy AI access from has to pay for them somehow.

You only need 64 gb of cpu ram to run gpt-oss, or one h100.

you can’t really buy H100s except in multiples of 8. If you want fewer, you must rent. Even then, hyperscalers tend to be a bit inflexible there; GCP only recently added support for smaller shapes, and they can’t yet be reserved, only on-demand or spot iirc.

Re: Providing ChatGPT to the U.S. federal workforce

#57

Post by bix6 »

thallium205 wrote: post #31
Isamu wrote: post #18

Don’t they mean to say “replacing the entire U.S. federal workforce with ChatGPT”? Surely that is the future everyone is looking to.

I'd rather interact with an AI than federal workers 80% of the time.

Absolutely not. Fed workers are epic. Get out of here with that nonsense.

Re: Providing ChatGPT to the U.S. federal workforce

#58

Post by [deleted] »

schmidtleonard wrote: post #53
siva7 wrote: post #37

Earlier quoted context omitted.

What's up with these ai companies? Lab A announces major news, B and C follow about one hour later. This is only possible if those follow the same bizarre marketing strategy to wrap up news and advancements in a secure safe until they need to pack it out after competitor made first move.

No, they just pay attention to each other (some combination of reading the lines, reading between the lines, listening to loose lips, maybe even a spy or two) and copycat + frontrun. The fast follower didn't have the release sitting in a safe so much as they rushed it out the door when prompted, and during the whole development they knew this was a possibility so they kept it able to be rushed out the door. Whatever…

[deleted]

Re: Providing ChatGPT to the U.S. federal workforce

#59

Post by janice1999 »

fzzzy wrote: post #49
janice1999 wrote: post #38

Earlier quoted context omitted.

Do these model releases really matter to cost if the hardware is still so very expensive and Nvidia still has a defacto monopoly? I can't buy x8 H100s to run a model and whatever company I buy AI access from has to pay for them somehow.

You only need 64 gb of cpu ram to run gpt-oss, or one h100.

I assume you're talking that's a quantised 20B model on a several thousand dollar Mac? That's really impressive and huge progress but is that indicative of companies serving thousands of users? They still have to buy Nvidia at the end of the day.
Post reply on HN