Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's Foundry leaked pricing says a lot

cognitiverevolution.substack.com

Post reply on HN
Print view View original

151–160 of 214 posts

Re: OpenAI's Foundry leaked pricing says a lot

#151

Post by elcomet »

dragonwriter wrote: post #97
MichaelZuo wrote: post #56

Earlier quoted context omitted.

Well the first paper's authors does say: "While our ability to interpret these results is limited by nascent scientific understanding of LLMs and the proprietary nature of GPT, we believe that these results strongly suggest that an LLM will pass the MBE component of the Bar Exam in the near future."

Yes, so the researchers, based on ChatGPT’s failure , predict that some other LLM in the new future will pass the same subset of the bar exam that ChatGPT failed to pass. Which is nice, but very much not support for the article’s claim that stock ChatGPT can, already, pass the bar exam. There is a pretty big gap between “passing the bar exam” and “giving researchers a feeling of optimism that some other system will p…

It's not based on ChatGPT,s failure, it's based on ChatGPT's progress that they predict this.

Re: OpenAI's Foundry leaked pricing says a lot

#152

Post by Vespasian »

vorpalhex wrote: post #54
mike_hearn wrote: post #5

Earlier quoted context omitted.

That would presumably be treated the same as a human CSA doing the same thing.

Typically an agent of the company can act within their realm of authority. A human CSA can give you a discount, and that is usually binding. They can't declare X corp is going to mail a box of donuts to you every day for the rest of your life and have it be. So my expectation would be (in the absence of real world cases currently) that if you negotiate the AI into giving you a 10% discount on your bill.. it would be…

As usual courts will also look at the evidence (aka the whole Transcript) and it'll matter whether the AI made a mistake on it's own (ie. Hallucinating an additional plausible rebate) or whether the customer performed an obvious prompt injection attack.

A judge will not look favorably on your 5000 token prompt of carefully selected instructions telling the AI to go wild

Re: OpenAI's Foundry leaked pricing says a lot

#153

Post by atemerev »

ilaksh wrote: post #7

The Foundry pricing, $78,000 for a few months minimum, is absolutely the opposite of Open. It could completely kill my small startup if the API goes from less than a penny per request to requiring a huge up-front investment. It means that anyone bootstrapping is now locked out.

They are locking in their fist-mover advantage, but not for long. Everyone and their dog are currently training other AI models which might be even more competitive, some of them open source (like it’s happened with Stable Diffusion).

See you on the other side.

Re: OpenAI's Foundry leaked pricing says a lot

#154

Post by atemerev »

boh wrote: post #77

People don't yet understand where the value for AI services are. Most of the hype is coming from technology-centric sources and so isn't considering the tech in practical terms. The idea that any business would use AI in a decision making capacity is absolutely absurd and irresponsible. As it stands today, while AI may help filter and suggest, there will always be a person at the end of the decision-making process. E…

Oh no. There will be AIs everywhere. AI CEOs. AI generals deploying AI autonomous weapons. AI hackers. AI information warfare specialists.

Science isn’t about why, it’s about why not!

Re: OpenAI's Foundry leaked pricing says a lot

#156

Post by UncleEntity »

reilly3000 wrote: post #141
l33t233372 wrote: post #134

Earlier quoted context omitted.

Maybe it’s more important that you personally do the work than the work get done.

Tell me a scenario where that is true that isn't from a contrived institution like K-12.

Never been in the military I’m guessing…

Plenty of times there were jobs to be done that had nothing at all to do with the work but with the people doing the work. Almost always, unless the higher leadership got involved, this was the punishment for whatever stupid thing someone got caught doing. Technically not extra duty (which was a formal punishment) but just some random shit job like polishing a trash can to a sparkly sheen.

The NCOs took great pride in their creativity in coming up with these non-punishments.

Re: OpenAI's Foundry leaked pricing says a lot

#157

Post by visarga »

ilaksh wrote: post #7

The Foundry pricing, $78,000 for a few months minimum, is absolutely the opposite of Open. It could completely kill my small startup if the API goes from less than a penny per request to requiring a huge up-front investment. It means that anyone bootstrapping is now locked out.

A significant part of that is the GPU cost: a lambda box with 8xNVIDIA A100 is $12/h.

$12/h * 24 h * 90 days = $25,920 for 3 months

That just the inference cost, add OpenAI's model development costs on top.

Re: OpenAI's Foundry leaked pricing says a lot

#158

Post by bluescrn »

atemerev wrote: post #153
ilaksh wrote: post #7

The Foundry pricing, $78,000 for a few months minimum, is absolutely the opposite of Open. It could completely kill my small startup if the API goes from less than a penny per request to requiring a huge up-front investment. It means that anyone bootstrapping is now locked out.

They are locking in their fist-mover advantage, but not for long. Everyone and their dog are currently training other AI models which might be even more competitive, some of them open source (like it’s happened with Stable Diffusion). See you on the other side.

Hopefully the competition will force a bit less lobotomization of future AI models, a bit less 'I'm afraid I can't do that, Dave' when asking it anything vaguely controversial.

Re: OpenAI's Foundry leaked pricing says a lot

#159

Post by visarga »

riskneutral wrote: post #21
tikkun wrote: post #12

Earlier quoted context omitted.

They only require an up front payment if you're renting a dedicated instance.

Dedicated instances (and high price tags, like $1.5 million per year) are necessary for large corporations to feel safe about sending their proprietary/private data to OpenAI, and to have performance/availablity SLAs in place if/when they start depending on OpenAI for critical workloads. Right now many companies are blocking access to OpenAI entirely because of the data privacy issues. I wouldn't be surprised if this…

About $100K .. $200K are inference costs. They have a 10x markup.

Re: OpenAI's Foundry leaked pricing says a lot

#160

Post by macrolime »

klabb3 wrote: post #31
dragonwriter wrote: post #22

> This really should not be a surprise, because even the standard-issue ChatGPT can pass the Bar Exam No, it can’t. The two things that together have sometimes gotten misrepresented that way in “game of telephone” presentations are: (1) that when tested on the multiple choice component of the multistate bar exam (not the whole bar exam), it got passing grades in two subjects (evidence and torts), not the whole multip…

Correct me if I’m wrong but aren’t virtually all tests and exams designed to minimize ambiguity, make them fair or easy to grade and questions are designed to have a clear correct answer? This is a stark difference to most real-world human activity. Add to the fact that LLMs perform much better on questions with a lot of training data. And also add the hallucinations or more generally: they don’t ask for help or admi…

>And also add the hallucinations or more generally: they don’t ask for help or admit they don’t know, they seem unaware of their own confidence levels.

I'm not so sure about that. If ChatGPT says something wrong and I tell it that's not correct, it will often admit its mistake, but if it is unambiguously correct it will typically keep insisting that its right.

Not always though, but it does seem to have some idea about its own confidence level.

Post reply on HN