Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's Foundry leaked pricing says a lot

cognitiverevolution.substack.com

Post reply on HN
Print view View original

91–100 of 214 posts

Re: OpenAI's Foundry leaked pricing says a lot

#91

Post by bombcar »

tshaddox wrote: post #42
version_five wrote: post #29

Earlier quoted context omitted.

I know this isnt the same, but: I've long had it on my list of things to try to train a classifier to predict the answer to multiple choice tests based on embedding of the questions. Many tests I've seen don't require actual intelligence to pass, just a plausible answer relative to the question phrasing

A lot of multiple choice tests are only there to provide some minimum bar of plausible deniability for the examiner. The multiple choice section of a driving test (at least in the two U.S. states I've taken them) is a great example. The questions are almost entirely of the form: Which of these activities is legally permitted while operating an automobile? A. Wearing your seatbelt. B. Being intoxicated. C. Driving 120…

Some questions will be gimmies like that but others will be “which is the default speed for a residential street: 15, 25, 35, 45”

Re: OpenAI's Foundry leaked pricing says a lot

#92

Post by gateorade »

ilaksh wrote: post #7

The Foundry pricing, $78,000 for a few months minimum, is absolutely the opposite of Open. It could completely kill my small startup if the API goes from less than a penny per request to requiring a huge up-front investment. It means that anyone bootstrapping is now locked out.

Why would you start a company that is entirely dependent on a third party’s closed product, over which you have zero control? This just feels immensely risky.

Re: OpenAI's Foundry leaked pricing says a lot

#93

Post by cyanf »

dilyevsky wrote: post #85
paxys wrote: post #84

Earlier quoted context omitted.

And I don't think "hot market" means what they think it means. Yes it is a hot market. Unemployment is at 3.4%, close to historic lows. The tech sector is even tighter. If you ask applicants to first get interviewed by a chat bot they are going to tell you to f off.

I really dont see the reason to be upset here. If someone is asking me to interview with AI I’m going to have my own model take the interview - problem solved

“Permutation City” practically predicted this.

Re: OpenAI's Foundry leaked pricing says a lot

#94

Post by sgk284 »

seydor wrote: post #2

It looks like OpenAI's first mover advantage won't last long, considering the speed at which these models are improving and compacting themselves. Seems like this new 'Moore's law' will be even more steep, at least in the beginning . So steep, that we can hope to be running these in our desktops instead of on someone else's computer.

We saw the same phenomenon with DALL-E. It was state-of-the-art for a blip in time before Midjourney and StableDiffusion surpassed it. And now with ControlNet + the ecosystem of domain-specific models, if you're doing serious generative art, OpenAI isn't even a part of the conversation.

If OpenAI makes their revenue off of charging exorbitant fees to use the LLMs, they'll no longer have incentive to ever open them (even if abuse / misuse concerns are addressed) AND they'll have no incentive to make them more efficient.

OpenAI has yet to show it can have a sustainable advantage. Every other player in the space benefits from an open model ecosystem and efficiency gains.

Re: OpenAI's Foundry leaked pricing says a lot

#95

Post by staticman2 »

dragonwriter wrote: post #22

> This really should not be a surprise, because even the standard-issue ChatGPT can pass the Bar Exam No, it can’t. The two things that together have sometimes gotten misrepresented that way in “game of telephone” presentations are: (1) that when tested on the multiple choice component of the multistate bar exam (not the whole bar exam), it got passing grades in two subjects (evidence and torts), not the whole multip…

It looks like it took the practice bar exam questions, not the "bar exam" which is kept secret to avoid cheating since questions may be reused from year to year, much like the SATS.

Re: OpenAI's Foundry leaked pricing says a lot

#96

Post by adishy »

rnd0 wrote: post #19
gjsman-1000 wrote: post #17

So much for "Open" AI. Open-source algorithm? No. Open access for free, or reasonable prices? No. Open emails and conversations of directors (accountability)? No. Open conversations of issues and ethics? No.

Thank fuck they weren't called "HomelandAI" or "LibertyAI" or somesuch!

I see your subliminal "The Boys" reference :)

Re: OpenAI's Foundry leaked pricing says a lot

#97

Post by dragonwriter »

MichaelZuo wrote: post #56
dragonwriter wrote: post #22

> This really should not be a surprise, because even the standard-issue ChatGPT can pass the Bar Exam No, it can’t. The two things that together have sometimes gotten misrepresented that way in “game of telephone” presentations are: (1) that when tested on the multiple choice component of the multistate bar exam (not the whole bar exam), it got passing grades in two subjects (evidence and torts), not the whole multip…

Well the first paper's authors does say: "While our ability to interpret these results is limited by nascent scientific understanding of LLMs and the proprietary nature of GPT, we believe that these results strongly suggest that an LLM will pass the MBE component of the Bar Exam in the near future."

Yes, so the researchers, based on ChatGPT’s failure, predict that some other LLM in the new future will pass the same subset of the bar exam that ChatGPT failed to pass. Which is nice, but very much not support for the article’s claim that stock ChatGPT can, already, pass the bar exam. There is a pretty big gap between “passing the bar exam” and “giving researchers a feeling of optimism that some other system will pass a particular subset of the bar exam in the ’near future’”. ChatGPT has demonstrated the ability to do the latter, not the former.

Re: OpenAI's Foundry leaked pricing says a lot

#98

Post by dragonwriter »

MichaelZuo wrote: post #67
JohnFen wrote: post #62

Earlier quoted context omitted.

Which is an overt admission that LLMs can't pass the bar exam right now.

ChatGPT 2.0 being able to do so would still be pretty relevant.

And if ChatGPT 2.0 does, than the future article that claims that stock ChatGPT 2.0 can will be justified. But the current article that claims that the stock ChatGPT of today can do that will still be wrong.

Re: OpenAI's Foundry leaked pricing says a lot

#99

Post by antondd »

3throwaway3141 wrote: post #24
ccakes wrote: post #23

> Streamline hiring – in such a hot market, personalizing outreach, assessing resumes, summarizing & flagging profiles, and suggesting interview questions. For companies who have an overabundance of candidates, perhaps even conducting initial interviews? That's a hiring red flag if I've ever seen one. The nightmare dystopia is just around the corner it seems.

Dystopia? I think of it as a great comedy! We could be only months away from the first-ever hire where both the applicant and the company have automated the entire process and may not even realize there's been an accepted offer.

I tend to update my CV on a yearly basis regardless of whether I’m looking for a job or not. This year I used ChatGPT to write up my most recent role. After some minor tweaks it wrote it as good as I would have done - using the same language, tone, etc. I can see this working other way around: if I get a pile of CVs for a role I’m hiring for I’m going to use ChatGPT to summarise candidates’ skills. I’m not sure how well this approach would work, but I’ll give it a go.

Re: OpenAI's Foundry leaked pricing says a lot

#100

Post by dragonwriter »

CamperBob2 wrote: post #88
dragonwriter wrote: post #22

> This really should not be a surprise, because even the standard-issue ChatGPT can pass the Bar Exam No, it can’t. The two things that together have sometimes gotten misrepresented that way in “game of telephone” presentations are: (1) that when tested on the multiple choice component of the multistate bar exam (not the whole bar exam), it got passing grades in two subjects (evidence and torts), not the whole multip…

"This talking dog is a dumbass. It can only pass HALF the bar exam. I don't see what all the hype is about." Not sure that's the indictment you and other ChatGPT detractors are presenting it as.

The claim in the article is that stock GPT can pass the bar exam today. It cannot. Period. That claim is simply false.

It is also false that it “can pass HALF the bar exam.” It passed 2 of 7 sections of the MBE, which is used as the multiple choice part of bar exams (which typically also include essay and practical portions)

Post reply on HN