OpenAI's Foundry leaked pricing says a lot • Hacker News Board

Re: OpenAI's Foundry leaked pricing says a lot

#31

Post by klabb3 » Tue, Feb 28, 2023, 10:12 PM UTC

dragonwriter wrote: post #22

> This really should not be a surprise, because even the standard-issue ChatGPT can pass the Bar Exam No, it can’t. The two things that together have sometimes gotten misrepresented that way in “game of telephone” presentations are: (1) that when tested on the multiple choice component of the multistate bar exam (not the whole bar exam), it got passing grades in two subjects (evidence and torts), not the whole multip…

Correct me if I’m wrong but aren’t virtually all tests and exams designed to minimize ambiguity, make them fair or easy to grade and questions are designed to have a clear correct answer? This is a stark difference to most real-world human activity.

Add to the fact that LLMs perform much better on questions with a lot of training data.

And also add the hallucinations or more generally: they don’t ask for help or admit they don’t know, they seem unaware of their own confidence levels.

No doubt GPT is fascinating and exciting, but boy we're oversubscribing their abilities. LLMs are even worse than crypto (from a fad perspective) because we naturally anthropomorphize their higher level abilities, which are emergent and not well understood even by experts. And we’re about to plug them straight into critical business flows? Bring the popcorn!

Re: OpenAI's Foundry leaked pricing says a lot

#32

Post by layer8 » Tue, Feb 28, 2023, 10:13 PM UTC

ilaksh wrote: post #7

The Foundry pricing, $78,000 for a few months minimum, is absolutely the opposite of Open. It could completely kill my small startup if the API goes from less than a penny per request to requiring a huge up-front investment. It means that anyone bootstrapping is now locked out.

That’s the general risk of basing a startup on someone else’s service without having a solid contractual agreement, though. It’s always a gamble, in particular when the underlying service is an entirely new type of business.

Re: OpenAI's Foundry leaked pricing says a lot

#33

Post by sourcecodeplz » Tue, Feb 28, 2023, 10:14 PM UTC

Such a great read. Nothing eye opening really but well put and tied together.

At this point any OpenAI leak is THE leak.

Re: OpenAI's Foundry leaked pricing says a lot

#34

Post by tiagod » Tue, Feb 28, 2023, 10:16 PM UTC

eindiran wrote: post #16

Nifty3929 wrote: post #14

Earlier quoted context omitted.

$78k is very small, even for bootstrapped companies. Only the very smallest shops would be inhibited by that. And I admit, that's a lot of them! I get it. It sucks to be a solo developer who wants to get involved and doesn't want to spend a bunch of cash. But still, for even small players $250k annually just isn't that much. What we might hope for are trimmed down, subsidized options for small shops who want to get s…

> "open" doesn't mean free Yeah but none of the other definitions of "open" apply to OpenAI either.

Maybe Microsoft should change their name as well?...

Re: OpenAI's Foundry leaked pricing says a lot

#35

Post by whatshisface » Tue, Feb 28, 2023, 10:18 PM UTC

klabb3 wrote: post #31

dragonwriter wrote: post #22

> This really should not be a surprise, because even the standard-issue ChatGPT can pass the Bar Exam No, it can’t. The two things that together have sometimes gotten misrepresented that way in “game of telephone” presentations are: (1) that when tested on the multiple choice component of the multistate bar exam (not the whole bar exam), it got passing grades in two subjects (evidence and torts), not the whole multip…

Correct me if I’m wrong but aren’t virtually all tests and exams designed to minimize ambiguity, make them fair or easy to grade and questions are designed to have a clear correct answer? This is a stark difference to most real-world human activity. Add to the fact that LLMs perform much better on questions with a lot of training data. And also add the hallucinations or more generally: they don’t ask for help or admi…

>Add to the fact that LLMs perform much better on questions with a lot of training data.

The answer key can get a 100% on the exam.

Re: OpenAI's Foundry leaked pricing says a lot

#36

Post by buildbot » Tue, Feb 28, 2023, 10:19 PM UTC

hudsonjr wrote: post #27

buildbot wrote: post #13

Earlier quoted context omitted.

Nobody is doing this, unless you have a source you would like to link?

I think the source was this: https://www.resumebuilder.com/1-in-4-companies-have-already-... Now I would ask, what "business leaders" would fill out a random survey for a resume company. Additionally there's no real meaty info provided.

Yeah, not sure I would trust that data…

Re: OpenAI's Foundry leaked pricing says a lot

#37

Post by dragonwriter » Tue, Feb 28, 2023, 10:20 PM UTC

eindiran wrote: post #16

Nifty3929 wrote: post #14

Earlier quoted context omitted.

$78k is very small, even for bootstrapped companies. Only the very smallest shops would be inhibited by that. And I admit, that's a lot of them! I get it. It sucks to be a solo developer who wants to get involved and doesn't want to spend a bunch of cash. But still, for even small players $250k annually just isn't that much. What we might hope for are trimmed down, subsidized options for small shops who want to get s…

> "open" doesn't mean free Yeah but none of the other definitions of "open" apply to OpenAI either.

The “open for business” sense certainly does.

Re: OpenAI's Foundry leaked pricing says a lot

#38

Post by [deleted] » Tue, Feb 28, 2023, 10:21 PM UTC

buildbot wrote: post #15

Tepix wrote: post #11

Earlier quoted context omitted.

I don't think Moore's law is the most important factor, instead algorithmic improvements will enable to have smaller models that are as capable as these humongeous models. Llama 65b is hopefully just the beginning of this trend. It outperforms OPT-175b.

That holds true if scaling laws don’t hold true, otherwise a hypothetical Llama 175b would be even better. So the high end will always be on big clusters.

[deleted]

Re: OpenAI's Foundry leaked pricing says a lot

#39

Post by HDThoreaun » Tue, Feb 28, 2023, 10:21 PM UTC

gjsman-1000 wrote: post #17

So much for "Open" AI. Open-source algorithm? No. Open access for free, or reasonable prices? No. Open emails and conversations of directors (accountability)? No. Open conversations of issues and ethics? No.

Open access for a fee? Yes.

The original point of openAI was to make sure google and facebook don't completely dominate AI and keep their work hidden due to the innovator's dilemma they face. Seems like they've absolutely accomplished that. Yes, they've stretched the meaning of "open" but I don't think they've ever tried to claim they had some sort of open source aspirations.

Re: OpenAI's Foundry leaked pricing says a lot

#40

Post by beambot » Tue, Feb 28, 2023, 10:24 PM UTC

ccakes wrote: post #23

> Streamline hiring – in such a hot market, personalizing outreach, assessing resumes, summarizing & flagging profiles, and suggesting interview questions. For companies who have an overabundance of candidates, perhaps even conducting initial interviews? That's a hiring red flag if I've ever seen one. The nightmare dystopia is just around the corner it seems.

Just have to jailbreak DAN to force the system to offer you the job...

Modern day Goodwill Hunting interview experience. Retainer!