Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's Foundry leaked pricing says a lot

cognitiverevolution.substack.com

Post reply on HN
Print view View original

111–120 of 214 posts

Re: OpenAI's Foundry leaked pricing says a lot

#111

Post by 0cVlTeIATBs »

klabb3 wrote: post #31
dragonwriter wrote: post #22

> This really should not be a surprise, because even the standard-issue ChatGPT can pass the Bar Exam No, it can’t. The two things that together have sometimes gotten misrepresented that way in “game of telephone” presentations are: (1) that when tested on the multiple choice component of the multistate bar exam (not the whole bar exam), it got passing grades in two subjects (evidence and torts), not the whole multip…

Correct me if I’m wrong but aren’t virtually all tests and exams designed to minimize ambiguity, make them fair or easy to grade and questions are designed to have a clear correct answer? This is a stark difference to most real-world human activity. Add to the fact that LLMs perform much better on questions with a lot of training data. And also add the hallucinations or more generally: they don’t ask for help or admi…

>aren’t virtually all tests and exams designed to minimize ambiguity, make them fair or easy to grade and questions are designed to have a clear correct answer?

Not sure if you mean law exams, but in my experience, engineering exams: yes. Leadership exams: quite the opposite.

Re: OpenAI's Foundry leaked pricing says a lot

#112

Post by esperent »

zamnos wrote: post #104
gateorade wrote: post #92

Earlier quoted context omitted.

Why would you start a company that is entirely dependent on a third party’s closed product, over which you have zero control? This just feels immensely risky.

Realistically, if you start a business, there are tons of such dependencies. If that's too scary for you, you're probably too risk adverse to be an entrepreneur.

The issue is not having dependencies. The issue is having one specific dependency that is a beta experimental product with no viable alternative, and which will tank your company when they raise prices.

Your company must always have alternatives for all third party services. The only exception is open source software that you could switch to hosting yourself if the SAAS company shuts down.

Re: OpenAI's Foundry leaked pricing says a lot

#113

Post by saurik »

meghan_rain wrote: post #70
HDThoreaun wrote: post #39

Earlier quoted context omitted.

Open access for a fee? Yes. The original point of openAI was to make sure google and facebook don't completely dominate AI and keep their work hidden due to the innovator's dilemma they face. Seems like they've absolutely accomplished that. Yes, they've stretched the meaning of "open" but I don't think they've ever tried to claim they had some sort of open source aspirations.

OpenAI = Microsoft, which you have conveniently left out in your FAANG enumeration

Which letter is Microsoft again? (I always forget.)

Re: OpenAI's Foundry leaked pricing says a lot

#114

Post by adt2bt »

HDThoreaun wrote: post #83
meghan_rain wrote: post #79

Earlier quoted context omitted.

Can you elaborate on the last sentence?

Google and Facebook have long had language models that have been reported to be vastly superior to what was publicly available. The one that a Google engineer claimed was sentient was the most well known example. OpenAI was founded in reaction to these models being hidden away.

OpenAI was founded in 2015, well before those models were created.

Re: OpenAI's Foundry leaked pricing says a lot

#115

Post by narrator »

klabb3 wrote: post #31
dragonwriter wrote: post #22

> This really should not be a surprise, because even the standard-issue ChatGPT can pass the Bar Exam No, it can’t. The two things that together have sometimes gotten misrepresented that way in “game of telephone” presentations are: (1) that when tested on the multiple choice component of the multistate bar exam (not the whole bar exam), it got passing grades in two subjects (evidence and torts), not the whole multip…

Correct me if I’m wrong but aren’t virtually all tests and exams designed to minimize ambiguity, make them fair or easy to grade and questions are designed to have a clear correct answer? This is a stark difference to most real-world human activity. Add to the fact that LLMs perform much better on questions with a lot of training data. And also add the hallucinations or more generally: they don’t ask for help or admi…

>aren’t virtually all tests and exams designed to minimize ambiguity, make them fair or easy to grade and questions are designed to have a clear correct answer?

The Bar Exam is designed to be very hard. Almost every single question on it is a trick question of one sort or another.

Re: OpenAI's Foundry leaked pricing says a lot

#117

Post by narrator »

gerdesj wrote: post #78
dragonwriter wrote: post #22

> This really should not be a surprise, because even the standard-issue ChatGPT can pass the Bar Exam No, it can’t. The two things that together have sometimes gotten misrepresented that way in “game of telephone” presentations are: (1) that when tested on the multiple choice component of the multistate bar exam (not the whole bar exam), it got passing grades in two subjects (evidence and torts), not the whole multip…

"No, it can’t." Will it turn up in court - in person? I'm fairly sure that the Bar requires a corpus vivens for its representatives. I've probably got the Latin very badly wrong ...

The other thing is that the legal system is not an API you can spam with until you get a proper response. If you make nonsense arguments in court, or use confabulated law as a lawyer, you will get your law license revoked. The legal system's resources are limited and it doesn't have unlimited time to hear a computer spout clever, but wrong or made up nonsense.

Re: OpenAI's Foundry leaked pricing says a lot

#118

Post by bee_rider »

ncphil wrote: post #87
dragonwriter wrote: post #22

> This really should not be a surprise, because even the standard-issue ChatGPT can pass the Bar Exam No, it can’t. The two things that together have sometimes gotten misrepresented that way in “game of telephone” presentations are: (1) that when tested on the multiple choice component of the multistate bar exam (not the whole bar exam), it got passing grades in two subjects (evidence and torts), not the whole multip…

Well, I _might_ be able to teach an AI how to do effective legal research and writing, or even to do pre and post trial discovery, but representing someone at trial? Not without full immunity from malpractice liability. It would be fun to see how an AI handled cross examination of a hostile witness, or dealt with a response to their hearsay objection like, "res gestae!". Still, a sophisticated AI could be a real help…

I guess eventually a locally-installable ChatGPT-alike will become available. Someone will figure out how to fine-tune it on legal jargon, and then try and use it to represent themselves.

Re: OpenAI's Foundry leaked pricing says a lot

#119

Post by kelseydh »

woeirua wrote: post #64
mc32 wrote: post #25

Earlier quoted context omitted.

According to this survey: https://it.slashdot.org/story/23/02/27/009234/survey-claims-... "Earlier this month, job advice platform Resumebuilder.com surveyed 1,000 business leaders who either use or plan to use ChatGPT. It found that nearly half of their companies have implemented the chatbot. And roughly half of this cohort say ChatGPT has already replaced workers at their companies...." There are anecdotes elsewher…

Yeah, I'm going to need some names here for companies, because there is no way 50% of the Fortune 500 have done _anything_ productive with ChatGPT in less than a month.

It's their employees trying to keep their jobs who have.

Re: OpenAI's Foundry leaked pricing says a lot

#120

Post by axiom92 »

cypress66 wrote: post #81
sanxiyn wrote: post #71

> Attention mechanism inference costs rise with the square of the inference window. This is incorrect. It is correct for the original Transformer, but OpenAI isn't using the original Transformer since GPT-3. Sparse Transformer scales O(N sqrt(N)). https://openai.com/research/sparse-transformer

Gpt3 includes dense attention layers that are n^2

Sure, but we don't know if ChatGPT is based on the original GPT-3 architecture.
Post reply on HN