Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's Foundry leaked pricing says a lot

cognitiverevolution.substack.com

Post reply on HN
Print view View original

181–190 of 214 posts

Re: OpenAI's Foundry leaked pricing says a lot

#181

Post by selestify »

eyelidlessness wrote: post #144
blagie wrote: post #107

Earlier quoted context omitted.

> And we’re about to plug them straight into critical business flows? Anyone who thinks this is a problem has never managed flesh-and-blood employees, and especially not minimum wage ones. LLMs don't need to be perfect. The bar they need to meet for a lot of work is just not very high. We're also about a year or two into LLMs, and their capabilities are still increasing rapidly. We don't know where their ceiling is.…

> Anyone who thinks this is a problem has never managed flesh-and-blood employees, and especially not minimum wage ones. I’ve worked with minimum wage employees. I’ve even worked with people who insist against all evidence and questioning that basic facts about reality like the year are contrary to known reality. I’ve never in my now very aging career worked with any such person who can influence billions of people b…

They literally didn't know what the current year was? Were they high and/or mentally disabled?

Re: OpenAI's Foundry leaked pricing says a lot

#182

Post by catach »

l33t233372 wrote: post #132
narrator wrote: post #115

Earlier quoted context omitted.

>aren’t virtually all tests and exams designed to minimize ambiguity, make them fair or easy to grade and questions are designed to have a clear correct answer? The Bar Exam is designed to be very hard. Almost every single question on it is a trick question of one sort or another.

Hard is different from ambiguous.

"Trick question" may include or be adjacent to "ambiguous" depending on methods.

Re: OpenAI's Foundry leaked pricing says a lot

#183

Post by nl »

l33t233372 wrote: post #136
nl wrote: post #55

Earlier quoted context omitted.

> GPT's model architecture was invented at Google anyway No it wasn't. Transformers were invented at Google, but "architecture" when talking about neural networks means how they are arranged (and to some extent the training objective function) rather than the building blocks used.

I’m not sure what you mean. The words “transformer architecture” are standard across the literature.

That's like saying "LSTM architecture". It says it uses a transformer but no description of how it is used.

For example, the GPT architecture (which GPT 2 & 3 are slight modifications of) comprises of an embedding layer followed by 12x(self-attention/layer norm/feed forward/layer norm. That's what that GPT "transformer architecture" is, not just the transformer block itself.

Strictly the architecture really also includes things like the embedding size and number of heads (which are in the GPT paper).

Re: OpenAI's Foundry leaked pricing says a lot

#184

Post by dalbasal »

daveguy wrote: post #122
MagicMoonlight wrote: post #48

Earlier quoted context omitted.

That's a good point. I could pay OpenAI for a corporate account and then get it to apply for a hundred remote jobs. Then plug it into slack etc and let it work those jobs while I collect the money.

Pretty sure that would fit the legal definition of fraud.

Idk but that might be one hell of an interesting trial.

Remote work opens up a big old gray area between slacking off and fraud. A judge would be deciding to take this from labour domain to another one.

Also, there's the prevelant lie that companies know what their employees do, how well and how muchnof it they do. CEOs have executives assure them of this. Executives have managers assure them of this. Boards require it, and legalistic systems also assume everyone has this.

That said, "he tricked us by using ai" will probably be easier to make than "he tricked us by being really lazy."

Re: OpenAI's Foundry leaked pricing says a lot

#185

Post by nl »

maleldil wrote: post #173
nl wrote: post #57

Earlier quoted context omitted.

> Open-source algorithm? No. The publish their algorithms just fine and no one has had issues replicating their work.

Where's the ChatGPT paper?

From the ChatGTP announcement: "ChatGTP is a sibling model to Instruct GPT"

The paper for that is linked from https://openai.com/research/instruction-following

Re: OpenAI's Foundry leaked pricing says a lot

#186

Post by twawaaay »

dragonwriter wrote: post #22

> This really should not be a surprise, because even the standard-issue ChatGPT can pass the Bar Exam No, it can’t. The two things that together have sometimes gotten misrepresented that way in “game of telephone” presentations are: (1) that when tested on the multiple choice component of the multistate bar exam (not the whole bar exam), it got passing grades in two subjects (evidence and torts), not the whole multip…

I think people are missing an important nuance when it comes to this "AI passed X exam" series of stories.

Can you say a bookshelf or a search engine passes a bar exam because you can ask it any question and you can find an answer there? Does a natural language interface to said bookshelf/search engine make the difference?

Storing and retrieving facts is not enough to be a lawyer.

Examination systems are built with an assumption an already intelligent person is taking it and verifying that this already intelligent person also learned knowledge necessary to do their job.

So even if AI could technically pass the bar exam it does not mean it is good enough to be a lawyer. It is not a general intelligence that can solve a variety of problems, it was just trained to remember a library of fats that a lawyer may need to know but is not enough to make a lawyer from a non-sentient program.

Re: OpenAI's Foundry leaked pricing says a lot

#187

Post by berniedurfee »

As CEO of Acme Mousetraps Inc, does this mean I can pay OpenAI $1.2M/yr to develop a mousetrap-aware model that can augment (replace?) my R&D team?

Will I be able to ask this custom model to describe a new, novel (patentable) mousetrap that’s better and cheaper to build than anything ever before?

Or will I at least be able to pay OpenAI to ensure Bing suggests using Acme mousetraps (clearly the best) when they ask what mousetraps they should buy?

Re: OpenAI's Foundry leaked pricing says a lot

#188

Post by sarabande »

twawaaay wrote: post #186
dragonwriter wrote: post #22

> This really should not be a surprise, because even the standard-issue ChatGPT can pass the Bar Exam No, it can’t. The two things that together have sometimes gotten misrepresented that way in “game of telephone” presentations are: (1) that when tested on the multiple choice component of the multistate bar exam (not the whole bar exam), it got passing grades in two subjects (evidence and torts), not the whole multip…

I think people are missing an important nuance when it comes to this "AI passed X exam" series of stories. Can you say a bookshelf or a search engine passes a bar exam because you can ask it any question and you can find an answer there? Does a natural language interface to said bookshelf/search engine make the difference? Storing and retrieving facts is not enough to be a lawyer. Examination systems are built with a…

I don't know about in the US, but I suspect it is similar to Germany, where the bar exam also requires you to argue/reason about/solve novel cases, i.e. not regurgitation of stored information. The AI likely wouldn't pass that.

Re: OpenAI's Foundry leaked pricing says a lot

#189

Post by srcreigh »

FeepingCreature wrote: post #135
l33t233372 wrote: post #131

Earlier quoted context omitted.

We are not “a year or two into LLMs”. To name one example, BERT is more than 5 years old.

Is 110 million parameters really a "large" language model though? Especially since the models gain novel skills as they scale up.

So if a new model is trained with 10x parameters of gpt3, is gpt now no longer an llm?
Post reply on HN