Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's plans according to sama

humanloop.com

Post reply on HN
Print view View original

151–160 of 269 posts

Re: OpenAI's plans according to sama

#151

Post by tikwidd »

twobitshifter wrote: post #136

Left the best part until the end. Scaling models larger is still paying off for openai. It’s not AGI yet, but how much bigger will a model need to get to max out? >The scaling hypothesis is the idea that we may have most of the pieces in place needed to build AGI and that most of the remaining work will be taking existing methods and scaling them up to larger models and bigger datasets. If the era of scaling was over…

After training my physics simulator on thousands of hours of video footage of trees moving in the wind, arborists tell me the trees are much more realistic (they are getting worried that I might put them out of business). But the physicists are still not satisfied. How many more videos do I need to generate the laws of motion?

Re: OpenAI's plans according to sama

#152

Post by ShamelessC »

atemerev wrote: post #10

All AI companies (OpenAI included) are now working full tilt on making AIs improve themselves (writing their own code, inventing new pipelines etc). I don't know why choose anything else to work on. This is a prime directive, that will bring the greatest payoff.

Well that is demonstrably untrue.

Re: OpenAI's plans according to sama

#153

Post by solarkraft »

razcle wrote: post #120
sharkjacobs wrote: post #24

> He reiterated his belief in the importance of open source and said that OpenAI was considering open-sourcing GPT-3. Part of the reason they hadn’t open-sourced yet was that he was skeptical of how many individuals and companies would have the capability to host and serve large LLMs. Am I reading this right? "We're not open sourcing GPT-3 because we don't think it would be useful to anyone else"

I think I worded this poorly. What he said was that a lot of people say they want open-source models but they underestimate how hard it is to serve them well. So he wondered how much real benefit would come from open-sourcing them. I think this is reasonable. Giving researchers access is great but for most small companies they're likely better off having a service provider manage inference for them rather than naviga…

You're saying the same thing.

"I'm not sharing my chocolate with you because you probably wouldn't like it"

Re: OpenAI's plans according to sama

#154

Post by legendofbrando »

brucethemoose2 wrote: post #118
purplecats wrote: post #32

> Cheaper and faster GPT-4 — This is their top priority. In general, OpenAI’s aim is to drive “the cost of intelligence” down as far as possible and so they will work hard to continue to reduce the cost of the APIs over time. this certainly aligns with the massive (albeit subjective and anecdotal) degradation in quality i've experienced with ChatGPT GPT-4 over the past few weeks. hopefully a superior (higher quality)…

Anthropic's Claude is said to be very good. Instruction tuned LLaMA 65B/Falcon 40B are good, especially with an embeddings database. ...But OpenAI has all the name recognition and ease of use now, so it might not even matter if others ambiguously surpass OpenAI models.

The problem with Claude is that it is quite literally impossible to get off the waiting list to use it. To OpenAI’s credit they actually ship the product in an accessible way to developers.

Re: OpenAI's plans according to sama

#155

Post by floydnoel »

alchemist1e9 wrote: post #84
sashank_1509 wrote: post #48

Really great news to give at cheaper and faster GPT4. As a GPT+ subscriber, the most annoying thing is the 25 message limit every 3 hours, I really want that removed. A bit sad to hear that the multimodal model will only come next year, was hoping to get it this year 100k to 1 Million context length, sounds phenomenal especially if it comes to GPT4. I've used Claudes 100k context length and I found it so useful that…

Did you have any tips on how you got access to Claude? I do the request access but never get any email or any contact.

I use Poe and got access to Claude 100k as soon as it was released. I think it's a better deal than paying OpenAI for sure, since you have access to GPT-4, Claude+, and others. They also have community bots, etc.

Re: OpenAI's plans according to sama

#156

Post by rurp »

xxprogamerxy wrote: post #63
sharkjacobs wrote: post #24

> He reiterated his belief in the importance of open source and said that OpenAI was considering open-sourcing GPT-3. Part of the reason they hadn’t open-sourced yet was that he was skeptical of how many individuals and companies would have the capability to host and serve large LLMs. Am I reading this right? "We're not open sourcing GPT-3 because we don't think it would be useful to anyone else"

He wants the release of the model to primarily benefit individuals and smaller teams as opposed to large deep-pocketed firms.

And he'll do that by... keeping ChatGPT models away from individuals and small teams and in the hands of a few large deep-pocketed firms?

The great thing about open source is that people can try different approaches and gravitate towards what works best for them. Sam knows that of course, he's just being disingenuous because the truth makes him look bad.

Re: OpenAI's plans according to sama

#157

Post by legendofbrando »

thorax wrote: post #127
m3kw9 wrote: post #52

If you look at their API limits, no serious company can use this to scale up beyond say 10k users. 3500 Reqs per min for gpt3.5 turbo. They have a long way to go to make it usable for the rest of the 95%

I've had to move to using Azure OpenAI service during business hours for the API-- much more stable unless the prompts stray into something a little odd and their API censorship blocks the calls.

I’ve been working directly with OpenAI’s access, are there any other advantages to doing this through Azure?

Re: OpenAI's plans according to sama

#158

Post by brucethemoose2 »

timeserious wrote: post #147
brucethemoose2 wrote: post #118

Earlier quoted context omitted.

Anthropic's Claude is said to be very good. Instruction tuned LLaMA 65B/Falcon 40B are good, especially with an embeddings database. ...But OpenAI has all the name recognition and ease of use now, so it might not even matter if others ambiguously surpass OpenAI models.

Can you ELI5 why an embeddings database helps here? Can pinecone/milvus be used to 'extend memory' of OSS and vendor LLMs without retraining?

First some context: llm "prompts" are actually the whole conversation + initial context. They learn nothing, hence the whole conversation gets fed into them every time, but the instruction following ones are trained to answer your most recent chat response.

In a nutshell, part of your llm prompt (usually your most recent question?) gets fed as a query for the embedding/vector database. It retrieves the most "similar" entries to your question (which is what an embedding database does), and that information is pasted into the context of the llm. Its kinda like pasting the first entry from a local Google search into the beginning of your question as "background."

Some implementations insert your old conversations (that are too big to fit into the llm's context window) into the database as they are pushed out.

This is what I have seen, anyway. Maybe some other implementations do things better.

Re: OpenAI's plans according to sama

#159

Post by bigyikes »

tikwidd wrote: post #151
twobitshifter wrote: post #136

Left the best part until the end. Scaling models larger is still paying off for openai. It’s not AGI yet, but how much bigger will a model need to get to max out? >The scaling hypothesis is the idea that we may have most of the pieces in place needed to build AGI and that most of the remaining work will be taking existing methods and scaling them up to larger models and bigger datasets. If the era of scaling was over…

After training my physics simulator on thousands of hours of video footage of trees moving in the wind, arborists tell me the trees are much more realistic (they are getting worried that I might put them out of business). But the physicists are still not satisfied. How many more videos do I need to generate the laws of motion?

Throw in the videos from the rest of the internet, and you might actually do it…

Re: OpenAI's plans according to sama

#160

Post by ChatGTP »

purplecats wrote: post #32

> Cheaper and faster GPT-4 — This is their top priority. In general, OpenAI’s aim is to drive “the cost of intelligence” down as far as possible and so they will work hard to continue to reduce the cost of the APIs over time. this certainly aligns with the massive (albeit subjective and anecdotal) degradation in quality i've experienced with ChatGPT GPT-4 over the past few weeks. hopefully a superior (higher quality)…

Should "intelligence" have ever costed anything?

It's like saying "air should cost money".

Post reply on HN