Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing ChatGPT and Whisper APIs

openai.com

Post reply on HN
Print view View original

241–250 of 696 posts

Re: Introducing ChatGPT and Whisper APIs

#241

Post by mlboss »

krelian wrote: post #33
polygamous_bat wrote: post #4

Earlier quoted context omitted.

> I have no idea how OpenAI can make money on this. This has to be a loss-leader to lock out competitors before they even get off the ground. The worst thing that can happen to OpenAI+ChatGPT right now is what happened to DallE 2, a competitor comes up with an alternative (even worse if it's free/open like Stable Diffusion) and completely undercuts them. Especially with Meta's new Llama models outperforming GPT-3, it…

I thought it was Midjourney who stole their thunder. Stable Diffusion is free but it's much harder to get good results with it. Midjourney on the other hand spits out art with a very satisfying style.

Stable diffusion + ControlNet is fire! Nothing compares to it. ControlNet allows you to have tight control over the output. https://github.com/lllyasviel/ControlNet

Re: Introducing ChatGPT and Whisper APIs

#242

Post by CuriouslyC »

drusepth wrote: post #101
TechBro8615 wrote: post #26

Earlier quoted context omitted.

Someone can still undercut them by offering an uncensored version.

For better or for worse, it seems like this would inherently need to come from a self-hostable, open-source version so 100% "liability" could be shifted from provider to user.

We'll be running highly quantized, somewhat distilled versions of something similar to Llama on our devices before long, and I don't think the RLHF part will take long to be replicated, the biggest block there is just data.

Re: Introducing ChatGPT and Whisper APIs

#243

Post by alchemist1e9 »

pelasaco wrote: post #222
minimaxir wrote: post #138

Earlier quoted context omitted.

It's speculated that ChatGPT uses 8x A100s, which flips the conclusion. Although the ChatGPT optimizations done to reduce costs could have also reduced the number of GPUs needed to run it.

I checked the price of a A100, and its costs 15k? Is that right?

And $2.25 per hour on 1 year reservation means 8,760 hours x 2.25 = $19,710 rent for the year. Not a bad yield for the provider at all, but makes sense given overheads and ROI expected.

Re: Introducing ChatGPT and Whisper APIs

#245

Post by CuriouslyC »

Zurrrrr wrote: post #148
jameshart wrote: post #96

Earlier quoted context omitted.

I do struggle with understanding why people think this is strangling the potential of GPT. Do you find yourself frustrated working with your colleagues, thinking, “you know, I bet if they felt more free to utter racist slurs or endorse illegal activities, we would get a ton more done around here”?

I can only see it affecting 'art', where you might want to have characters that are despicable say despicable things. But really we shouldn't be using AI to make our art for us anyway. Help, sure, but it shouldn't be literally writing our stories.

So you feel that when progress enables us to provide more abundance for humanity, we should artificially limit that abundance for everyone so that a few people aren't inconvenienced?

Re: Introducing ChatGPT and Whisper APIs

#247

Post by bluecoconut »

Support for the ChatGPT endpoint now added to lambdaprompt[1]! (solves a similar problem as langchain, with almost no boilerplate!) Props to openai for making such a usable endpoint, was very easy to wrap.

Example code using the new function and endpoint:

    import lambdaprompt as lp
    convo = lp.AsyncGPT3Chat([{'system': 'You are a {{ type_of_bot }}'}])
    await convo("What should we get for lunch?", type_of_bot="pirate")
> As a pirate, I would suggest we have some hearty seafood such as fish and chips or a seafood platter. We could also have some rum to wash it down! Arrr!

(In order to use lambdaprompt, just `pip install lambdaprompt` and export OPENAI_API_KEY=...)

[1] https://github.com/approximatelabs/lambdaprompt

Re: Introducing ChatGPT and Whisper APIs

#248

Post by sacred_numbers »

habitue wrote: post #166

Speculation: GPT-turbo is a new chinchilla optimal model with the equivalent capabilities as GPT-3.5. So it's literally just smaller, faster and cheaper to run. The reason I don't think it's just loss-leading is that they made it faster too. That heavily implies a smaller model.

It could be even smaller than a Chinchilla optimal model. The Chinchilla paper was about training the most capable models with the least training compute. If you are optimizing for capability and inference compute you can "over-train" by providing much more data per parameter than even Chinchilla, or you can train a larger model and then distill it to a smaller size. Increasing context size increases inference compute, but the increased capabilities of high context size might allow you to skimp on parameters and lead to a net decrease in compute. There's probably other strategies as well, but those are the ones I know of.

Re: Introducing ChatGPT and Whisper APIs

#250

Post by CuriouslyC »

sebzim4500 wrote: post #219
visarga wrote: post #161

Earlier quoted context omitted.

They probably shrunk the model from 175B to 17B. That's your 10:1 price reduction.

Wouldn't that almost certainly lead to measurable loss of capabilities?

If the model was quantized/distilled correctly, not for a large swath of use cases/problem domain. For anything where loss was not measured during distillation, very likely.
Post reply on HN