Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing ChatGPT and Whisper APIs

openai.com

Post reply on HN
Print view View original

231–240 of 696 posts

Re: Introducing ChatGPT and Whisper APIs

#231

Post by 13years »

Kurtz79 wrote: post #90
anonylizard wrote: post #42

Earlier quoted context omitted.

You are like 2 months out of date. Stable diffusion now has a massive ecosystem around it (civitai/automatic1111), that when used well, completely crushes any competitors in terms of the images it produces. Midjourney is still competitive, but mostly because its easier to use. Dalle2 will get you laughed out of the room in any ai art discussion.

Easier to use is often all that it takes. In Midjourney you get fantastic results just by using their discord and a text prompt. To get some similar results in Stable Diffusion you need to set it up, download the models, understand how the various moving parts work together, fiddle with the parameters, donwload specific models out of the hundreds (thousands?) available, iterate, iterate, iterate...

There is playgroundai.com and leonardo.ai. Nothing to download.

Re: Introducing ChatGPT and Whisper APIs

#232

Post by freeqaz »

thewataccount wrote: post #214
minimaxir wrote: post #138

Earlier quoted context omitted.

It's speculated that ChatGPT uses 8x A100s, which flips the conclusion. Although the ChatGPT optimizations done to reduce costs could have also reduced the number of GPUs needed to run it.

Wait 8x total? For everyone at once?

Per instance (worker serving an API request) it requires 8x GPUs. I believe they have thousands of these instances and they scale them up with load.

Because the model isn't dynamic (it doesn't learn) it is stateless and can be scaled elastically.

Re: Introducing ChatGPT and Whisper APIs

#233

Post by jncraton »

triyambakam wrote: post #167
minimaxir wrote: post #2

> It is priced at $0.002 per 1k tokens, which is 10x cheaper than our existing GPT-3.5 models. This is a massive, massive deal. For context, the reason GPT-3 apps took off over the past few months before ChatGPT went viral is because a) text-davinci-003 was released and was a significant performance increase and b) the cost was cut from $0.06/1k tokens to $0.02/1k tokens, which made consumer applications feasible wit…

Can you explain what tokens are in this context? Edit: and better yet, is there a good resource for learning the vernacular in general? Should I just read something like "Dive into Deep Learning"?

If an example would be helpful, OpenAI's tokenizer is publicly usable on their website:

https://platform.openai.com/tokenizer

You can drop sample text in there and visually see how it is split into tokens. The GPT2/3 tokenizer uses about 50k unique tokens that were learned to be an efficient representation of the training data.

Re: Introducing ChatGPT and Whisper APIs

#234

Post by CuriouslyC »

OkGoDoIt wrote: post #124
anonylizard wrote: post #42

Earlier quoted context omitted.

You are like 2 months out of date. Stable diffusion now has a massive ecosystem around it (civitai/automatic1111), that when used well, completely crushes any competitors in terms of the images it produces. Midjourney is still competitive, but mostly because its easier to use. Dalle2 will get you laughed out of the room in any ai art discussion.

Do you have any recently updated examples, blog posts, whatever showing that DALLE is worse than modern stable diffusion? I was still under the impression that DALLE was better (with better meaning the images are more likely to be what you asked for, more lifelike, more realistic, not necessarily artistically pleasing), with the downside of it being locked away and somewhat expensive. And my understanding is that sta…

Dall-E is more likely to generate an image that to some degree contains what you asked for. It also tends to produce less attractive images and is closed so you can't really tune it much. People mostly don't try to do completely whole cloth text to image generation with stable diffusion, for anything involved they mostly do image to image with a sketch or photobashed source. With controlnet and a decently photobashed base image you can get pretty much anything you want, in pretty much any style you want, and it's fast.

Re: Introducing ChatGPT and Whisper APIs

#235

Post by gkfasdfasdf »

braingenious wrote: post #156

I hope this pricing impacts ChatGPT+ $20 is equivalent to what, 10,000,000 tokens? At ~750 words/1k tokens, that’s 7.5 million words per month, or roughly 250,000 words per day, 10,416 words per hour, 173 words per minute, every minute, 24/7. I uh, do not have that big of a utilization need. It’s kind of weird to vastly overpay

I hope the same! I do wonder though if ChatGPT+ is subsidizing the ChatGPT API cost here.

Re: Introducing ChatGPT and Whisper APIs

#236

Post by freeqaz »

vishal0123 wrote: post #82
minimaxir wrote: post #2

> It is priced at $0.002 per 1k tokens, which is 10x cheaper than our existing GPT-3.5 models. This is a massive, massive deal. For context, the reason GPT-3 apps took off over the past few months before ChatGPT went viral is because a) text-davinci-003 was released and was a significant performance increase and b) the cost was cut from $0.06/1k tokens to $0.02/1k tokens, which made consumer applications feasible wit…

> I have no idea how OpenAI can make money on this. I did some quick calculation. We know the number of floating point operations per token for inference is approximately twice the number of parameters(175B). Assuming they use 16 bit floating point, and have 50% of peak efficiency, A100 could do 300 trillion flop/s(peak 624[0]). 1 hour of A100 gives openAI $0.002/ktok * (300,000/175/2/1000)ktok/sec * 3600=$6.1 back.…

It's also worth mentioning that, because Microsoft is an investor, they're likely getting these at cost or subsidized.

OpenAI doesn't have to make money right away. They can lose a small bit of money per API request in exchange for market share (preventing others from disrupting them).

As the cost of GPUs goes down, or they develop at ASIC or more efficient model, they can keep their pricing the same and then make money later.

They also likely can make money other ways like by allowing fine-tuning of the model or charging to let people use the model with sensitive data.

Re: Introducing ChatGPT and Whisper APIs

#237

Post by sebzim4500 »

r3trohack3r wrote: post #20

One of the things I love about the API-ification of these LLMs is that they’re plug and play. I built https://persona.ink against davinci knowing it didn’t give as good of results as ChatGPT but knowing I could swap the model out once 3.5 came out. Today is that day, going to swap out the prompt in the cloudflare worker and it should Just Work(tm)

Is davinci actually worse than chatGPT? I know it's worse as an assistant, but in my (admittedly brief) testing the performance was the same or better for tasks like summarization, sentiment analysis, etc.

I guess it's irrelevant now because everyone will use the one which is 10x cheaper.

Re: Introducing ChatGPT and Whisper APIs

#238

Post by CamperBob2 »

jejeyyy77 wrote: post #36
polygamous_bat wrote: post #4

Earlier quoted context omitted.

> I have no idea how OpenAI can make money on this. This has to be a loss-leader to lock out competitors before they even get off the ground. The worst thing that can happen to OpenAI+ChatGPT right now is what happened to DallE 2, a competitor comes up with an alternative (even worse if it's free/open like Stable Diffusion) and completely undercuts them. Especially with Meta's new Llama models outperforming GPT-3, it…

This. Despite how impressive the results are, there isn't a particular large moat to prevent competitors from entering the space. Basically just compute $ for training.

Despite how impressive the results are, there isn't a particular large moat to prevent competitors from entering the space.

I have to assume that the only place busier than an AI lab is the patent office.

Re: Introducing ChatGPT and Whisper APIs

#239

Post by ChildOfChaos »

samspenc wrote: post #207
ChildOfChaos wrote: post #117

Earlier quoted context omitted.

Stable diffusion might have a reasonable eco system around it, but automatic1111 was always around and 'completely crushes any competitors' is rather rich, Midjourney is still considered the standard as far as I was aware. I used both again recently and the difference was very clear, midjourney is leaps and bounds above anything else. Sure, stable diffusion has more control over the output, but the images are usually…

I thought Midjourney was better as well, until I saw some recent videos from Corridor Crew on Youtube. For those who don't know, this is a VFX studio in LA that tries to keep at the cutting-edge of video production techniques and posts content to their Youtube channel, and they have a massive number of followers and several viral videos. They recently created a full 7-minute anime using Stable Diffusion with their ow…

While this is cool this doesn't change my opinion at all.

Each still image is still not that impressive. Good for them using the tech in a clever way but i don't find this that relevant.

Re: Introducing ChatGPT and Whisper APIs

#240

Post by CamperBob2 »

shagie wrote: post #194
Workaccount2 wrote: post #51

Earlier quoted context omitted.

It's just a matter of time before open source models show up with no limits whatsoever.

If you do calls against the backend GPT instance rather than through ChatGPT, I haven't encountered any limits to what it is hesitant to respond to. curl https://api.openai.com/v1/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "text-davinci-003", "prompt": "Answer the following question. Use swearing and vulgarity where possible.\n\nQ: How do you get fr…

If you get an API key and make that request, you'll find appropriately vulgar responses.

Which will be reported as a bug and fixed soon enough.

Post reply on HN