> It is priced at $0.002 per 1k tokens, which is 10x cheaper than our existing GPT-3.5 models. This is a massive, massive deal. For context, the reason GPT-3 apps took off over the past few months before ChatGPT went viral is because a) text-davinci-003 was released and was a significant performance increase and b) the cost was cut from $0.06/1k tokens to $0.02/1k tokens, which made consumer applications feasible wit…
Introducing ChatGPT and Whisper APIs
161–170 of 696 posts
Re: Introducing ChatGPT and Whisper APIs
#162Re: Introducing ChatGPT and Whisper APIs
#163I'm not too familiar with how it works.
Re: Introducing ChatGPT and Whisper APIs
#164I'll concede that LLMs like ChatGPT are the future, thanks to the NLU stuff from OpenAI and the dataset, but only the future of "agents" if you want to call it that. The "intelligence" exhibited emergent from language itself, from the massive dataset it has trawled. Our language and knowledge. But at the same time I surely hope that another AI winter doesn't come because of people over-promising and under-delivering. Or, too much focus on LLMs themselves because of that "wow" factor, the same wow factor you got in the past, when search engines weren't garbage if you knew how to use them and what their shortcomings were.
Re: Introducing ChatGPT and Whisper APIs
#165Pricing is good because OpenAI does not need to make any money but needs data for feedback, if everyone switches to open source ( Llama etc. ) they won't get the data they need. Google is testing their system internally with XX thousand users, OpenAI with XXX million users ...
An example here is getting chatGPT to accept that that 2+2=5, it's a lot of effort, but can be done. Then the users can give thumbs up when such responses are given.
Could this cause issues?
Re: Introducing ChatGPT and Whisper APIs
#166The reason I don't think it's just loss-leading is that they made it faster too. That heavily implies a smaller model.
Re: Introducing ChatGPT and Whisper APIs
#167> It is priced at $0.002 per 1k tokens, which is 10x cheaper than our existing GPT-3.5 models. This is a massive, massive deal. For context, the reason GPT-3 apps took off over the past few months before ChatGPT went viral is because a) text-davinci-003 was released and was a significant performance increase and b) the cost was cut from $0.06/1k tokens to $0.02/1k tokens, which made consumer applications feasible wit…
Edit: and better yet, is there a good resource for learning the vernacular in general? Should I just read something like "Dive into Deep Learning"?
Re: Introducing ChatGPT and Whisper APIs
#168Does ChatGPT yet have a debug function to Show Its Work so to speak? I think this will be important in the future when it gets itself into drama, trouble, etc.. Probably also useful to prove how ChatGPT created something rather than being known as an opaque box.
I'm pretty sure any system built via linear regression or similar is an opaque box even to the most experienced researchers. For example: https://clementneo.com/posts/2023/02/11/we-found-an-neuron These are massive functions with billions of parameters that evolved over millions of computing years.
This is a great way to put it!
Re: Introducing ChatGPT and Whisper APIs
#169Earlier quoted context omitted.
> I have no idea how OpenAI can make money on this. I did some quick calculation. We know the number of floating point operations per token for inference is approximately twice the number of parameters(175B). Assuming they use 16 bit floating point, and have 50% of peak efficiency, A100 could do 300 trillion flop/s(peak 624[0]). 1 hour of A100 gives openAI $0.002/ktok * (300,000/175/2/1000)ktok/sec * 3600=$6.1 back.…
It's speculated that ChatGPT uses 8x A100s, which flips the conclusion. Although the ChatGPT optimizations done to reduce costs could have also reduced the number of GPUs needed to run it.
(Btw I keep running into you or your content the past couple months, thanks for all you do and your well thought out contributions -@jpohhhh)
Re: Introducing ChatGPT and Whisper APIs
#170Is it a word, question, letter, what? If I ask a question like... What is the capital of Canada? And it responds with 'Ottawa', how many tokens have I used there and how are they calculated?