Live data from Hacker News

Introducing ChatGPT and Whisper APIs

openai.com

161–170 of 696 posts

Re: Introducing ChatGPT and Whisper APIs

#161

> It is priced at $0.002 per 1k tokens, which is 10x cheaper than our existing GPT-3.5 models. This is a massive, massive deal. For context, the reason GPT-3 apps took off over the past few months before ChatGPT went viral is because a) text-davinci-003 was released and was a significant performance increase and b) the cost was cut from $0.06/1k tokens to $0.02/1k tokens, which made consumer applications feasible wit…

They probably shrunk the model from 175B to 17B. That's your 10:1 price reduction.

Re: Introducing ChatGPT and Whisper APIs

#163
Question, Can I give openAI some data for it to process so I can use it to my own advantage. Say I want to train it on specific topic of information I've gathered over the years. Can I some how give it that data and then I can use the API to get back out data in a chat or some other forms of questions?

I'm not too familiar with how it works.

Re: Introducing ChatGPT and Whisper APIs

#164
I find myself missing the golden age of Google, where it actually returned results and answers that you wanted, on subjects you were looking for. Even now, versus two years ago, I tried finding a snippet of a notice in a newspaper with specifics about a name change someone petitioned in California. I found it then, and had bookmarked it, but trying to find it again just turns up absolute garbage, thankfully I found the bookmark. I can go to ChatGPT and ask it about the best vegetable or fruit to grow in a several gallon Kratky setup, looking for the same sort of answer in Google returns absolute garbage.

I'll concede that LLMs like ChatGPT are the future, thanks to the NLU stuff from OpenAI and the dataset, but only the future of "agents" if you want to call it that. The "intelligence" exhibited emergent from language itself, from the massive dataset it has trawled. Our language and knowledge. But at the same time I surely hope that another AI winter doesn't come because of people over-promising and under-delivering. Or, too much focus on LLMs themselves because of that "wow" factor, the same wow factor you got in the past, when search engines weren't garbage if you knew how to use them and what their shortcomings were.

Re: Introducing ChatGPT and Whisper APIs

#165
post #16

Pricing is good because OpenAI does not need to make any money but needs data for feedback, if everyone switches to open source ( Llama etc. ) they won't get the data they need. Google is testing their system internally with XX thousand users, OpenAI with XXX million users ...

Something that has been bothering me for a while is whether poisoning of OpenAI's dataset is possible, willingly or otherwise.

An example here is getting chatGPT to accept that that 2+2=5, it's a lot of effort, but can be done. Then the users can give thumbs up when such responses are given.

Could this cause issues?

Re: Introducing ChatGPT and Whisper APIs

#166
Speculation: GPT-turbo is a new chinchilla optimal model with the equivalent capabilities as GPT-3.5. So it's literally just smaller, faster and cheaper to run.

The reason I don't think it's just loss-leading is that they made it faster too. That heavily implies a smaller model.

Re: Introducing ChatGPT and Whisper APIs

#167

> It is priced at $0.002 per 1k tokens, which is 10x cheaper than our existing GPT-3.5 models. This is a massive, massive deal. For context, the reason GPT-3 apps took off over the past few months before ChatGPT went viral is because a) text-davinci-003 was released and was a significant performance increase and b) the cost was cut from $0.06/1k tokens to $0.02/1k tokens, which made consumer applications feasible wit…

Can you explain what tokens are in this context?

Edit: and better yet, is there a good resource for learning the vernacular in general? Should I just read something like "Dive into Deep Learning"?

Re: Introducing ChatGPT and Whisper APIs

#168
post #57

Does ChatGPT yet have a debug function to Show Its Work so to speak? I think this will be important in the future when it gets itself into drama, trouble, etc.. Probably also useful to prove how ChatGPT created something rather than being known as an opaque box.

I'm pretty sure any system built via linear regression or similar is an opaque box even to the most experienced researchers. For example: https://clementneo.com/posts/2023/02/11/we-found-an-neuron These are massive functions with billions of parameters that evolved over millions of computing years.

> These are massive functions with billions of parameters that evolved over millions of computing years.

This is a great way to put it!

Re: Introducing ChatGPT and Whisper APIs

#169

Earlier quoted context omitted.

> I have no idea how OpenAI can make money on this. I did some quick calculation. We know the number of floating point operations per token for inference is approximately twice the number of parameters(175B). Assuming they use 16 bit floating point, and have 50% of peak efficiency, A100 could do 300 trillion flop/s(peak 624[0]). 1 hour of A100 gives openAI $0.002/ktok * (300,000/175/2/1000)ktok/sec * 3600=$6.1 back.…

It's speculated that ChatGPT uses 8x A100s, which flips the conclusion. Although the ChatGPT optimizations done to reduce costs could have also reduced the number of GPUs needed to run it.

Would multiplying the GPUs by 8 decrease another part of the equation by 1/8, i.e. X flops on 1 GPU = Y seconds, X flops on 8 GPUs = Y / 8?

(Btw I keep running into you or your content the past couple months, thanks for all you do and your well thought out contributions -@jpohhhh)

Re: Introducing ChatGPT and Whisper APIs

#170
ELI5 What is a token?

Is it a word, question, letter, what? If I ask a question like... What is the capital of Canada? And it responds with 'Ottawa', how many tokens have I used there and how are they calculated?

Post reply on HN