Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing ChatGPT and Whisper APIs

openai.com

Post reply on HN
Print view View original

221–230 of 696 posts

Re: Introducing ChatGPT and Whisper APIs

#221

Post by cmelbye »

manmal wrote: post #208
travisjungroth wrote: post #192

Earlier quoted context omitted.

If you think you're overpaying just hit the API yourself.

Any idea how to encode the previous messages when sending a followup question? E.g.: 1. I ask Q1 2. API responds with A1 3. I ask Q2, but want it to preserve Q1 and A1 as context Does Q2 just prefix the conversation like this? „I previously asked {Q1}, to which you answered {A1}. {Q2}“

This is explained in the OpenAI docs. There is a chat completion API and you pass in the prior messages from both the user and the assistant.

Re: Introducing ChatGPT and Whisper APIs

#222

Post by pelasaco »

minimaxir wrote: post #138
vishal0123 wrote: post #82

Earlier quoted context omitted.

> I have no idea how OpenAI can make money on this. I did some quick calculation. We know the number of floating point operations per token for inference is approximately twice the number of parameters(175B). Assuming they use 16 bit floating point, and have 50% of peak efficiency, A100 could do 300 trillion flop/s(peak 624[0]). 1 hour of A100 gives openAI $0.002/ktok * (300,000/175/2/1000)ktok/sec * 3600=$6.1 back.…

It's speculated that ChatGPT uses 8x A100s, which flips the conclusion. Although the ChatGPT optimizations done to reduce costs could have also reduced the number of GPUs needed to run it.

I checked the price of a A100, and its costs 15k? Is that right?

Re: Introducing ChatGPT and Whisper APIs

#223

Post by CuriouslyC »

evilduck wrote: post #144
Cantinflas wrote: post #53

Earlier quoted context omitted.

It's amazing that "being two months out of date" in AI means that you are already a dinosaur

Besides HN, what other venues are popular for staying current on this topic?

if you hang out in reddit.com/r/stablediffusion you'll always be up to date

Re: Introducing ChatGPT and Whisper APIs

#224

Post by sebzim4500 »

manmal wrote: post #208
travisjungroth wrote: post #192

Earlier quoted context omitted.

If you think you're overpaying just hit the API yourself.

Any idea how to encode the previous messages when sending a followup question? E.g.: 1. I ask Q1 2. API responds with A1 3. I ask Q2, but want it to preserve Q1 and A1 as context Does Q2 just prefix the conversation like this? „I previously asked {Q1}, to which you answered {A1}. {Q2}“

Probably something like that.

You could try formatting it like

Question 1: ... Answer 1: ...

...

Question n: ... Answer n: ...

It makes you vulnerable to prompt injection, but for most cases this would probably work fine.

Re: Introducing ChatGPT and Whisper APIs

#225

Post by braingenious »

juice_bus wrote: post #206
braingenious wrote: post #156

I hope this pricing impacts ChatGPT+ $20 is equivalent to what, 10,000,000 tokens? At ~750 words/1k tokens, that’s 7.5 million words per month, or roughly 250,000 words per day, 10,416 words per hour, 173 words per minute, every minute, 24/7. I uh, do not have that big of a utilization need. It’s kind of weird to vastly overpay

Most of the value for me with ChatGPT+ is getting access when the system is at capacity.

I wouldn’t mind paying a premium for the convenience (maybe $5 per month, billed monthly, max), but I’m definitely not spending $20.

Re: Introducing ChatGPT and Whisper APIs

#226

Post by rchaud »

rtsil wrote: post #47
minimaxir wrote: post #2

> It is priced at $0.002 per 1k tokens, which is 10x cheaper than our existing GPT-3.5 models. This is a massive, massive deal. For context, the reason GPT-3 apps took off over the past few months before ChatGPT went viral is because a) text-davinci-003 was released and was a significant performance increase and b) the cost was cut from $0.06/1k tokens to $0.02/1k tokens, which made consumer applications feasible wit…

It is so massive that I can't help but think about what happened with Google Maps API a few years ago where they had extremely low pricing for years then hiked the price by 1400% once enough people were locked into applications based on that API.

That's exactly what's going to happen. Low prices now, wait until your business becomes dependent on it, then jack it up to whatever you need it to be.

Re: Introducing ChatGPT and Whisper APIs

#227

Post by 13years »

gremlinsinc wrote: post #213
Cantinflas wrote: post #53

Earlier quoted context omitted.

It's amazing that "being two months out of date" in AI means that you are already a dinosaur

That's what the singularity is all about, a moment in time when 2 seconds late turns you into a dinosaur, be greatful it's 2 months, not 2 weeks, 2 days, or 2 minutes.

The AI utopia seems to be evolving into just a new rat race. I'm obsolete before I can think about it.

Re: Introducing ChatGPT and Whisper APIs

#228

Post by CuriouslyC »

Kurtz79 wrote: post #90
anonylizard wrote: post #42

Earlier quoted context omitted.

You are like 2 months out of date. Stable diffusion now has a massive ecosystem around it (civitai/automatic1111), that when used well, completely crushes any competitors in terms of the images it produces. Midjourney is still competitive, but mostly because its easier to use. Dalle2 will get you laughed out of the room in any ai art discussion.

Easier to use is often all that it takes. In Midjourney you get fantastic results just by using their discord and a text prompt. To get some similar results in Stable Diffusion you need to set it up, download the models, understand how the various moving parts work together, fiddle with the parameters, donwload specific models out of the hundreds (thousands?) available, iterate, iterate, iterate...

Setting up the environment and tooling around in the code is not a burden, it's a nice change of pace from the boring code I have to deal with normally. Likewise, playing around to build intuition about how prompts and parameters correspond to neighborhoods in latent space is quite fun.

Beyond that, being able to go to sleep with my computer doing a massive batch job state space exploration and wake up with a bunch of cool stuff to look at gives me Christmas vibes daily.

Re: Introducing ChatGPT and Whisper APIs

#229

Post by vineyardmike »

thewataccount wrote: post #214
minimaxir wrote: post #138

Earlier quoted context omitted.

It's speculated that ChatGPT uses 8x A100s, which flips the conclusion. Although the ChatGPT optimizations done to reduce costs could have also reduced the number of GPUs needed to run it.

Wait 8x total? For everyone at once?

Each model needs 8x to run at the same time per request.

Re: Introducing ChatGPT and Whisper APIs

#230

Post by Zetaphor »

manmal wrote: post #208
travisjungroth wrote: post #192

Earlier quoted context omitted.

If you think you're overpaying just hit the API yourself.

Any idea how to encode the previous messages when sending a followup question? E.g.: 1. I ask Q1 2. API responds with A1 3. I ask Q2, but want it to preserve Q1 and A1 as context Does Q2 just prefix the conversation like this? „I previously asked {Q1}, to which you answered {A1}. {Q2}“

In addition to the other comment this type of memory is a feature in LLM frameworks like Langchain
Post reply on HN