Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing ChatGPT and Whisper APIs

openai.com

Post reply on HN
Print view View original

421–430 of 696 posts

Re: Introducing ChatGPT and Whisper APIs

#421

Post by DoesntMatter22 »

OkGoDoIt wrote: post #124
anonylizard wrote: post #42

Earlier quoted context omitted.

You are like 2 months out of date. Stable diffusion now has a massive ecosystem around it (civitai/automatic1111), that when used well, completely crushes any competitors in terms of the images it produces. Midjourney is still competitive, but mostly because its easier to use. Dalle2 will get you laughed out of the room in any ai art discussion.

Do you have any recently updated examples, blog posts, whatever showing that DALLE is worse than modern stable diffusion? I was still under the impression that DALLE was better (with better meaning the images are more likely to be what you asked for, more lifelike, more realistic, not necessarily artistically pleasing), with the downside of it being locked away and somewhat expensive. And my understanding is that sta…

Dalle 2 was great initially but the SD BLEW past it. I mean way way way past it. Dalle2 is like a Model T Ford and SD is a Fighter Jet. It's that different. Dalle-2 is dead already.

Re: Introducing ChatGPT and Whisper APIs

#422

Post by dankwizard »

whatshisface wrote: post #362
npunt wrote: post #338

Earlier quoted context omitted.

Yeah we're in an AI landgrab right now where at- or below-cost pricing is buying marketshare, lock-in, and underdevelopment of competitors. Smart move for them to pour money into it.

We have got to find a word for plans that are plainly harmful yet advantageous to their executors that's more descriptive than "smart..."

Tactical

Re: Introducing ChatGPT and Whisper APIs

#423

Post by 13years »

jonplackett wrote: post #360
13years wrote: post #244

Earlier quoted context omitted.

> I am scared for all people Probably could stop there.

Yeah that is actually true. Yeah that’s probably truest. But I’m more scared for some than others short term. I’m less immediately scared for anyone doing work that interacts with the physical world. Weird how it turned out the robotics was harder than the thinking

Yes, short term agreed.

AI has flipped on its head much of what we thought early AI would be like with creativity being one of its most successful targets.

I suspect the surprises will only continue to be, lets say, more surprising.

I've actually written a lot about that recently as well. FYI https://dakara.substack.com/p/ai-and-the-end-to-all-things

Re: Introducing ChatGPT and Whisper APIs

#424

Post by m3kw9 »

sva_ wrote: post #59
alexb_ wrote: post #9

>Through a series of system-wide optimizations, we’ve achieved 90% cost reduction for ChatGPT since December This is seriously impressive. A MILLION tokens for 2 dollars is absolutely fucking insane. I hope that the gains reached here can also be found by open source and non-controlled AI projects. If so, that could be huge for the advancement of AI.

A lot of people assumed GPT4 would be an even bigger model, but I've been thinking it'll probably be more about more efficient compute. This is at least some evidence that they're working on that.

GPT3s

Re: Introducing ChatGPT and Whisper APIs

#425

Post by daveguy »

CamperBob2 wrote: post #238
jejeyyy77 wrote: post #36

Earlier quoted context omitted.

This. Despite how impressive the results are, there isn't a particular large moat to prevent competitors from entering the space. Basically just compute $ for training.

Despite how impressive the results are, there isn't a particular large moat to prevent competitors from entering the space. I have to assume that the only place busier than an AI lab is the patent office.

Hopefully the patent office will recognize that tacking on "...but with AI" isn't novel or non-obvious and a lot of the fever patents will be denied quickly.

Re: Introducing ChatGPT and Whisper APIs

#426

Post by kgeist »

cmelbye wrote: post #221
manmal wrote: post #208

Earlier quoted context omitted.

Any idea how to encode the previous messages when sending a followup question? E.g.: 1. I ask Q1 2. API responds with A1 3. I ask Q2, but want it to preserve Q1 and A1 as context Does Q2 just prefix the conversation like this? „I previously asked {Q1}, to which you answered {A1}. {Q2}“

This is explained in the OpenAI docs. There is a chat completion API and you pass in the prior messages from both the user and the assistant.

I used the same trick with the previous GPT3 API (da-vinci) and it worked well, I'd pass as one big prompt:

  User: hello (previous prompt)
  Bot: hi (previous response)
  User: who are you? (new prompt)
  Bot: (here it continues conversation)
I wonder how the new ChatGPT API differs, other than the fact that it's structured (you use JSON to represent the conversation memory separately instead of one large prompt).

I guess I will spend the next day playing around with the new API to figure it out.

Re: Introducing ChatGPT and Whisper APIs

#427

Post by neilv »

LrnByTeach wrote: post #415
minimaxir wrote: post #2

> It is priced at $0.002 per 1k tokens, which is 10x cheaper than our existing GPT-3.5 models. This is a massive, massive deal. For context, the reason GPT-3 apps took off over the past few months before ChatGPT went viral is because a) text-davinci-003 was released and was a significant performance increase and b) the cost was cut from $0.06/1k tokens to $0.02/1k tokens, which made consumer applications feasible wit…

This is playbook taken from Amazon prime. This massive price cut, I believe, is intended to undercut competing open source ChatGPT equivalent initiatives. OpenAI/Micorsoft may be losing money with this new pricing, but that is on purpose. At these lower prices most of the OpenSource alternatives in the works will have difficult time continuing projects. After few years, when most open source alternatives have died, O…

When it's to drive out the competition, I think it's called "dumping". (I first heard of this as "chip dumping", as in semiconductor chips.) https://en.wikipedia.org/wiki/Dumping_(pricing_policy)

It can also be to build a market, to encourage customers to invest in building atop this.

In any case, I think no customers should be making assumptions about costs too far ahead. (Since the price could go up or the price model change, the supplier could get out of that business, supplier could give your competitor a better deal or just cut you off , near-future tech evolution necessary to be competitive might have very different pricing or availability to you, etc.)

Re: Introducing ChatGPT and Whisper APIs

#428

Post by TechBro8615 »

salted-fry wrote: post #358
skeaker wrote: post #131

Earlier quoted context omitted.

I tried to ask it if Goku could beat a quadrillion bees in a fight and it said it couldn't tell me because that would be encouraging violence. I think it would be great if it would just tell me instead

Perhaps you were using a different version, but I just tried and ChatGPT didn't seem to have any ethical issues with the question (although it was cagey about giving any definite answer): https://i.imgur.com/5aIjtMz.png

Thank you for posting a link to an image instead of polluting the future training data of GPT-4 with the output of GPT-3 :)

I wish more people would do this. I'm getting pretty sick of the walls of text.

Re: Introducing ChatGPT and Whisper APIs

#430

Post by kkielhofner »

ALittleLight wrote: post #377
kkielhofner wrote: post #334

Earlier quoted context omitted.

But those A100s only come by eight and it’s speculated the model requires eight (VRAM). For a three year reservation that comes to over $96k/yr - to support one concurrent request.

What do you mean one concurrent request? Can't you have a huge batch size to basically support a huge number of concurrent requests? e.g. Endpoint feeds a queue, queue fills a batch, batched results generate replies. You are simultaneously fulfilling many requests.

Hopefully they’re doing plenty of batching - you don’t even need to roll your own as you’re describing. Inference servers like Triton will dynamically batch requests with SLA params for max response time (for example).

That said I don’t think anyone anyone outside of OpenAI knows what’s going on operationally. Same goes for VRAM usage, potential batch sizes, etc. This is all wild speculation. Same goes for whatever terms OpenAI is getting out of MS/Azure.

What isn’t wild speculation is that even with three year reserve pricing last gen A100x8 (H100 is shipping) will set you back $100k/yr - plus all of the usual cloud bandwidth, etc fees that would likely increase that by at least 10-20%.

We’re talking about their pricing and costs here. This gives a general idea what anyone trying to self host this would be up against - even if they could get the model.

Post reply on HN