91–100 of 136 posts
It's $3.80 for the full 1000 prompt tokens plus 32k token long context. But OpenAI doesn't have a monopoly on long context, RWKV is free and open source and has virtually an unlimited context (as long as you remind it every once and a while). However ChatGPT really cannot be matched at this point perhaps except LLaMa Alpaca 7B EDIT: I accidentially typed Alpine, it's Alpaca. https://github.com/tatsu-lab/stanford_alpa…
Earlier quoted context omitted.
Keep in mind OpenAI is able to lose money on GPT*. Inference for the same predictive power will indeed improve (Facebook's recent model is a step in the right direction) but efficiency of running on given hardware likely won't follow moore's law for long.
Are there truely top tier ML optimized chips? I would imagine the hardware space still having room to grow if just because most of the brainpower having been focused on x86, etc.
I'm more impressed that it's less than 3 times as expensive as Davinci GPT3. In usecases where most tokens are in the context even only 50% more expensive, while allowing 4x as much context. And you can pay a 100% surcharge to get another fourfold increase in context size. It only looks expensive when compared to the GPT-3.5-Turbo ChatGPT offering which is incredibly cheap (or alternatively Davinci is overpriced by n…
Worth remembering GPT3 was 10-20 times more expensive 2yrs ago. There's a super-moore's law type learning function going on here and I suspect in 1yr GPT4.5 will be the same cost as GPT3.5 today. It's insane... feels like iPhone 3G to iPhone 4 level quality improvement every year.
GPT-4 is a scary improvement over 3.5, especially for handling code. It will be the literal definition of awesome when these models get a large enough context space to hold a small-medium sized codebase. I've been playing around with it for an hour seeing what it can do to refactor some of the things we have with the most tech debt, and it is astounding how well it does with how little context I give it.
Is it conceivable that ChatGPT could review itself and produce a superior revision to a human?
Earlier quoted context omitted.
> Although the question is: If ChatGPT can do almost anything GPT3.5 can do, then why is it priced almost 10x less? ChatGPT is GPT-3.5; with specific fine tuning / chat-oriented training, and without customer fine-tuning (at least, currently) available. It’s the particular GPT-3.5 interface that OpenAI wants people to preferentially use, so the price structure artificially encourages this.
The ChatGPT API is using a smaller model than the original "GPT 3.5". The original, presumably larger, model is used by Legacy ChatGPT.
If it is in fact true, doesn't it fall into "false advertisement" category? That is, OpenAI using ChatGPT for demonstration purposes but then charging for an API that pretends to be ChatGPT but is in fact based on a much smaller model?