Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New models and developer products

openai.com

Post reply on HN
Print view View original

231–240 of 561 posts

Re: New models and developer products

#231

Post by ebiester »

pclmulqdq wrote: post #107
ebiester wrote: post #59

Earlier quoted context omitted.

I don't understand the lock-in argument here. Yes, if a competitor comes in there will be switching cost as everything is re-learned. However, from a code perspective, it is a function of the key and a relatively small API. New regulations outstanding, what is stoping someone from moving from OpenAI to Anthropic (for example) other than the cost of learning how to effectively utilize Anthropic for your use case? Open…

[flagged]

I have no financial vested interest.

To prove it, I'll post Dr. Emily Bender's fantastic podcast about the problematic issues behind the current LLM wave. https://peertube.dair-institute.org/w/qpKuiNLTuHHMnvWGjnA2D8

(I think they are behind the most cogent critiques and worth knowing.)

Re: New models and developer products

#233

Post by andybak »

andygeorge wrote: post #227
thethimble wrote: post #214

Earlier quoted context omitted.

Or, alternatively, copyright risk is a major concern for real customers, and this is a major step forward in addressing that. Not everything needs to be so cynical. What’s good for investors can be good for users as well.

did an AI write this

Nice jab but it doesn't sound like AI so it's not as cutting as your probably thought.

Re: New models and developer products

#235

Post by Der_Einzige »

saliagato wrote: post #67
Swizec wrote: post #35

Earlier quoted context omitted.

> 128k context? Absolutely insane 128k context is great and all, but how effective are the middle 100,000 tokens? LLMs are known to struggle with remembering stuff that isn't at the start or end of the input. Known as the Lost Middle https://arxiv.org/abs/2307.03172

sama said they improved it

We can't just take his word for it. This needs experimental verification. It's likely not even close to solved.

Re: New models and developer products

#236

Post by petercooper »

singularity2001 wrote: post #131

did they break the api? from openai import OpenAI Traceback (most recent call last): File " ", line 1, in ImportError: cannot import name 'OpenAI' from 'openai' If so where is the current documentation?

v1.0/1.1 of the `openai` Python package differ significantly from the 0.x versions. You'll want to upgrade the package before following the instructions you were following. More info here: https://github.com/openai/openai-python/discussions/631

Re: New models and developer products

#237

Post by Der_Einzige »

minimaxir wrote: post #140
raylad wrote: post #115

So with 128K context window, if you actually input 100K it would cost you: Input: $0.01 per 1K tokens * 100 = $1.00 $1.00 per query? Given that each query uses the entire context window, the session would start at $1 for the first query and go up from there? Or do I have it wrong?

It would be $1 for each individual API call, if you were continuing the conversation based on the same 100K input. ChatGPT is stateless.

This is a sad fact, and one which they should have implemented a fix for.

We know medium term memory works. Sentence transformers and everyone playing with pooled embeddings knows what it is because they're using it. I should be able to map my previous history to a smaller number of tokens using embedding pooling to give a notion of a lossy "medium term" memory independent of RAG.

Re: New models and developer products

#239

Post by MagicMoonlight »

saliagato wrote: post #105

You can now [1] pay from $2 to $3 million to pretrain custom gpt-n model. This has gone unnoticed but seems really neat. Provided that a start-up has enough money spend on that, it would certainly give competitive advantage. [1] https://openai.com/form/custom-models Edit: forgot to put the link

Well it won’t because they’ll use the model you paid for and take your customers.

Re: New models and developer products

#240

Post by MagicMoonlight »

zavertnik wrote: post #13

And here I was in bliss with the 32k context increase 3 days ago. 128k context? Absolutely insane. It feels like now the bottle neck in GPT workflows is no longer GPT, but instead its the wallet! Such an amazing time to be alive.

It’s insane because it makes no sense. When you read a book you don’t remember the last 100,000 words. It’s so wildly inefficient to do it that way.
Post reply on HN