Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New models and developer products

openai.com

Post reply on HN
Print view View original

121–130 of 561 posts

Re: New models and developer products

#121

Post by sharemywin »

colordrops wrote: post #90
whytai wrote: post #71

Every day this video ages more and more poorly [1]. categories of startups that will be affected by these launches: - vectorDB startups -> don't need embeddings anymore - file processing startups -> don't need to process files anymore - fine tuning startups -> can fine tune directly from the platform now, with GPT4 fine tuning coming - cost reduction startups -> they literally lowered prices and increased rate limits…

I haven't been paying attention, why are embeddings not needed anymore?

Retrieval: augments the assistant with knowledge from outside our models, such as proprietary domain data, product information or documents provided by your users. This means you don’t need to compute and store embeddings for your documents, or implement chunking and search algorithms. The Assistants API optimizes what retrieval technique to use based on our experience building knowledge retrieval in ChatGPT.

The model then decides when to retrieve content based on the user Messages. The Assistants API automatically chooses between two retrieval techniques:

it either passes the file content in the prompt for short documents, or performs a vector search for longer documents Retrieval currently optimizes for quality by adding all relevant content to the context of model calls. We plan to introduce other retrieval strategies to enable developers to choose a different tradeoff between retrieval quality and model usage cost.

Re: New models and developer products

#122

Post by echelon »

whytai wrote: post #71

Every day this video ages more and more poorly [1]. categories of startups that will be affected by these launches: - vectorDB startups -> don't need embeddings anymore - file processing startups -> don't need to process files anymore - fine tuning startups -> can fine tune directly from the platform now, with GPT4 fine tuning coming - cost reduction startups -> they literally lowered prices and increased rate limits…

We don't want Open AI to win everything.

Re: New models and developer products

#123

Post by minimaxir »

ebiester wrote: post #59
minimaxir wrote: post #4

Most of the products announced (and the price cuts) appear to be more about increasing lock-in to the OpenAI API platform, which is not surprising given increased competition in the space. The GPTs/GPT Agents and Assistants demos in particular showed that they are a black box within a black box within a black box that you can't port anywhere else. I'm mixed on the presentation and will need to read the fine print on…

I don't understand the lock-in argument here. Yes, if a competitor comes in there will be switching cost as everything is re-learned. However, from a code perspective, it is a function of the key and a relatively small API. New regulations outstanding, what is stoping someone from moving from OpenAI to Anthropic (for example) other than the cost of learning how to effectively utilize Anthropic for your use case? Open…

> OpenAI doesn't have some sort of egress feed for your database.

That's what they're trying to incentivize, especically with being able to upload files for their own implementation of RAG. You're not getting the vector representation of those files back, and switching to another provider will require rebuilding and testing that infrastructure.

Re: New models and developer products

#124

Post by ushakov »

esafak wrote: post #110
ushakov wrote: post #96

Earlier quoted context omitted.

We just launched our AI-based API-Testing tool ( https://ai.stepci.com ), despite having competitors like GitHub Co-Pilot. Why? Because they lack specificity. We're domain experts, we know how to prompt it correctly to get the best results for a given domain. The moat is having model do one task extremely well rather than do 100 things "alright"

If you just launched it is too soon to speak.

Of course! Today our assumption is that LLMs are commodities and our job is to get the most out of them for the type of problem we're solving (API Testing for us!)

Re: New models and developer products

#125

Post by nextworddev »

lazzlazzlazz wrote: post #93
colordrops wrote: post #90

Earlier quoted context omitted.

I haven't been paying attention, why are embeddings not needed anymore?

OP is incorrect. Embeddings are still needed since (1) context windows can't contain all data and (2) data memorization and continuous retraining is not yet viable.

"yet"

Re: New models and developer products

#126

Post by coder543 »

pclmulqdq wrote: post #108
coder543 wrote: post #66

Earlier quoted context omitted.

OpenAI offering 128k context is very appealing, however. I tried some Mistral variants with larger context windows, and had very poor results… the model would often offer either an empty completion or a nonsensical completion, even though the content fit comfortably within the context window, and I was placing a direct question either at the beginning or end, and either with or without an explanation of the task and…

Comparing a 7B parameter model to a 1.8T parameter model is kind of silly. Of course it's behind on accuracy, but it also takes 1% of the resources.

The person I replied to had decided to compare Mistral to what was launched, so I went along with their comparison and showed how I have been unsatisfied with it. But, these open models can certainly be fun to play with.

Regardless, where did you find 1.8T for GPT-4 Turbo? The Turbo model is the one with the 128K context size, and the Turbo models tend to have a much lower parameter count from what people can tell. Nobody outside of OpenAI even knows how many parameters regular GPT-4 has. 1.8T is one of several guesses I have seen people make, but the guesses vary significantly.

I’m also not convinced that parameter counts are everything, as your comment clearly implies, or that chinchilla scaling is fully understood. More research seems required to find the right balance: https://espadrine.github.io/blog/posts/chinchilla-s-death.ht...

Re: New models and developer products

#127

Post by renewiltord »

morkalork wrote: post #87
whytai wrote: post #71

Every day this video ages more and more poorly [1]. categories of startups that will be affected by these launches: - vectorDB startups -> don't need embeddings anymore - file processing startups -> don't need to process files anymore - fine tuning startups -> can fine tune directly from the platform now, with GPT4 fine tuning coming - cost reduction startups -> they literally lowered prices and increased rate limits…

If you want to be a start-up using AI, you have to be in another industry with access to data and a market that OpenAI/MS/Google can't or won't touch. Otherwise you end up eaten like above.

Writer.ai is quite successful, and is totally in another industry that Google+MS participate in.

Re: New models and developer products

#129

Post by bluecrab »

shanusmagnus wrote: post #116

This is kind of the wrong place for this, but given the burst of attention from LLM-loving people: is there any open source chat scaffolding that actually provides a good UI for organizing chat streams and doing stuff with them? A trivial example is how the LHS of the ChatGPT UI only allows you a handful of characters to name your chat, and you can't even drag the pane to the right to make it bigger; so I have all th…

Also natural language search of the chat history would be great.

Re: New models and developer products

#130

Post by nextworddev »

shanusmagnus wrote: post #116

This is kind of the wrong place for this, but given the burst of attention from LLM-loving people: is there any open source chat scaffolding that actually provides a good UI for organizing chat streams and doing stuff with them? A trivial example is how the LHS of the ChatGPT UI only allows you a handful of characters to name your chat, and you can't even drag the pane to the right to make it bigger; so I have all th…

Organize how?
Post reply on HN