It is interesting that the updates are largely developer experience updates. It doesn't appear that significant innovations are happening on the core models outside of performance/cost improvements. Both devex and perf/cost are important to be sure, but incremental.
New models and developer products
171–180 of 561 posts
Re: New models and developer products
#172Does anyone know when this will be coming to Azure OpenAI?
If Azure's history when rolling out GPT-4 is any indication, probably a couple months and/or a staged rollout.
Re: New models and developer products
#173Every day this video ages more and more poorly [1]. categories of startups that will be affected by these launches: - vectorDB startups -> don't need embeddings anymore - file processing startups -> don't need to process files anymore - fine tuning startups -> can fine tune directly from the platform now, with GPT4 fine tuning coming - cost reduction startups -> they literally lowered prices and increased rate limits…
TBH those are low-hanging fruits for OpenAI. Much of the value still being captured by OpenAI's own model.
The sad thing is, GPT-4 is its own league in the whole LLM game, whatever those other startups are selling, it isn't competing with OpenAI.
Re: New models and developer products
#174Is there a special "developer" designation? I am a paying API customer, but can't see gpt-4-1106-preview in the playground and can't use it via the API.
As other comments have noted it seems to be rolling out at 1pm PST today
Re: New models and developer products
#175> We’re also launching a feature to return the log probabilities for the most likely output tokens generated by GPT-4 Turbo and GPT-3.5 Turbo in the next few weeks, which will be useful for building features such as autocomplete in a search experience. This is very surprising to me. Are they not worried about people not just training on GPT-4 outputs to steal the model capabilities, but doing full blown logit knowled…
How many GBs worth of logits would you need to reverse engineer their model? Also, if it’s a conglomerate of models that they’re using, you’d end up in a blind alley.
Re: New models and developer products
#176It is interesting that the updates are largely developer experience updates. It doesn't appear that significant innovations are happening on the core models outside of performance/cost improvements. Both devex and perf/cost are important to be sure, but incremental.
presumably next model is coming next year?
Re: New models and developer products
#177Earlier quoted context omitted.
I haven't been paying attention, why are embeddings not needed anymore?
Retrieval: augments the assistant with knowledge from outside our models, such as proprietary domain data, product information or documents provided by your users. This means you don’t need to compute and store embeddings for your documents, or implement chunking and search algorithms. The Assistants API optimizes what retrieval technique to use based on our experience building knowledge retrieval in ChatGPT. The mod…
Really cool to see the Assistants API's nuanced document retrieval methods. Do you index over the text besides chunking it up and generating embeddings? I'm curious about the indexing and the depth of analysis for longer docs, like assessing an author's tone chapter by chapter—vector search might have its limits there. Plus, the process to shape user queries into retrievable embeddings seems complex. Eager to hear more about these strategies, at least what you can spill!
Re: New models and developer products
#178Most of the products announced (and the price cuts) appear to be more about increasing lock-in to the OpenAI API platform, which is not surprising given increased competition in the space. The GPTs/GPT Agents and Assistants demos in particular showed that they are a black box within a black box within a black box that you can't port anywhere else. I'm mixed on the presentation and will need to read the fine print on…
I think it's more about finding places to add value than "lock in" per se. It seems they're adding value with improved developer experience and cost/performance rather than on the models themselves. Not necessarily nefarious attempts to lock in customers, but it may have the same outcome :)
Re: New models and developer products
#179Every day this video ages more and more poorly [1]. categories of startups that will be affected by these launches: - vectorDB startups -> don't need embeddings anymore - file processing startups -> don't need to process files anymore - fine tuning startups -> can fine tune directly from the platform now, with GPT4 fine tuning coming - cost reduction startups -> they literally lowered prices and increased rate limits…
depends on how much developers are willing to embrace the risk of building everything on OpenAI and getting locked onto their platform. What's stopping OpenAI from cranking up the inference pricing once they choke out the competition? That combined with the expanded context length makes it seem like they are trying to lead developers towards just throwing everything into context without much thought, which could be p…
I suspect it is in OpenAI's interest to have their API as a loss leader for the foreseeable future, and keep margins slim once they've cornered the market. The playbook here isn't to lock in developers and jack up the API price, it's the marketplace play: attract developers, identify the highest-margin highest-volume vertical segments built atop the platform, then gobble them up with new software.
They can then either act as a distributor and take a marketplace fee or go full Amazon and start competing in their own marketplace.
Re: New models and developer products
#180Earlier quoted context omitted.
Comparing a 7B parameter model to a 1.8T parameter model is kind of silly. Of course it's behind on accuracy, but it also takes 1% of the resources.
The person I replied to had decided to compare Mistral to what was launched, so I went along with their comparison and showed how I have been unsatisfied with it. But, these open models can certainly be fun to play with. Regardless, where did you find 1.8T for GPT-4 Turbo? The Turbo model is the one with the 128K context size, and the Turbo models tend to have a much lower parameter count from what people can tell. N…
It's an order of magnitude comparison.
Let's just agree it's 100x-300x more parameters, and let's assume the open ai folks are pretty smart and have a sense for the optimal number of tokens to train on.