Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New models and developer products

openai.com

Post reply on HN
Print view View original

341–350 of 561 posts

Re: New models and developer products

#341

Post by dragonwriter »

lazzlazzlazz wrote: post #91
whytai wrote: post #71

Every day this video ages more and more poorly [1]. categories of startups that will be affected by these launches: - vectorDB startups -> don't need embeddings anymore - file processing startups -> don't need to process files anymore - fine tuning startups -> can fine tune directly from the platform now, with GPT4 fine tuning coming - cost reduction startups -> they literally lowered prices and increased rate limits…

Embeddings are still important (context windows can't contain all data + memorization and continuous retraining is not yet viable), and vertical AI agent startups can still lead on UX.

Separate embedding DBs are less important if you are working with OpenAI, since their Assistants API exists to (among other things) let you bring in additional data and let them worry about parsing it, storing it, and doing RAG with it. Its like "serverless", but for Vector DBs and RAG implementations instead of servers.

Re: New models and developer products

#342

Post by lemmsjid »

constantly wrote: post #321
govg wrote: post #263

Earlier quoted context omitted.

Is it the same as avoiding AWS because they will take your software and run it themselves to steal your clients?

It’s more like running a PaaS product backed by AWS and then your customers realizing they can just use AWS directly, pay less, and have less complexity. And they have done this before for what it’s worth.

The assumption behind why you'd use the program is that you have access to a proprietary dataset of sufficient size to build a large model around (say, for example, call center transcripts). This almost certainly means OpenAI doesn't have access to train their other models off that data, and it almost certainly means your customers can't take the same data and go straight to OpenAI.

I'm assuming that the target customer of this is people whose moat is proprietary data. If their moat is a unique approach to building a model, then it would indeed be dangerous to engage OpenAI. But then I'd think OpenAI would be hesistent to engage as well.

Re: New models and developer products

#343

Post by edgyquant »

lolinder wrote: post #318
ebiester wrote: post #59

Earlier quoted context omitted.

I don't understand the lock-in argument here. Yes, if a competitor comes in there will be switching cost as everything is re-learned. However, from a code perspective, it is a function of the key and a relatively small API. New regulations outstanding, what is stoping someone from moving from OpenAI to Anthropic (for example) other than the cost of learning how to effectively utilize Anthropic for your use case? Open…

> However, from a code perspective, it is a function of the key and a relatively small API. You're thinking of traditional apps and APIs. In an AI application, most of the work is in prompt engineering, not wiring up the API to your app. Prompts that work well for one model will fail horribly for another. People spend months refining their prompts before they're safe to share with users, and switching platforms will…

Switching from one API to another generally requires refactoring. I’ve not had much problems moving between LLMs (openai to Anthropic)

Re: New models and developer products

#344

Post by dragonwriter »

riku_iki wrote: post #162
whytai wrote: post #71

Every day this video ages more and more poorly [1]. categories of startups that will be affected by these launches: - vectorDB startups -> don't need embeddings anymore - file processing startups -> don't need to process files anymore - fine tuning startups -> can fine tune directly from the platform now, with GPT4 fine tuning coming - cost reduction startups -> they literally lowered prices and increased rate limits…

> - vectorDB startups -> don't need embeddings anymore they don't provide embedings, but storage and query engines for embeddings, so still very relevant > - file processing startups -> don't need to process files anymore curious what is that exactly?.. > - vertical ai agent startups -> GPT marketplace sure, those startups will be selling their agents on marketplace

> they don't provide embedings, but storage and query engines for embeddings, so still very relevant

But you don't need any of the chain of: extract data, calculate embeddings, store data indexed by embeddings, detect need to retrieve data by embeddings and stuff it into LLM context along with your prompt if you use OpenAI's Assistants API, which, in addition to letting you store your own prompts and manage associated threads, also lets you upload data for it to extract, store, and use for RAG on the level of either a defined Assistant or a particular conversation (Thread.)

Re: New models and developer products

#345

Post by nmfisher »

jack_riminton wrote: post #253

For all the naysayers in the comments, the elephant in the room that no one quite wants to admit, is that GPT4 is still far better than everything else out there

I cancelled my GPT4 subscription because I found Claude more useful for code and Qwen for Chinese language tasks.

It might be better on average but I don’t think it’s better for every task.

All the others are only going to get better too.

Re: New models and developer products

#346

Post by dragonwriter »

minimaxir wrote: post #18
burcs wrote: post #8

Earlier quoted context omitted.

Do you see a way around prompt injection? It feels like any feature they release is going to be susceptible to it.

I suspect OpenAI's black box workflow has some safeguards for it.

The needed safeguards are almost certainly very much app specific, since if you are working with private data at all, its going to be intended to influence the output and behavior in some ways but not in others, and those ways are themselves app dependent;

Re: New models and developer products

#348

Post by dragonwriter »

nwoli wrote: post #229
gwern wrote: post #103

> We’re also launching a feature to return the log probabilities for the most likely output tokens generated by GPT-4 Turbo and GPT-3.5 Turbo in the next few weeks, which will be useful for building features such as autocomplete in a search experience. This is very surprising to me. Are they not worried about people not just training on GPT-4 outputs to steal the model capabilities, but doing full blown logit knowled…

I guess the EO takes care of that in their eyes (outlawing open models). They’re probably right too

> I guess the EO takes care of that in their eyes (outlawing open models).

The EO doesn't do anything even approximately like outlawing open models.

Re: New models and developer products

#349

Post by infecto »

sbohacek wrote: post #203
infecto wrote: post #26

Earlier quoted context omitted.

Anthropic never even had a day. I said this before in another Anthropic thread but I signed up 6 months ago for API access and they never responded. An employee in that thread apologized and said to try again, did it, week later still nothing. As far as commercial viability, they never had it.

I have not tried, but I assumed that API access to Anthropic's Claude is available through AWS Bedrock.

Which I think is the case now but the beta access to Claude 2 has had a signup on their site for months. I am not as interested in having to go through AWS Bedrock before even experience the potential performance of the API. I give a lot of praise to OpenAI for how quickly they are both scaling and releasing.

Re: New models and developer products

#350

Post by infecto »

taf2 wrote: post #92
infecto wrote: post #26

Earlier quoted context omitted.

Anthropic never even had a day. I said this before in another Anthropic thread but I signed up 6 months ago for API access and they never responded. An employee in that thread apologized and said to try again, did it, week later still nothing. As far as commercial viability, they never had it.

I got access to Claude 2 - it’s really good and have been chatting with their sales team. Seems they were reasonably responsive- but overall with OpenAI 128k context and price anthropic has no edge

Maybe my company address is not good enough. Never heard from them unfortunately. Tried a couple times.
Post reply on HN