Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New models and developer products

openai.com

Post reply on HN
Print view View original

281–290 of 561 posts

Re: New models and developer products

#281

Post by danShumway »

bluecrab wrote: post #152
burcs wrote: post #8

Earlier quoted context omitted.

Do you see a way around prompt injection? It feels like any feature they release is going to be susceptible to it.

Use an llm to evaluate the input and categorise it.

This gets suggested in every single conversation about LLMs, but I've never seen a working demo of chained-LLM safety measures that has managed to stand up to public access.

I feel fairly confident at this point that chained LLMs aren't a solution to prompt injection.

And with the number of open and free models available, we're at a point now where people claiming that there's an easy fix for prompt injection need to prove it. If it's this easy to fix, then build a working demo that can't be beaten by public attackers.

Re: New models and developer products

#282

Post by ushakov »

zizee wrote: post #258

In people's experience with these sorts of tools, have they assisted with maintainance of codebases? This might be directly, or indirectly via more readable, bette organized code. The reason I ask is that these tools seem to excel in helping to write new code. In my experience I think there is an upper limit to the amount of code a single developer can maintain. Eventually you can't keep everything in your head, so m…

We are doing this for API-Testing now. You should check out our website

https://ai.stepci.com

Re: New models and developer products

#283

Post by reitzensteinm »

anotherpaulg wrote: post #83
tornato7 wrote: post #78

According to [1], the new gpt-4-1106-preview model should be available to all, but the API is telling me "The model `gpt-4-1106-preview` does not exist or you do not have access to it." Anyone able to call it from the API? 1. https://help.openai.com/en/articles/8555510-gpt-4-turbo

Same. I am eager to run my code editing benchmark [1] against it, to compare it with gpt-4-0314 and gpt-4-0613. Edit: Ha, I just re-read the announcement [2] and it says 1pm in the 5th sentence: We’ll begin rolling out new features to OpenAI customers starting at 1pm PT today. [1] https://aider.chat/docs/benchmarks.html [2] https://openai.com/blog/new-models-and-developer-products-an...

I'm also eager for you to run your code editing benchmark against it. :)

Re: New models and developer products

#284

Post by w-m »

throw2321 wrote: post #268
zizee wrote: post #258

In people's experience with these sorts of tools, have they assisted with maintainance of codebases? This might be directly, or indirectly via more readable, bette organized code. The reason I ask is that these tools seem to excel in helping to write new code. In my experience I think there is an upper limit to the amount of code a single developer can maintain. Eventually you can't keep everything in your head, so m…

I've been thinking about this for a while now, wrt two points: 1. This will be the end of traditional SWEs and the rise of the age of debuggers, human debuggers who spend their days setting up breakpoints and figuring bugs in a sea of LLM generated code. 2. Hiring will switch from using Leetcode questions to "pull out your debugger and figure out what's wrong with this code".

What makes you think the LLM couldn’t run a debugging session from the content of a JIRA ticket and the whole code base + documentation?

Re: New models and developer products

#285

Post by abound »

aantix wrote: post #111

Can I pay someone to have my ChatGPT transcripts searchable?

Probably not the answer you're looking for, but the web UI has chat history export built-in, and from there you could search it yourself with local tools (plain grep, or more ElasticSearch-like engines), or use the new 128k context to ask questions of your chat history with GPT-4 (though that seems a bit, recursive?)

Re: New models and developer products

#286

Post by lucubratory »

freedomben wrote: post #48
WanderPanda wrote: post #28

Does anyone have an idea why they are so open about Whisper? Is it the poster child project for OAI people scratching their open source itch? Is there just no commercial value in speech to text?

I've been wondering this as well. I'm super glad, but it seems so different than every other thing they do. There's definitely commercial value, so I find it surprising.

I think it makes more sense to just consider why they're even building it. Their goal is to build an AGI, for which they think they need data and compute. They need market reach and revenue to make data access feasible and open investor's wallets for compute, and anything that makes the data easier to get and isn't too hard to do is going to help them on their main goal. Whisper being as widely available as possible is going to result in a lot more human origin language, not just in their services that are trainable, but on the web as a whole. Releasing whisper does basically nothing to increase output of machine generated text, and increases the amount of human text on the internet, so it's a net win. The actual calculation is then going to be on how hard it is to make, and my guess is that for the top AI research team in the world with Microsoft resources, it turned out to be a pretty easy problem to comprehensively solve.

Re: New models and developer products

#287

Post by swatcoder »

faeriechangling wrote: post #51
minimaxir wrote: post #4

Most of the products announced (and the price cuts) appear to be more about increasing lock-in to the OpenAI API platform, which is not surprising given increased competition in the space. The GPTs/GPT Agents and Assistants demos in particular showed that they are a black box within a black box within a black box that you can't port anywhere else. I'm mixed on the presentation and will need to read the fine print on…

It's a good strategy. For me, avoiding the moat means either a big drop in quality and just ending up in somebody elses moat, or a big drop in quality and a lot more money spent. I've looked into it and maybe the most practical end-to-end system for owning my own LLM is to run a couple of 3090s on a consumer motherboard at substantial running cost to keep them up 24/7 and that's not powerful enough to cut it and rath…

Everybody's got their own calculus about how competitive their space is and what this tech can do for them, but some might be best off dancing around lock-in by being careful about what they use from OpenAI and how tightly they integrate with it.

This is very early in the maturity cycle for this tech. The options that will be available for private inference and fine tuning, for cloud-gpu/timeshare inference and fine tuning, and for competing hosted solutions are going to vastly different as months go by. What looks like squeezing value out of OpenAI today might look a lot like technical debt and frustrating lock-in a year from now.

That's what they're hoping you chase after, and if your product is defined by this technology, maybe that's what you have to do. But if you're just thinking about feature opportunities for a more robust product, judiciousness could pay off better than rushing. For now.

Re: New models and developer products

#288

Post by seydor »

whytai wrote: post #71

Every day this video ages more and more poorly [1]. categories of startups that will be affected by these launches: - vectorDB startups -> don't need embeddings anymore - file processing startups -> don't need to process files anymore - fine tuning startups -> can fine tune directly from the platform now, with GPT4 fine tuning coming - cost reduction startups -> they literally lowered prices and increased rate limits…

more startups should focus on foundation models, it's where the meat is. Ideally there won't be a need for any startup as the platform should be able to self-build whatever the customer wants.

Re: New models and developer products

#289

Post by obmelvin »

visarga wrote: post #274
riku_iki wrote: post #162

Earlier quoted context omitted.

> - vectorDB startups -> don't need embeddings anymore they don't provide embedings, but storage and query engines for embeddings, so still very relevant > - file processing startups -> don't need to process files anymore curious what is that exactly?.. > - vertical ai agent startups -> GPT marketplace sure, those startups will be selling their agents on marketplace

It's easy to host your query engine somewhere else and integrate it as a search function in chatGPT. Quite easy to switch providers of search.

As in, use an existing search and call it via 'function calling' as part of the assistants routine - rather than uploading documents to the assistant API?

Re: New models and developer products

#290

Post by goosinmouse »

minimaxir wrote: post #123
ebiester wrote: post #59

Earlier quoted context omitted.

I don't understand the lock-in argument here. Yes, if a competitor comes in there will be switching cost as everything is re-learned. However, from a code perspective, it is a function of the key and a relatively small API. New regulations outstanding, what is stoping someone from moving from OpenAI to Anthropic (for example) other than the cost of learning how to effectively utilize Anthropic for your use case? Open…

> OpenAI doesn't have some sort of egress feed for your database. That's what they're trying to incentivize, especically with being able to upload files for their own implementation of RAG. You're not getting the vector representation of those files back, and switching to another provider will require rebuilding and testing that infrastructure.

Thats exactly what i thought. Smart strategy on OpenAI's part given that its extremely easy (and free) to do RAG with pgvector.
Post reply on HN