Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New models and developer products

openai.com

Post reply on HN
Print view View original

351–360 of 561 posts

Re: New models and developer products

#351

Post by jairuhme »

Aeolun wrote: post #329
lolinder wrote: post #318

Earlier quoted context omitted.

> However, from a code perspective, it is a function of the key and a relatively small API. You're thinking of traditional apps and APIs. In an AI application, most of the work is in prompt engineering, not wiring up the API to your app. Prompts that work well for one model will fail horribly for another. People spend months refining their prompts before they're safe to share with users, and switching platforms will…

I’d be more worried about this if OpenAI had a track record of increasing prices, but the opposite happens. I get more for the same price basically every 6 months.

I'd imagine that will start to switch back the other way at some point. Decrease prices to gain market share and get you locked in, then increase prices to earn more money to keep VC's happy

Re: New models and developer products

#353

Post by modeless »

ianbicking wrote: post #331
modeless wrote: post #61

Earlier quoted context omitted.

You're right, I think that's exactly what happened. Silence is when you get the most hallucinations. But there is a trick supported by some implementations that helps a lot. Whisper does have a special token that it predicts for silence. You can look at the probability of that token even when it's not picked during sampling. Hallucinations often have a relatively high probability for the nospeech token compared to ac…

This is what you mean? https://apps.microsoft.com/detail/9NC624PBFGB7 But I don't have Windows :(

I plan to do a Linux build when I have time and release it as a Snap or Flatpak or Appimage or something. But it relies on an Nvidia GPU, so I can't release a macOS version right now. I could probably put something together with llama.cpp and whisper.cpp but it wouldn't be as fast, even on Apple's best. I'll probably do it eventually along with AMD support using ROCm or Vulkan, but I have a whole lot of other things to get to first and very limited time to work on it.

Right now my target is people with high end gaming PCs, because they can have a really good experience with the right software but most AI stuff is ridiculously hard to install. My goal is one click install with no required dependencies.

Re: New models and developer products

#354

Post by wokwokwok »

edgyquant wrote: post #343
lolinder wrote: post #318

Earlier quoted context omitted.

> However, from a code perspective, it is a function of the key and a relatively small API. You're thinking of traditional apps and APIs. In an AI application, most of the work is in prompt engineering, not wiring up the API to your app. Prompts that work well for one model will fail horribly for another. People spend months refining their prompts before they're safe to share with users, and switching platforms will…

Switching from one API to another generally requires refactoring. I’ve not had much problems moving between LLMs (openai to Anthropic)

Then you’re either not testing your prompts or doing something trivial.

Remember: a good model with a good prompt will generate bad outputs sometimes.

A bad model with a bad prompt will generate a good output sometimes.

That is simply a fact with these non deterministic models.

You have to do many iterations for each prompt to verify they are working correctly.

> I’ve not had much problems moving between LLMs…

If you want to move your prompts to a different model, you’re effectively replacing one:

f(prompt + seed) => output

With different black box implementation.

Unless you’re measuring the output over multiple iterations of (seed) and verifying your prompt still does the right thing, it’s actually very likely that what you’ve done if take an application with a known output space and converted it to an application with an unknown output space…

…that partially overlaps the original output space!

So it looks like it’s the same.

…but it isn’t, and the “isn’t” is in weird edge cases.

Unless you’re measuring that, you simply now have an app that does “eh, who knows?”

So yes. Porting is trivial if you don’t care if you have the same functionality.

…but reliably porting is much harder (or longer).

Re: New models and developer products

#355

Post by modeless »

azinman2 wrote: post #312
modeless wrote: post #61

Earlier quoted context omitted.

You're right, I think that's exactly what happened. Silence is when you get the most hallucinations. But there is a trick supported by some implementations that helps a lot. Whisper does have a special token that it predicts for silence. You can look at the probability of that token even when it's not picked during sampling. Hallucinations often have a relatively high probability for the nospeech token compared to ac…

So why doesn't the model score that higher then? I'm guessing there's an inherent trade off and they picked/trained it with enough silence vs non-silence?

Low quality training data, almost certainly.

Re: New models and developer products

#356

Post by ethbr1 »

keyle wrote: post #327
swatcoder wrote: post #287

Earlier quoted context omitted.

Everybody's got their own calculus about how competitive their space is and what this tech can do for them, but some might be best off dancing around lock-in by being careful about what they use from OpenAI and how tightly they integrate with it. This is very early in the maturity cycle for this tech. The options that will be available for private inference and fine tuning, for cloud-gpu/timeshare inference and fine…

While I agree with you, as a happy GPT4 plus customer, I'm worried about the inevitable enshittification downhill roll that will eventually ensue. Once marketing gets in charge of product, it's doomed. And I can't think of a product startup that it hasn't happened to. Particularly with this type of growth, at some point, the suits start to out number the techies 10:1. This is why openeness and healthy competition is…

It's not marketing, it's economics.

If you set money on fire -- eventually there's a time when you need to stop doing that.

Re: New models and developer products

#357

Post by famouswaffles »

anotherpaulg wrote: post #83
tornato7 wrote: post #78

According to [1], the new gpt-4-1106-preview model should be available to all, but the API is telling me "The model `gpt-4-1106-preview` does not exist or you do not have access to it." Anyone able to call it from the API? 1. https://help.openai.com/en/articles/8555510-gpt-4-turbo

Same. I am eager to run my code editing benchmark [1] against it, to compare it with gpt-4-0314 and gpt-4-0613. Edit: Ha, I just re-read the announcement [2] and it says 1pm in the 5th sentence: We’ll begin rolling out new features to OpenAI customers starting at 1pm PT today. [1] https://aider.chat/docs/benchmarks.html [2] https://openai.com/blog/new-models-and-developer-products-an...

Hey. Would really love to know the results of your benchmark testing.

Re: New models and developer products

#358

Post by famouswaffles »

Topfi wrote: post #56

I am very much looking forward to, but also dreading, testing gpt-4-turbo as part of my workflow and projects. The lowered cost and much larger context window are very attractive; however, I cannot be the only one who remembers the difference in output quality and overall perceived capability between gpt-3.5 and gpt-3.5-turbo, combined with the intransparent switching from one model to the other (calling the older, o…

Would really love to know the results of your benchmark testing.

Re: New models and developer products

#359

Post by gumballindie »

atleastoptimal wrote: post #324

Given that their main goal is still AGI, how does offering better developer tools and nifty custom models that can look at your dog for you help? Is it just bolstering revenue? They said they don't use API input to train their models so it isn't making them constantly smarter via more people using them.

AGI is what they’ll use to motivate investment in their company. A never reaching goal that promises to deliver growth at some point in the next two decades. That will provide funding to make existing models useful to more than just an over enthusiastic market. If they fail no problem, they “never really meant to make chatgpt work because their goal has always been agi”.

Re: New models and developer products

#360

Post by crosen99 »

atleastoptimal wrote: post #324

Given that their main goal is still AGI, how does offering better developer tools and nifty custom models that can look at your dog for you help? Is it just bolstering revenue? They said they don't use API input to train their models so it isn't making them constantly smarter via more people using them.

They stumbled into a position where they can make a crap ton of money going up the stack, which can fund the ongoing march toward AGI. (The revenue not only is cash in their pocket, but it’s also driving up their evaluation for future investment.)
Post reply on HN