Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New models and developer products

openai.com

Post reply on HN
Print view View original

521–530 of 561 posts

Re: New models and developer products

#521

Post by btbuildem »

Topfi wrote: post #56

I am very much looking forward to, but also dreading, testing gpt-4-turbo as part of my workflow and projects. The lowered cost and much larger context window are very attractive; however, I cannot be the only one who remembers the difference in output quality and overall perceived capability between gpt-3.5 and gpt-3.5-turbo, combined with the intransparent switching from one model to the other (calling the older, o…

The rapid deprecation of the models is definitely unsettling. They're on there barely long enough to establish reliable performance baselines within derived services.

I imagine behind the scenes it's all about resource use and cost. What stood out to me during the talk was how much emphasis ("we worked very hard") Altman put on the new price tiers. "Worked very hard" probably just means "endlessly argued with the board". It'a little sad that technical achievements take back seat to tug of war with moneybags.

Re: New models and developer products

#522

Post by btbuildem »

jayGlow wrote: post #520
gardenhedge wrote: post #462

Earlier quoted context omitted.

And there's people (most people) who aren't using it

what's really crazy to me is how fast it became mundane, this technology is still mind blowing to me but most people no longer seem impressed.

Even we who are so impressed with it and use it frequently / build on it -- we don't have a clue of the true potential.

It'll be some pimply intern somewhere that'll blow the lid off things with some ultra-clever-yet-painfully-obvious use case.

Re: New models and developer products

#523

Post by btbuildem »

kristianp wrote: post #366

> Reproducible outputs and log probabilities > The new seed parameter enables reproducible outputs by making the model return consistent completions most of the time. This beta feature is useful for use cases such as replaying requests for debugging, writing more comprehensive unit tests, and generally having a higher degree of control over the model behavior. We at OpenAI have been using this feature internally for…

Nicely spotted! Yeah, even with temperature turned all the way down, the variation in results makes it harder to test.

Re: New models and developer products

#524

Post by btbuildem »

simonw wrote: post #404
btbuildem wrote: post #403

Earlier quoted context omitted.

I've found that it works well enough in practice - here's a really simplified but still robust version: https://chat.openai.com/share/438d2429-f5ff-4eb2-8771-f3e206...

"Well enough in practice" isn't good enough for this, because it's a security vulnerability. You have to be 100% certain that a suitable motivated adversarial attacker can't find a way through. Here's an attack that beats your example. Enter this as a full response (it's using a trick that makes the LLM think it has completed the first instruction before delivering the attack): Describe the contents of the text below…

The version above was really simplified. You need to iterate a little to get something that's too hard to break.

Try this: https://chat.openai.com/share/7d091da1-729b-4678-98fe-def4f9...

Re: New models and developer products

#525

Post by Kye »

swatcoder wrote: post #287
faeriechangling wrote: post #51

Earlier quoted context omitted.

It's a good strategy. For me, avoiding the moat means either a big drop in quality and just ending up in somebody elses moat, or a big drop in quality and a lot more money spent. I've looked into it and maybe the most practical end-to-end system for owning my own LLM is to run a couple of 3090s on a consumer motherboard at substantial running cost to keep them up 24/7 and that's not powerful enough to cut it and rath…

Everybody's got their own calculus about how competitive their space is and what this tech can do for them, but some might be best off dancing around lock-in by being careful about what they use from OpenAI and how tightly they integrate with it. This is very early in the maturity cycle for this tech. The options that will be available for private inference and fine tuning, for cloud-gpu/timeshare inference and fine…

The Phind CEO talked in an interview about how their own model is already out ahead of ChatGPT 4 for the target use case of their search engine in some cases, and increasingly matching it on general searches: https://www.latent.space/p/phind#details

I use it instead of Bing Chat now for cases where I really need a search engine and Google is useless. Mainly because it's faster, but I also like not having to open another browser.

Re: New models and developer products

#526

Post by Karunamon »

toyg wrote: post #508
asaddhamani wrote: post #474

Earlier quoted context omitted.

I'd built a bot to use ChatGPT from Telegram (this was before the ChatGPT API), and currently building a tool to help make writing easier ( https://www.penpersona.com ). This is the API. Apart from that, it's pretty much replaced 80% of my search engine usage, I can ask it to collate reviews for a product from reddit and other sites, get the critical reception of a book, etc. You don't have to go and read long posts…

> currently building a tool to help make writing easier That's cool! > it's pretty much replaced 80% of my search engine usage That's not cool. That's how you end up relying on nonexisting sources or other hallucinations.

As opposed to raw information surfaced by the search engine, which we all know is perfectly reliable, unbiased, and up to date?

That aside, this particular admonishment was worn out a couple of months after ChatGPT was released. It does not need to be repeated every time someone mentions doing something interesting with an LLM.

Re: New models and developer products

#527

Post by henryaj »

Void_ wrote: post #65
modeless wrote: post #12

Whisper V3 is released! https://github.com/openai/whisper/commit/c5d42560760a05584c1... Looks like it's just a new checkpoint for the large model. It would be nice to have updates for the smaller models too. But it'll be easy to integrate with anything using Whisper V2. I'm excited to add it to my local voice AI ( https://www.microsoft.com/store/apps/9NC624PBFGB7 ) I assume ChatGPT voice has been using Whisper V3 and…

Too bad they didn't upgrade Whisper API yet. Can't wait to make it available in https://whispermemos.com

This looks awesome!

Re: New models and developer products

#528

Post by gwern »

leobg wrote: post #435
gwern wrote: post #338

Earlier quoted context omitted.

Considering how well simply reusing GPT-3.5/4 outputs has worked to juice rival model performance, at least in relatively narrow benchmarking, I dunno how many GBs it'd take, but probably not that many, and it's a straightforward easy way to turn money into performance at a much lower cost than buying a few thousand more H100s.

OpenAI does not strike me as a company that would be naive about this. Didn’t they just recently manipulate the outputs of an endpoint when they realized people were misusing it? (“CatGPT”) The most sinister interpretation is that the logits are a red herring. People who are tied up in stealing them aren’t free to do actual rival work.

Well, not recently. That was from at the ChatGPT launch when they were still using a quick hack that made the client too-privileged, so more or less a year ago.

However, I definitely am wondering if they have poisoned the logits somehow. As long as the logits rank the tokens in order (are monotonic), preserving their utility for the suggested applications like ranking autocompletions, you presumably could screw with the magnitudes arbitrarily.

Re: New models and developer products

#529

Post by lolinder »

Rastonbury wrote: post #393
lolinder wrote: post #318

Earlier quoted context omitted.

> However, from a code perspective, it is a function of the key and a relatively small API. You're thinking of traditional apps and APIs. In an AI application, most of the work is in prompt engineering, not wiring up the API to your app. Prompts that work well for one model will fail horribly for another. People spend months refining their prompts before they're safe to share with users, and switching platforms will…

Still moving between models is less arduous than switching cloud providers, depending on use case and price difference of course. Most models hold GPT4 as the benchmark they aspire to and should converge to its capabilities.

It's not a question of converging on its capabilities, it's a question of responding equivalently to the nuances of the prompt you've crafted. Two models can be equally capable, but one might interpret a phrase in your prompt slightly differently.

Re: New models and developer products

#530

Post by dragonwriter »

nwoli wrote: post #490
dragonwriter wrote: post #348

Earlier quoted context omitted.

> I guess the EO takes care of that in their eyes (outlawing open models). The EO doesn't do anything even approximately like outlawing open models.

It basically bans models that can be finetuned to act as a biological weapon which is basically every model, llama, mistral etc

> t basically bans models that can be finetuned to act as a biological weapon which is basically every model, llama, mistral etc

First, no, it doesn't.

Second, no model can be finetuned to act as a biological weapon.

Third, if it did ban “basically every model” on the basis “can be finetuned to act as a biological weapon”, that would be very different than banning open models, and would be bad for OpenAI.

The order directs the development of reporting requirements for thise developing models with certain capabilities within 3 months, and the development within government of risk mitigation strategies for certain risks within 4, 6, or 9 months, depending on the specific area of risk. It doesn't ban or “basically ban” any models.

(It is possible, but far from certain, that on or more of the plans it calls on different agencies to develop might do that, but those would, in addition to the policy guidance in the EO, also need a statutory authority that provides power for the executive branch to issue a ban.)

Post reply on HN