Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 API General Availability

openai.com

Post reply on HN
Print view View original

471–480 of 562 posts

Re: GPT-4 API General Availability

#471

Post by zarzavat »

andsoitis wrote: post #452
twbarr wrote: post #278

Earlier quoted context omitted.

No, but what is it? Not your lawyer, not legal advice, but it's not a trade secret, they've given it to researchers. It's not a trademark because it's not an origin identifier. The structure might be patentable, but the weights won't be. It's certainly not a mask work. It might have been a contract violation for the guy who redistributed it, but I'm not a party to that contract.

> It might have been a contract violation for the guy who redistributed it, but I'm not a party to that contract. Wouldn’t that violate the Nemo dat quod non habet legal principle and so you cannot hide behind the claim that you weren’t party to the contact? https://en.wikipedia.org/wiki/Nemo_dat_quod_non_habet

No because the weights are not IP protected by the entity that trained the model, so they cannot prevent you to redistribute it because it doesn’t belong to them in any legal sense. GPU cycles alone don’t make IP.

The contracts in these cases are somewhat similar to an NDA, without the secrecy aspect. Restricted disclosure of public information. You can agree to such a contract if you want to, and a court might even enforce it, but it doesn’t affect anybody else’s rights to distribute that information.

Contracts are not statutes, they only bind the people directly involved. To restrict the actions of random strangers, you need to get elected.

Re: GPT-4 API General Availability

#472

Post by weird-eye-issue »

ftxbro wrote: post #114
jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

After one of the ubuntu snap updates my firefox stopped working with OpenAI API playground it worked still with every other site. I retried and restarted so many times and it didn't work. Eventually I switched browser to chromium and it worked. I still don't know the problem and it was unnerving, I would have a lot of anxiety to build something important with it. I tried again just now and I got "Oops! We ran into an…

Lmao. You had a browser issue when running Firefox on Linux (.000001% of users) and now you are making connections between that and their API stability?

Re: GPT-4 API General Availability

#473

Post by jwr »

weird-eye-issue wrote: post #469
jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

I built a production app on top of OpenAI and yeah there are frequent errors and timeouts but you literally just have to add some code to account for these and it works fine... For example exponential back off is the first step, then adding retrying on timeouts (we use streaming and if there are 30 seconds in between getting data back we retry the whole request - rare but happens), then fixing anything else that pops…

> Just fix your code and stop expecting OpenAI to hold your hand

:-)

My code does retry and the entire application is written to detect and work around breakage. But eventually I do need to get enough content from OpenAI API to be able to make progress, and I am not.

At the moment, for example, all requests just time out after 12 minutes (on my side). No amount of "fixing my code" will help, and I don't want OpenAI to hold my hand, I just want it to a) return some data at least sometimes, b) not charge me for data not delivered.

Let's look at my billing page: over the last hour it shows 8 requests. A total of 52584 tokens. Not a single response made it back to me.

Re: GPT-4 API General Availability

#474

Post by ben_w »

ineedasername wrote: post #355
jiggawatts wrote: post #307

Earlier quoted context omitted.

There's no scaling issues to speak of. These AIs are stateless , which makes them embarrassingly parallel. They can always just throw more GPUs at it. Microsoft even had some videos where they bragged about how these models can be run on any idle GPU around the world, dynamically finding resources wherever it is available! If there's not enough GPUs at a certain price point, raise prices. Then lower prices later when…

> embarrassingly parallel I don’t see why such a thing should be embarrassing. Or, at least no more so than being acute or obtuse. Just as long as nothing is askew.

"Embarrassingly parallel" is a term of art: https://en.wikipedia.org/wiki/Embarrassingly_parallel

Re: GPT-4 API General Availability

#475

Post by weird-eye-issue »

hobs wrote: post #148
nunodonato wrote: post #100

Earlier quoted context omitted.

if you want to use it in prod, go with Azure

And get only 20 K tokens per minute, where a decent size question can use up 500 tokens, pretty much a joke for most larger websites. https://learn.microsoft.com/en-us/azure/cognitive-services/o...

That's the default limit for GPT-4 which has more demand than any other LLM in the world.

Re: GPT-4 API General Availability

#476

Post by kozikow »

londons_explore wrote: post #252
alpark3 wrote: post #174

>Developers wishing to continue using their fine-tuned models beyond January 4, 2024 will need to fine-tune replacements atop the new base GPT-3 models (ada-002, babbage-002, curie-002, davinci-002), or newer models (gpt-3.5-turbo, gpt-4). Once this feature is available later this year, we will give priority access to GPT-3.5 Turbo and GPT-4 fine-tuning to users who previously fine-tuned older models. We acknowledge…

This tells me that either there were very few commercial users of finetuned models, or they need to decommission the infrastructure to free up GPU's for more valuable projects.

Just the models available for fine tunings are waay behind gpt4.

I have much better performance by "prompt tuning" - when question arises, I search 30 similiar examples in training set, and send it to non-tuned GPT and ask the question and get much better performance than fine-tuned older models.

Re: GPT-4 API General Availability

#477

Post by weird-eye-issue »

dahwolf wrote: post #277

Personally, I'm forever locked out of OpenAI. I had the silly idea of trying to change the signin method of my account. Which isn't possible. So I figured to just delete the account and create a new one with the correct signin method. Turns out they don't delete anything. Both the email address and phone number are held hostage. As you try to create a new account, it will point out that those are in use. I can easily…

Stop making this so difficult. Just use a different phone number. There are sites to do this for literally $0.10.

But yeah whining on HN would be more productive

Re: GPT-4 API General Availability

#478

Post by weird-eye-issue »

ChicagoDave wrote: post #364
eiiot wrote: post #360

Earlier quoted context omitted.

Yes, that’s an artificial limitation put in place so people don’t abuse plus.

So the chat interface is rate-limited, but not the API. So I could build long prompt scripts and run them against the api?

What do you think the API is for?

Re: GPT-4 API General Availability

#479

Post by joaogante »

bredren wrote: post #385
PostOnce wrote: post #197

Promote and proliferate local LLMs. If you use GPT, you're giving OpenAI money to lobby the government so they'll have no competitors, ultimately screwing yourself, your wallet, and the rest of us too. OpenAI has no moat, unless you give them money to write legislation. I can currently run some scary smart and fast LLMs on a 5 year old laptop with no GPU. The future is, at least, interesting.

Running an LLM locally and paying for access to OpenAI are two separate concerns. But to address both: is it very relevant what LLM you use right now? Local or hosted, openAI or other? It seems like the interface has converged around chat-based prompts. New ideas for tuning or improving the efficiency of foundational models are published almost every week. If one wants to build a product on top of of generative AI, w…

> Couldn’t someone say prototype the entire product on some lower-quality LLM and occasionally pass requests to GPT4 to validate behavior?

It can be done -- it is the basis for assisted generation and related work. It does require full access to the model, to be time and money-efficient. See https://huggingface.co/blog/assisted-generation

Disclaimer: I'm the author of the blog post linked above.

Re: GPT-4 API General Availability

#480

Post by Gasp0de »

lhl wrote: post #446
andai wrote: post #273

Earlier quoted context omitted.

Can you elaborate on scary smart and fast? It's been a month or two since I've tried but the results were depressingly slow and useless for more or less every task I tried. Every time a model is claimed to be "90% of GPT-3" I get excited and every time it's very disappointing. (On that note, after using GPT-4, GPT-3 now seems disappointing almost every time I interact with it.)

In terms of speed, we're talking about 140t/s for 7B models, and 40t/s for 33B models on a 3090/4090 now.[1] (1 token ~= 0.75 word) It's quite zippy. llama.cpp performs close on Nvidia GPUs now (but they don't have a handy chart) and you can get decent performance on 13B models on M1/M2 Macs. You can take a look at a list of evals here: https://llm-tracker.info/books/evals/page/list-of-evals - for general usage, I th…

Those GPUs are 1200$ and upwards. This is equivalent to 20,000,000 tokens on GPT-4. I don't think I will ever use this many tokens for my personal use.
Post reply on HN