Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 API General Availability

openai.com

Post reply on HN
Print view View original

371–380 of 562 posts

Re: GPT-4 API General Availability

#371

Post by dcow »

slaymaker1907 wrote: post #338
jstummbillig wrote: post #222

Earlier quoted context omitted.

Just a heads up: If you are more interested in being effective than being an evangelist, beware. While you can run all kinds of GPTs locally, GPT-4 still smokes everything right now – and even it is not actually good enough to not be a lynchpin for a lot of cases yet.

I guess ignoring copyright and treating the whole internet as your training data does have its advantages.

Yes? That’s the point. Who cares about an outdated concept that has no digital analog? All the artists have moved on already #midjourney.

Re: GPT-4 API General Availability

#372

Post by PostOnce »

andai wrote: post #273
PostOnce wrote: post #197

Promote and proliferate local LLMs. If you use GPT, you're giving OpenAI money to lobby the government so they'll have no competitors, ultimately screwing yourself, your wallet, and the rest of us too. OpenAI has no moat, unless you give them money to write legislation. I can currently run some scary smart and fast LLMs on a 5 year old laptop with no GPU. The future is, at least, interesting.

Can you elaborate on scary smart and fast? It's been a month or two since I've tried but the results were depressingly slow and useless for more or less every task I tried. Every time a model is claimed to be "90% of GPT-3" I get excited and every time it's very disappointing. (On that note, after using GPT-4, GPT-3 now seems disappointing almost every time I interact with it.)

Different quantizations can give you a big speedup if you've had "depressingly slow" issues. Even the slowest ones (that fit in RAM) will run at basically interactive speed, not instant, but also not "email speed". I have a laptop with a 2018 CPU and I'm working with them just fine.

Text generation style instead of chat style is another avenue that makes the feedback time not so annoying for a developer.

at 100ms/token, it's faster than most people type, I think. That's what you might get on an old laptop with a 7B model.

There's a useful leaderboard here to help you pick a model: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...

It really depends on your task, lots and lots of natural language type tasks give great results, the models seem to have extensive knowledge of many fields. So for some kinds of Q&A bot (technical or not), for copy blurbs, for fiction, game NPCs, etc, the models (especially 13B and up) can be breathtaking, even moreso considering they run on bottom-dollar consumer hardware (I paid $250 for the laptop I'm developing on).

There are of course some things that neither the local LLMs nor GPT4 can do, like create useful OpenSCAD models :)

Things keep getting better, newer quantization methods give you more smarts in the same amount of RAM at basically the same speed -- the models are getting better, there are more permissively licensed ones now.

Re: GPT-4 API General Availability

#373

Post by Art9681 »

kristianp wrote: post #297
PostOnce wrote: post #197

Promote and proliferate local LLMs. If you use GPT, you're giving OpenAI money to lobby the government so they'll have no competitors, ultimately screwing yourself, your wallet, and the rest of us too. OpenAI has no moat, unless you give them money to write legislation. I can currently run some scary smart and fast LLMs on a 5 year old laptop with no GPU. The future is, at least, interesting.

Gpt-4 runs on 8 x 220B params[1] and gpt is about 220B params(?). Local LLMs can be good for some tasks, but they are much slower and less capable than the size of model and hardware that openai brings to their apis. Even running a 7B model on the CPU in ggml is much slower than the gpt-3-turbo api, in my experience with a 12th gen i7 intel laptop. [1] GPT4 is 8 x 220B params = 1.7T params: https://news.ycombinator.c…

It's been well documented by now that the number of parameters does not necessarily translate to a better model. My guess is that OpenAI has learned a thing or two from the endless papers published daily that your "instance" of the model is not what it seems. They likely have a workflow that picks the best model suitable for your prompt. Some people may get a 13B permutation because it is "good enough" to produce a common answer to a common prompt. Why waste precious compute resources on a prompt that is common? Would it not be feasible to collect the data of the top worldwide prompts and produce a small model that can answer those? Why would OpenAI spend precious compute time on the typical user's "write a short story of...".

I would guesstimate that the great majority of prompts are trash. People playing with a toy and amusing themselves. The platform sends those to the trash models.

For the other tiny percentage that produces a prompt the size of a paragraph, using the techniques published by OpenAI themselves, they likely get the higher tier models. This is also why I believe many are recently complaining about the quality of the outputs. When your chat history is filled with "have waifu pretend to be my girlfriend" then whatever memory the model is maintaining will be poisoned by the quality of your past prompts.

Garbage in, garbage out. I am certain that the #1 priority for OpenAI/Microsoft is lowering the cost of each prompt while satisfying the majority.

The majority is not in HN.

Re: GPT-4 API General Availability

#374

Post by PostOnce »

kristianp wrote: post #297
PostOnce wrote: post #197

Promote and proliferate local LLMs. If you use GPT, you're giving OpenAI money to lobby the government so they'll have no competitors, ultimately screwing yourself, your wallet, and the rest of us too. OpenAI has no moat, unless you give them money to write legislation. I can currently run some scary smart and fast LLMs on a 5 year old laptop with no GPU. The future is, at least, interesting.

Gpt-4 runs on 8 x 220B params[1] and gpt is about 220B params(?). Local LLMs can be good for some tasks, but they are much slower and less capable than the size of model and hardware that openai brings to their apis. Even running a 7B model on the CPU in ggml is much slower than the gpt-3-turbo api, in my experience with a 12th gen i7 intel laptop. [1] GPT4 is 8 x 220B params = 1.7T params: https://news.ycombinator.c…

https://mpost.io/phi-1-a-compact-language-model-outpaces-gpt...

a 1billion parameter model beats 175billion parameter GPT3.5

OpenAI wants us all to drink the kool-aid.

Re: GPT-4 API General Availability

#375

Post by ShadowBanThis01 »

npew wrote: post #287
alpark3 wrote: post #174

>Developers wishing to continue using their fine-tuned models beyond January 4, 2024 will need to fine-tune replacements atop the new base GPT-3 models (ada-002, babbage-002, curie-002, davinci-002), or newer models (gpt-3.5-turbo, gpt-4). Once this feature is available later this year, we will give priority access to GPT-3.5 Turbo and GPT-4 fine-tuning to users who previously fine-tuned older models. We acknowledge…

(I work at OpenAI) We're planning to cover the cost for fine-tuning replacement models. We're still working through the exact mechanics that will work best for customers, and will be reaching out to customers to get feedback on different approaches in the next few weeks.

Why does OpenAI demand your phone number, and a particular KIND of phone number at that? For example they won't accept VOIP numbers. I'm not about to give them my real phone number.

It's a deal-breaker for many.

Re: GPT-4 API General Availability

#376

Post by ShamelessC »

projectileboy wrote: post #225

Relevant comment thread from people describing how much worse GPT-4 has gotten lately: https://www.reddit.com/r/ChatGPT/comments/14ruui2/i_use_chat...

I have followed many of these types of posts. In every single instance, no one provides even the _simplest_ amount of evidence. No before/after with the same prompt.

OpenAI even has a whole repository specifically for this - GPT-eval. No one uses it.

I'm not saying the theories are wrong. Maybe there is something behind the hunches that so many people seem to have about degradation. But there isn't _any_ proof. None. Whatsoever. And people are taking _internet comments_ as that proof instead? I mean, sure, it's easy to be cynical about companies in this day and age; which is why I would ultimately believe someone if they provided actual evidence. But, again - not a single ounce of proof has been provided in any one of these threads.

Furthermore, the lack of rigor being applied even with the various anecdotes is appalling.

Which version are you talking about? GPT-4 or GPT-3? Are you using the API or the web interface? Are you aware that output is non-deterministic? Are you aware that your own psychological biases will skew your opinions on the matter? One or more of these questions tend to go unanswered.

Just please, show me some robust proof. If you can't because you didn't think to; you _surely_ must realize that many people are building entire businesses on top of this tech and at least _one_ of them is running these types of evaluations. Furthermore, the model is state-of-the-art for research now as well and if you can _prove_ that there is degradation in the model that they are lying about (in a research paper), you will get citations. And yet, there is nothing. Zilch. Nada.

Re: GPT-4 API General Availability

#377

Post by robbintt »

dahwolf wrote: post #277

Personally, I'm forever locked out of OpenAI. I had the silly idea of trying to change the signin method of my account. Which isn't possible. So I figured to just delete the account and create a new one with the correct signin method. Turns out they don't delete anything. Both the email address and phone number are held hostage. As you try to create a new account, it will point out that those are in use. I can easily…

Use a 2nd email and google voice phone number

Re: GPT-4 API General Availability

#378

Post by esperent »

hnfong wrote: post #357
esperent wrote: post #311

Earlier quoted context omitted.

This is a terrible excuse that I see trotted out far to often to justify going to developing countries and barely even paying workers that country's minimum wage. You absolutely can pay considerably more than minimum wage without disrupting the local economy. They're paying people as low as $1.32 per hour for an absolutely horrible job. I'm not expecting them to pay western wages. But even bumping that up to $2.50 or…

I'm curious. When you buy a loaf of bread from the local market, are they cheaper than first world prices? If so, do you pay double the listed price and demand the shop pay double the price to hire workers so as to not exploit them? Are your expenses in said developing country lower than what you would have paid if you were in a richer country? Are you donating the difference to the local community? Just curious.

It's weird how people always trot out phrases like "I'm just curious" or "I'm just asking questions here" when they try to justify exploitation. Is it so that you have plausible deniability when you inevitably get called on it? Because that doesn't work.

Re: GPT-4 API General Availability

#379

Post by jeromegv »

ShadowBanThis01 wrote: post #375
npew wrote: post #287

Earlier quoted context omitted.

(I work at OpenAI) We're planning to cover the cost for fine-tuning replacement models. We're still working through the exact mechanics that will work best for customers, and will be reaching out to customers to get feedback on different approaches in the next few weeks.

Why does OpenAI demand your phone number, and a particular KIND of phone number at that? For example they won't accept VOIP numbers. I'm not about to give them my real phone number. It's a deal-breaker for many.

Seems clear that it’s for bots. And they refuse voip numbers because it’s a hell of a lot easier to buy and generate hundreds of voip numbers.

Re: GPT-4 API General Availability

#380

Post by ed_mercer »

joeythedolphin wrote: post #257
PostOnce wrote: post #197

Promote and proliferate local LLMs. If you use GPT, you're giving OpenAI money to lobby the government so they'll have no competitors, ultimately screwing yourself, your wallet, and the rest of us too. OpenAI has no moat, unless you give them money to write legislation. I can currently run some scary smart and fast LLMs on a 5 year old laptop with no GPU. The future is, at least, interesting.

Great point -- I was thinking of renewing my $20/subscription but I will keep it cancelled. We must not fund AI propaganda machines.

Forgive me as I’m out of the loop. What propaganda are you referring to?
Post reply on HN