Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 API General Availability

openai.com

Post reply on HN
Print view View original

361–370 of 562 posts

Re: GPT-4 API General Availability

#361

Post by cj »

r3trohack3r wrote: post #238
fnordpiglet wrote: post #180

Earlier quoted context omitted.

If you don’t own the weights you don’t own anything. This is why open models are so crucial. I don’t understand any business who is building fine tuned models against closed models.

> I don’t understand any business who is building fine tuned models against closed models Do you have any recommendations for good open models that businesses could use today? From what I've seen in the space, I suspect businesses are building fine tuned models against closed models because those are the only viable models to build a business model on top of. The quality of open models isn't competitive.

PSA: anyone working at a company with $50k+ of spend with AWS, reach out to your rep expressing interest in AI. You’ll be on a call with 6 solution architects and AI specialists in a matter of days. They’re incredibly knowledgeable and freely recommend non-AWS alternatives when the use case calls for it.

Re: GPT-4 API General Availability

#362

Post by Rexxar »

actionfromafar wrote: post #249
tensor wrote: post #242

Earlier quoted context omitted.

It doesn't have to be copyrightable to be intellectual property.

Patents? Trademark? What do you mean?

Maybe this: https://en.wikipedia.org/wiki/Database_right but it doesn't exist in every countries.

Re: GPT-4 API General Availability

#363

Post by km3r »

esperent wrote: post #314
km3r wrote: post #161

Earlier quoted context omitted.

> Use of cheap human labor in developing countries through shady anonymous companies (look up the company Sama who pay Kenyan workers about $1.5 an hour). What is wrong about injecting millions into developing nations? The rest I agree with, although I don't think it was ever really 'open' so its not getting shitty, it always was. Thankfully, "there is no moat" and other LLMs will be open, just a few months behind Op…

> What is wrong about injecting millions into developing nations? Please don't try to reframe this to make exploitation a positive thing. See my other comment here. https://news.ycombinator.com/item?id=36625438

So you'd rather OpenAI crush all business in the area by outcompeting them for workers, ensuring local businesses struggle to hire?

Re: GPT-4 API General Availability

#364

Post by ChicagoDave »

eiiot wrote: post #360
ChicagoDave wrote: post #354

Can I use the API more than 25 times in 3 hours?

Yes, that’s an artificial limitation put in place so people don’t abuse plus.

So the chat interface is rate-limited, but not the API.

So I could build long prompt scripts and run them against the api?

Re: GPT-4 API General Availability

#365

Post by avindroth »

selalipop wrote: post #76
avindroth wrote: post #12

Earlier quoted context omitted.

I am building an extensive LLM-powered app, and had a chance to compare the two using the API. Empirically, I have found 3.5 to be fairly unusable for the app's use case. How are you evaluating the two models?

It depends on the domain, but chain of thought can get 3.5 to be extremely reliable, and especially with the new 16k variant I built notionsmith.ai on 3.5: for some time I experimented with GPT 4 but the result was significantly worse to use because of how slow it became, going from ~15 seconds per generated output to a minute plus. And you could work around that with things like streaming output for some use cases,…

It depends a lot on the domain, even for CoT. I don't think there are enough NLU evaluations just yet to robustly compare GPT-3.5 w/ CoT/SC vs. GPT-4 wrt domain.

For instance, with MATH dataset, my own n=500 evaluation showed no difference between GPT-3.5 (w/ and w/o CoT) and GPT-4. I was pretty surprised by that.

Re: GPT-4 API General Availability

#366

Post by squeaky-clean »

jiggawatts wrote: post #307
ineedasername wrote: post #296

Earlier quoted context omitted.

They simply want control over the rollout of their product and how it is used. That, and perhaps opening the flood gates would produce scaling bottlenecks they’d rather stay ahead of than get behind. So they open things carefully, pull back when necessary like when they limited use of the public GPT-4 version of ChatGPT. That doesn’t seem too unreasonable. And yes sure, some amount of it might be attempts to manufact…

There's no scaling issues to speak of. These AIs are stateless , which makes them embarrassingly parallel. They can always just throw more GPUs at it. Microsoft even had some videos where they bragged about how these models can be run on any idle GPU around the world, dynamically finding resources wherever it is available! If there's not enough GPUs at a certain price point, raise prices. Then lower prices later when…

More GPUs currently don't exist. Nvidia is at capacity for production, and they have to compete with other companies who are also bidding on these GPUs. It's not an issue of raising the price point. The GPUs they want to buy have to be purchased months in advance.

Re: GPT-4 API General Availability

#367

Post by PostOnce »

willsmith72 wrote: post #301
PostOnce wrote: post #197

Promote and proliferate local LLMs. If you use GPT, you're giving OpenAI money to lobby the government so they'll have no competitors, ultimately screwing yourself, your wallet, and the rest of us too. OpenAI has no moat, unless you give them money to write legislation. I can currently run some scary smart and fast LLMs on a 5 year old laptop with no GPU. The future is, at least, interesting.

My laptop already works too hard doing development and having chrome open, it's just not feasible. A good hosted alternative, sure, but local is not going to scale to the masses.

I have a Dell 7490 (intel 8350u cpu) I paid $250 for and I have no trouble running 13B models through a custom interactive interface I wrote as a hobby project in an afternoon. It can still get a lot better. I made it async the following day and its even more fun.

Most of peoples' problem is watching the AI type, it's not instant, but then not all (or even most) applications need to be instant. You can also avoid that by having it return everything at once instead of streaming style.

Local absolutely can scale. All kinds of fun things can be done on a machine with 16GB of RAM, or 8GB if you work harder.

Re: GPT-4 API General Availability

#368

Post by PostOnce »

andai wrote: post #275
PostOnce wrote: post #197

Promote and proliferate local LLMs. If you use GPT, you're giving OpenAI money to lobby the government so they'll have no competitors, ultimately screwing yourself, your wallet, and the rest of us too. OpenAI has no moat, unless you give them money to write legislation. I can currently run some scary smart and fast LLMs on a 5 year old laptop with no GPU. The future is, at least, interesting.

Which models are you using and for which tasks? I have found local models largely a waste of time (except for very simple tasks with very heavy prompting). But perhaps there are some recent breakthroughs I haven't seen yet.

I'm using a variety of 7 and 13B models (and a 3B one for fast feedback loop debugging) at between 8bit and 4_K_M quantizations.

Depending on your pre-prompt, your fine-tune (i.e. which model you downloaded), and your specific task, the results can be startlingly good, it's crazy that you can do this on a $250 laptop. I stay up nights working on it lately, it's so interesting.

More importantly, things change by the day. New models, new methods, new software, new interfaces... the possibilities are endless... unless we let OpenAI corrupt our government(s).

Re: GPT-4 API General Availability

#369

Post by ftxbro »

bestcoder69 wrote: post #359
ftxbro wrote: post #352

Earlier quoted context omitted.

OK but I mean I don't know them and it could have been someone pretending to be them, and probably it's easily possible to trick me about API keys. We are discussing it on a hacker news website do you seriously think tricks couldn't be hidden in a repo like that.

https://github.com/search?q=repo%3Asimonw%2Fllm%20OPENAI_API... Let us know what you find.

I found a few github issues related to api key security and management. I'm not 100% sure of your point.

Re: GPT-4 API General Availability

#370

Post by Art9681 »

twbarr wrote: post #278
tensor wrote: post #242

Earlier quoted context omitted.

It doesn't have to be copyrightable to be intellectual property.

No, but what is it? Not your lawyer, not legal advice, but it's not a trade secret, they've given it to researchers. It's not a trademark because it's not an origin identifier. The structure might be patentable, but the weights won't be. It's certainly not a mask work. It might have been a contract violation for the guy who redistributed it, but I'm not a party to that contract.

I'm going to play devil's advocate and state that a lot of what you mentioned will be relevant to a tiny part of the world that has the means to enforce this. The law will be forced to change as a response to AI. Many debates will be had. Many crap laws will be made by people grasping at straws but it's too late. Putting red tape around this technology puts that nation at a technological disadvantage. I would go as far as labeling a national security threat.

I'm calling it now. Based on what I see today. Europe will position itself as a leader in AI legislation, and its economy will give way to the nations that want to enter the race and grab a chunk of the new economy.

It's a Catch 22. You either gimp your own technological progress, or start a war with a nation that does not. Pretty sure Russia and China don't really care about the ethics behind it. There are plenty of nations capable enough in the same boat.

Now what? OK, so in some hypothetical future China has an uncensored model with free reign over the internet. The US and Europe has banned this. What's stopping anyone from running the Chinese model? There isn't enough money in the world to enforce software laws.

How long have they tried to take down The Pirate Bay? Pretty much every permutation of every software that's ever been banned can be found and ran with impunity if you have the technical knowledge to do so. No law exists that can prevent that.

If it did, OpenAI wouldn't exist.

Post reply on HN