Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 API General Availability

openai.com

Post reply on HN
Print view View original

211–220 of 562 posts

Re: GPT-4 API General Availability

#211

Post by selalipop »

purplecats wrote: post #195
selalipop wrote: post #141

Earlier quoted context omitted.

Can you try notionsmith.ai and let me know what you think? I've been working on LLMs for creative tasks and believe a mix of chain of thought and injecting stochasticity (like instructing the LLM to use certain random letters pulled from an RNG in a certain way at certain points) can go a long way in terms of getting closer to human-like creativity

really cool idea! been looking for something like this for a long time. its too bad it freezes my tab and is unusable

Yup, it's a fun side project so I decided from the get-go I wasn't going to cater to anything non-standard

It relies on WebSockets, Js, and a reasonably stable connection to run since it's built on Blazor

Re: GPT-4 API General Availability

#212

Post by stavros »

senko wrote: post #86
ClassicOrgin wrote: post #58

Earlier quoted context omitted.

I'm interested in the cost of gpt-3.5-turbo-instruct. I've got a basic website using text-davinci-003 that I would like to launch but can't because text-davinci-003 is too expensive. I've tried using just gpt-3.5-turbo but it won't work because I'm expecting a formatted JSON to be returned and I can just never get consistency.

With the latest 3.5-turbo, you can try forcing it to call your function with a well-defined schema for arguments. If the structure is not overly complex, this should work.

It's great at returning well-formatted JSON, but it can hallucinate arguments or values to arguments.

Re: GPT-4 API General Availability

#213

Post by PostOnce »

john2x wrote: post #201
PostOnce wrote: post #197

Promote and proliferate local LLMs. If you use GPT, you're giving OpenAI money to lobby the government so they'll have no competitors, ultimately screwing yourself, your wallet, and the rest of us too. OpenAI has no moat, unless you give them money to write legislation. I can currently run some scary smart and fast LLMs on a 5 year old laptop with no GPU. The future is, at least, interesting.

Care to share some links? My lack of GPU is the main blocker for me from playing with local-only options. I have an old laptop with 16GB RAM and no GPU. Can I run these models?

https://github.com/ggerganov/llama.cpp

https://huggingface.co/TheBloke

There's a LocalLLaMA subreddit, irc channels, and a whole big community around the web working on it on GitHub nd elsewhere.

edit: I forgot to directly answer you: yes you can run these models. 16GB of plenty. Different quantizations give you different amounts of smarts and speed. There are tables that tell you how much RAM is needed per which quantization you choose, as well as how fast it can produce results (ms per token). e.g. https://github.com/ggerganov/llama.cpp#quantization where RAM required a little more than the file size, but there are tables that list it explicitly which I don't have immediately at hand.

Re: GPT-4 API General Availability

#214

Post by rmorey »

Zpalmtree wrote: post #146
gamegoblin wrote: post #31

Biggest news here from a capabilities POV is actually the gpt-3.5-turbo-instruct model. gpt-3.5-turbo is the model behind ChatGPT. It's chat-fine-tuned which makes it very hard to use for use-cases where you really just want it to obey/complete without any "chatty" verbiage. The "davinci-003" model was the last instruction tuned model, but is 10x more expensive than gpt-3.5-turbo, so it makes economical sense to hack…

I'm hoping gpt-3.5-turbo-instruct isn't super neutered like chatgpt. davinci-003 can be a lot more fun and answer on a wide range of topics where ChatGPT will refuse to answer.

such as?

Re: GPT-4 API General Availability

#215

Post by gamegoblin »

mrinterweb wrote: post #209
gamegoblin wrote: post #69

Earlier quoted context omitted.

I'm assuming they will price it the same as normal gpt-3.5-turbo. I won't use it if it's more than 2x the price of turbo, because I can usually get turbo to do what I want, it just takes more tokens sometimes. Have you tried getting your formatted JSON out via the new Functions API? I does cure a lot of the deficiencies in 3.5-turbo.

From what I can find, pricing of GPT-4 is roughly 25x that of 3.5 turbo. https://openai.com/pricing https://platform.openai.com/docs/deprecations/

In this thread we’re talking about gpt-3.5-turbo-instruct, not GPT4

Re: GPT-4 API General Availability

#216

Post by danenania »

henry_viii wrote: post #153
athyuttamre wrote: post #120

Earlier quoted context omitted.

Thanks! The website is no longer active, just updated my bio.

I know you guys are busy literally building the future but could you consider adding a search field in ChatGPT so that users can search their previous chats?

I'd also love to see a search field. That's my #1 feature request not related to the model.

Re: GPT-4 API General Availability

#217

Post by stavros »

minimaxir wrote: post #22
ravenstine wrote: post #13

Earlier quoted context omitted.

My experience is that GPT-3.5 is not better or even nearly as good as GPT-4. Will it work for most use cases? Probably, yes. But GPT-3.5 effectively ignores instructions much more often than GPT-4 and I've found it far far easier to trip up with things as simple as trailing spaces; it will sometimes exhibit really odd behavior like spelling out individual letters when you give it large amounts of text with missing gr…

The initial gpt-3.5-turbo was flakey and required significant prompt engineering. The updated gpt-3.5-turbo-0613 fixed all the issues I had even after stripping out the prompt engineering.

I use it to generate nonsense fairytales for my sleep podcast (https://deepdreams.stavros.io/), and it will ignore my (pretty specific) instructions and add scene titles to things, and write the text in dramatic format instead of prose, no matter how much I try.

Re: GPT-4 API General Availability

#219

Post by redox99 »

mike_hearn wrote: post #75
it_citizen wrote: post #59

Earlier quoted context omitted.

I keep reading “GPT4 got nerfed” but I have been using from day 1, and while it definitely gives bad answers, I cannot say that it was nerfed for sure. Is there any actual evidences other than some user subjective experiences?

ChatGPT is definitely more restricted than the API. Example: https://news.ycombinator.com/item?id=36179783

They are comparing text-davinci-003 with ChatGPT which presumably uses gpt-3.5-turbo, so quite different models.

They are killing text-davinci-003 btw.

Re: GPT-4 API General Availability

#220

Post by [deleted] »

superkuh wrote: post #84

Yikes. They're actually killing off text-davinci-003. RIP to the most capable remaining model and RIP to all text completion style freedom. Now it's censored/aligned chat or instruct models with arbitrary input metaphor limits for everything. gpt3.5-turbo is terrible in comparison. This will end my usage of openai for most things. I doubt my $5-$10 API payments per month will matter. This just lights more of a fire u…

[deleted]
Post reply on HN