GPT-4 API General Availability • Hacker News Board

Re: GPT-4 API General Availability

#131

Post by selalipop » Thu, Jul 06, 2023, 8:47 PM UTC

thewataccount wrote: post #38

minimaxir wrote: post #2

With how good gpt-3.5-turbo-0613 is (particularly with system prompt engineering), there's no longer as much of a need to use the GPT-4 API especially given its massive 20x-30x price increase. The mass adoption of the ChatGPT APIs compared to the old Completion APIs proves my initial blog post on the ChatGPT API correct: developers will immediately switch for a massive price reduction if quality is the same (or bette…

What usecases are you using it for? I mostly use it for generating tests, making documentation, refactoring, code snippets, etc. I use it daily for work along with copilot/x. In my experience GPT3.5turbo is... rather dumb in comparison. It makes a comment explaining what a method is going to do and what arguments it will have - then misses arguments altogether. It feels like it has poor memory (and we're talking rela…

If you're manually interacting with the model, GPT 4 is almost always going to be better.

Where 3.5 excels is with programmatic access. You can ask it for 2x as much text between setup so the end result is well formed and still get a reply that's cheaper and faster than 4 (for example, ask 3.5 for a response, then ask it to format that response)

Re: GPT-4 API General Availability

#132

Post by pomber » Thu, Jul 06, 2023, 8:47 PM UTC

If anyone wants to try the API for the first time, I've made this guide recently: https://gpt.pomb.us/

Re: GPT-4 API General Availability

#133

Post by kossTKR » Thu, Jul 06, 2023, 8:47 PM UTC

moffkalast wrote: post #119

santiagobasulto wrote: post #56

Earlier quoted context omitted.

I felt the same thing. The first version of GPT-4 I tried was crazy smart. Scary smart. Something happened afterwards…

The even more interesting part is that none of us got to try the internal version which was allegedly yet another step above that.

Would gladly pay more for a none nerfed version if they were actually honest.

The current versions is close to the original 3.5 version, while 3.5 has become horribly bad, such a scam to not disclose what's going on, especially for a paid service.

Re: GPT-4 API General Availability

#134

Post by jerrygenser » Thu, Jul 06, 2023, 8:48 PM UTC

pantulis wrote: post #67

bbotond wrote: post #62

Earlier quoted context omitted.

What I don’t understand is why is an API needed to create embeddings. Isn’t this something that could be done locally?

You would need to have a local copy of the GPT model, which are not exactly OpenAI's plans.

For embeddings, you can use smaller transformers/llms or sentence2vec and often get good enough results.

You don't need very large models to generate usable embeddings.

Re: GPT-4 API General Availability

#135

Post by Zetobal » Thu, Jul 06, 2023, 8:48 PM UTC

jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

The azure endpoints are great though.

Re: GPT-4 API General Availability

#136

Post by ren_engineer » Thu, Jul 06, 2023, 8:48 PM UTC

hospitalJail wrote: post #33

I imagine the API quality isnt nerfed on a given day like ChatGPT can be. There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist) Every once in a while, I see redditors complain of it being nerfed. Sometimes I go back to gpt3.5 and am mind boggled how much worse it is. Makes me wonder if t…

Recently people have claimed GPT4 is an ensemble model with 8 different models under the hood. My guess is that the "nerfing"(I've noticed it as well at random times) is when the model directs a question to the wrong underlying model

Re: GPT-4 API General Availability

#137

Post by [deleted] » Thu, Jul 06, 2023, 8:48 PM UTC

[deleted]

Re: GPT-4 API General Availability

#138

Post by refulgentis » Thu, Jul 06, 2023, 8:49 PM UTC

sroussey wrote: post #103

hungrigekatze wrote: post #96

Earlier quoted context omitted.

Check out this post from a round table dialogue with Greg Brockman from OpenAI. The GPT models that were in existence / in use in early 2023 were not the performance-degraded quantized versions that are in production now: https://www.reddit.com/r/mlscaling/comments/146rgq2/chatgpt_...

Oh interesting. I thought that’s what turbo was.

It was, that's what the comment says?

Re: GPT-4 API General Availability

#139

Post by atulvi » Thu, Jul 06, 2023, 8:49 PM UTC

I'm not sure what I expected now

    500 {'error': {'message': 'Request failed due to server shutdown', 'type': 'server_error', 'param': None, 'code': None}} {'Date': 'Thu, 06 Jul 2023 20:48:07 GMT', 'Content-Type': 'application/json', 'Content-Length': '141', 'Connection': 'keep-alive', 'access-control-allow-origin': '*', 'openai-model': 'gpt-4-0613', 'openai-organization'

Re: GPT-4 API General Availability

#140

Post by esperent » Thu, Jul 06, 2023, 8:50 PM UTC

athyuttamre wrote: post #111

jwr wrote: post #89

Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out.…

(I'm an engineer at OpenAI) Very sorry to hear about these issues, particularly the timeouts. Latency is top of mind for us and something we are continuing to push on. Does streaming work for your use case? https://github.com/openai/openai-cookbook/blob/main/examples... We definitely want to investigate these and the billing issues further. Would you consider emailing me your org ID and any request IDs (if you have t…

It's kind of incredible how fast OpenAI (now also known as ClosedAI) is going through the enshittification process. Even Facebook took around a decade to reach this level.

OpenAI has an amazing core product, but in the span of six months:

* Went from an amazing and inspiring open company that even put "Open" in their name to a fully locked up commercial beast.

* Non-existent customers support and all kinds of borderline illegal billing practice. You guys are definitely aware that when there's a network error on the API or ChatGPT, the user still gets charged. And there's a lot of these errors. I get roughly one per hour or two.

* Frustratingly loose interpretation of EU data protection rules. For example, the setting to say "don't use my personal chat data" is connected to the setting to save conversations. So you can't disable it without losing all your chat history.

* Clearly nerfing the ChatGPT v4 products, at least according to hundreds or even thousands of commenters here and on reddit, while denying to have made any changes.

* Use of cheap human labor in developing countries through shady anonymous companies (look up the company Sama who pay Kenyan workers about $1.5 an hour).

* Not to mention the huge questions around the secret training dataset and whether large portions of it consist of illegally obtained private data (see the recent class court case in California)