Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 is not getting worse

coagulopath.com

Post reply on HN
Print view View original

171–180 of 186 posts

Re: GPT-4 is not getting worse

#171

Post by saliagato »

jw1224 wrote: post #35

There is an ongoing bug[1] with OpenAI's API, where it stops streaming responses after exactly 5 minutes. When I first came across the issue, I debugged it by writing a prompt along these lines: > Output every number from 1 to 10,000 in written form (e.g. "one", "two", etc.). Respond with one number per line in numeric order. As expected, the API would begin counting every number just as I asked. This would continue…

I can get 8k + tokens response from OpenAI API without problems (with gpt-3.5-turbo-16k) but I encounter errors from Azure OpenAI when using the same prompt and parameters. Obviously it sucks because I can't get long response from gpt-4-32k on Azure

Re: GPT-4 is not getting worse

#172

Post by layer8 »

nomel wrote: post #163
layer8 wrote: post #122

Earlier quoted context omitted.

The inability to resolve issues like the mentioned 5-minute timeout with OpenAI support even after months should be a red flag to not build any products based on their services.

There’s no real alternatives for a product that would need their service to begin with.

That still means it’s a no-go to build a business on such a product, in my book.

Re: GPT-4 is not getting worse

#173

Post by layer8 »

ilaksh wrote: post #159
layer8 wrote: post #122

Earlier quoted context omitted.

The inability to resolve issues like the mentioned 5-minute timeout with OpenAI support even after months should be a red flag to not build any products based on their services.

Can you give me a use case where generating a response for more than five minutes is important or even desirable?

It’s obviously a problem for the parent (https://news.ycombinator.com/user?id=jw1224), but the issue is not primarily whether such a limit is reasonable or not, it’s that it’s undocumented and seemingly unintentional (relative to the limits they do document), and most importantly it’s that the lack of response from OpenAI support indicates that you can’t count on them providing any reliable and timely level of support. This is just unacceptable for a service you’d want to build a business on.

Re: GPT-4 is not getting worse

#174

Post by JimDabell »

cloudking wrote: post #124
JimDabell wrote: post #34

All of the tests are one-shot questions and answers. Where I have found GPT-4 to be degrading significantly is with sustained discussion about technical topics. It starts forgetting important parts of the discussion almost straight away, long before the size of the context window becomes a factor. This wasn’t the case when it was new.

If your discussions involve code, you should try including the latest code on each prompt. The token window will start forgetting earlier parts of the conversation. For example: Now improve this feature with xyz Current code: -- {code} --

As I said, it starts forgetting important parts of the discussion almost straight away, long before the size of the context window becomes a factor.

Re: GPT-4 is not getting worse

#175

Post by CamperBob2 »

pyinstallwoes wrote: post #170
CamperBob2 wrote: post #135

Earlier quoted context omitted.

That seems like a compelling argument at first, but the churches are full of intelligent people who will fail the same test because of childhood indoctrination. They have been conditioned not to care about what's objectively true, while the LLM simply doesn't possess a concept of objective truth in the first place. I suspect it will be easier for ML researchers to fix that problem in their software models than it wou…

Objective truth doesn’t exist. Any system which defines truth can only point to objects within that systems language space.

We don't live in a "language space." We live in a material world where objective truth -- the kind that remains after you stop believing in it -- indisputably exists.

Re: GPT-4 is not getting worse

#176

Post by phatskat »

antonvs wrote: post #153
visarga wrote: post #104

Earlier quoted context omitted.

I have this too, but on information extraction tasks, if the input document is too long or complex it will skip sections saying something along the lines of (instead of the list) (instead of the details) Very annoying when you want to extract everything and is hard to fix. Maybe they had some examples solved by abbreviation in the fine-tuning set and it damaged the model. It doesn't make sense to train this behaviour…

Using GPT models like that is unreliable anyway. I was talking to someone who was excited that he was able to use ChatGPT to convert a short tabular text listing of about 70 rows to a comma-separated list of the ids on each row, without having to e.g. import it into a spreadsheet or database. I pointed out that this couldn't be relied on, but he was skeptical. So I checked the output he pasted - there were only 69 id…

This boggles my mind. I know doing it by hand, or using a tool takes more time and effort, but if you’re going to automate it away _anyway_ you should be doing it in a reliable and retractable way.

I had a vim session saved that had a command history with a handful of regex patterns to format copy/pasted bank statement pdfs converted to docx (don’t ask) that would format the data into a CSV. Load the session, paste the ugly data, and cycle through the command history. It was ugly but reliable, and after the initial time sink of writing it once, it became quite efficient and wasn’t made more so (say a VimScript function) because it worked well enough.

Re: GPT-4 is not getting worse

#177

Post by appplication »

layer8 wrote: post #172
nomel wrote: post #163

Earlier quoted context omitted.

There’s no real alternatives for a product that would need their service to begin with.

That still means it’s a no-go to build a business on such a product, in my book.

I agree 100%. It’s irresponsible and frankly a stupid idea for anyone to bet on building a business that relies on ChatGPTs API (or really any private API in 2023). We’ve seen time and time again (and more and more recently) companies weaponizing API dependence and fees to drive sucker-ons out of business.

If you want something to build a business around, build with an OSS model. Your competitive moat is going to be trash, but that’s going to be true for anyone building an AI startup that’s backed by any LLM technology. It’s a fool’s gold rush right now no matter how you slice it.

Re: GPT-4 is not getting worse

#178

Post by agentgumshoe »

fn-mote wrote: post #94
agentgumshoe wrote: post #84

Earlier quoted context omitted.

What did it use as sources for that?

The arguments are similar those made during during the height of the Cold War. Just substitute a different country name (or two). This is by no means the first time this situation has occurred.

I think it would be much better if it just came back with "Be excellent to each other"

Re: GPT-4 is not getting worse

#179

Post by [deleted] »

appplication wrote: post #177
layer8 wrote: post #172

Earlier quoted context omitted.

That still means it’s a no-go to build a business on such a product, in my book.

I agree 100%. It’s irresponsible and frankly a stupid idea for anyone to bet on building a business that relies on ChatGPTs API (or really any private API in 2023). We’ve seen time and time again (and more and more recently) companies weaponizing API dependence and fees to drive sucker-ons out of business. If you want something to build a business around, build with an OSS model. Your competitive moat is going to be…

[deleted]

Re: GPT-4 is not getting worse

#180

Post by syndicatedjelly »

rendall wrote: post #167
syndicatedjelly wrote: post #166

Earlier quoted context omitted.

This gave me an idea to add the following to custom instructions: * At the end of all answers, related to programming, provide a list of 4 keywords or topics that represent first principles for the subject so I can research them later.*

How is this working?

It’s alright. Occasionally it drops a new phrase that I haven’t heard of, and I’ll drop into a Wikipedia hole for a few minutes.
Post reply on HN