Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory and new controls for ChatGPT

openai.com

Post reply on HN
Print view View original

161–170 of 270 posts

Re: Memory and new controls for ChatGPT

#161

Post by skywhopper »

txutxu wrote: post #127
nafizh wrote: post #38

My use of ChatGPT has just organically gone down 90%. It's unable to do any sort of task of non-trivial complexity e.g. complex coding tasks, writing complex prose that conforms precisely to what's been asked etc. Also I hate the fact that it has to answer everything in bullet points, even when it's not needed, clearly rlhf-ed. At this point, my question types have become what you would ask a tool like perplexity.

> that conforms precisely to what's been asked This. People talks about prompt engineering, but then it fails on really simple details, like "on lowercase", "composed by max two words", etc... and when you point at the failure, apologizes, and composes something else that forgets the other 95% of the original prompt. Or worse, apologizes and makes again the very same mistake.

This sucks, but it's unlikely to be fixable, given that LLMs don't actually have any comprehension or reasoning capability. Get too far into fine-tuning responses and you're back to "classic" AI problems.

Re: Memory and new controls for ChatGPT

#162

Post by thelittleone »

stainablesteel wrote: post #104
omalled wrote: post #87

Earlier quoted context omitted.

Can you say in one or two sentences what you mean by “lazy at coding” in this context?

it was really good at some point last fall, solving problems that it had previously completely failed at, albeit after a lot of iterations via autogpt. at least for the tests i was giving it which usually involved heavy stats and complicated algorithms, i was surprised it passed. despite it passing the code was slower than what i had personally solved the problem with, but i was completely impressed because i asked h…

Hamstringing high value tasks (complete code) to give forthcoming premium offerings greater differentiation could be a strategy. But in counter to this, doing so would open the door for competitors.

Re: Memory and new controls for ChatGPT

#163

Post by skywhopper »

nafizh wrote: post #86
OJFord wrote: post #62

Earlier quoted context omitted.

I haven't really tried to use it for coding, other than once (recently, so not before some decline) indirectly, which I was pretty impressed with: I asked about analyst expectations for the Bank of England base rate, then asked it to compare a fixed mortgage with a 'tracker' (base rate + x; always x points over the base rate). It spat out the repayment figures and totals over the two years, with a bit of waffle, and…

To be honest, when I say it has significantly worsened, I am comparing to the time when GPT-4 just came out. It really felt like we were on the verge of 'AGI'. In 3 hours, I coded up a complex piece of web app with chatgpt which completely remembered what we have been doing the whole time. So, it's sad that they have decided against the public having access to such strong models (and I do think it's intentional, not…

I'm guessing it's not about safety, but about money. They're losing money hand over fist, and their popularity has forced them to scale back the compute dedicated to each response. Ten billion in Azure credits just doesn't go very far these days.

Re: Memory and new controls for ChatGPT

#164

Post by thelittleone »

BigParm wrote: post #78

Often I’ll play dumb and withhold ideas from ChatGPT because I want to know what it thinks. If I give it too many thoughts of mine, it gets stuck in a rut towards my tentative solution. I worry that the memory will bake this problem in.

Sounds like communication between me with my wife.

Re: Memory and new controls for ChatGPT

#165

Post by BytesAndGears »

comboy wrote: post #134
bluish29 wrote: post #28

It is already ignoring your prompt and custom instructions. For example, If I explicity ask it to provide a code instead of an overview it will respond by apologizing and then provide the same overview answer with minimal if no code. Will memory provide a solution to that or will be a different thing to ignore?

It used to respect custom instructions soon after GPT4 came out. I have instruction that it should always include [reasoning] part which is meant not to be read by the user. It improved quality of the output and gave some additional interesting information. It never does it know even though I never changed my custom instructions. It even faded away slowly along the updates. In general I would be much more happy user…

Yeah I have a line in my custom prompt telling it to give me citations. When custom prompts first came out, it would always give me information about where to look for more, but eventually it just… didn’t anymore.

I did find recently that it helps if you put this sentence in the “What would you like ChatGPT to know about you” section:

> I require sources and suggestions for further reading on anything that is not code. If I can't validate it myself, I need to know why I can trust the information.

Adding that to the bottom of the “about you” section seems to help more than adding something similar to the “how would you like ChatGPT to respond”.

Re: Memory and new controls for ChatGPT

#166

Post by nprateem »

anotherpaulg wrote: post #51

This is a bit off topic to the actual article, but I see a lot of top ranking comments complaining that ChatGPT has become lazy at coding. I wanted to make two observations: 1. Yes, GPT-4 Turbo is quantitatively getting lazier at coding. I benchmarked the last 2 updates to GPT-4 Turbo, and it got lazier each time. 2. For coding, asking GPT-4 Turbo to emit code changes as unified diffs causes a 3X reduction in lazy co…

> This is a bit off topic to the actual article

It wouldn't be the top comment if it wasn't

Re: Memory and new controls for ChatGPT

#169

Post by mistermann »

bbor wrote: post #140
mistermann wrote: post #133

Earlier quoted context omitted.

I have a feeling full powered LLM's are reserved for the more equal animals. I hope some people remember and document details of this era, future generations may be so impressed with future reality that they may not even think to question it's fidelity, if that concept even exists in the future.

…could you clarify? Is this about “LLMs can be biased, thus making fake news a bigger problem”?

I confidently predict that we sheep will not have access to the same power our shepherds will have.

Re: Memory and new controls for ChatGPT

#170

Post by Kranar »

ekms wrote: post #147
Kranar wrote: post #92

Earlier quoted context omitted.

Sure, but consider not using it for complex tasks. My productivity has skyrocketed with ChatGPT precisely because I don't use it for complex tasks, I use it to automate all of the trivial boilerplate stuff. ChatGPT writes excellent API documentation and can also document snippets of code to explain what they do, it does 80% of the work for unit tests, it can fill in simple methods like getters/setters, initialize con…

Is it better at those types of things than copilot? Or even just conventional boilerplate IDE plugins?

If there is an IDE plugin then I use it first and foremost, but some refactoring can't be done with IDE plugins. Today I had to write some pybind11 bindings, basically export some C++ functionality to Python. The bindings involve templates and enums and I have a very particular way I like the naming convention to be when I export to Python. Since I've done this before so I copied and pasted examples of how I like to export templates to ChatGPT and then asked it to use that same coding style to export some more classes. It managed to do it without fail.

This is a kind of grunt work that years ago would have taken me hours and it's demoralizing work. Nowadays when I get stuff like this, it's just such a breeze.

As to copilot, I have not used it but I think it's powered by GPT4.

Post reply on HN