Live data from Hacker News

Memory and new controls for ChatGPT

openai.com

191–200 of 270 posts

Re: Memory and new controls for ChatGPT

#191

I have wanted nothing more than this feature. The work I try to do with ChatGPT requires a longer memory than its default nature. I will get to a point where I I have 80% of what I want out of a conversation, then it forgets critical early parts of the conversation. Then it just unravels into completely forgetting everything. I want to teach ChatGPT some basic tenants and then build off of those. This will be the cle…

Use api, embed history and retrieve.

Re: Memory and new controls for ChatGPT

#192

I have wanted nothing more than this feature. The work I try to do with ChatGPT requires a longer memory than its default nature. I will get to a point where I I have 80% of what I want out of a conversation, then it forgets critical early parts of the conversation. Then it just unravels into completely forgetting everything. I want to teach ChatGPT some basic tenants and then build off of those. This will be the cle…

Use api, embed history and retrieve.

I've tried this route. Same problems. At least this was the case last year.

Re: Memory and new controls for ChatGPT

#193

This is a bit off topic to the actual article, but I see a lot of top ranking comments complaining that ChatGPT has become lazy at coding. I wanted to make two observations: 1. Yes, GPT-4 Turbo is quantitatively getting lazier at coding. I benchmarked the last 2 updates to GPT-4 Turbo, and it got lazier each time. 2. For coding, asking GPT-4 Turbo to emit code changes as unified diffs causes a 3X reduction in lazy co…

I have not noticed any reduction in laziness with later generations, although I don't use ChatGPT in the same way that Aider does. I've had a lot of luck with using a chain-of-thought-style system prompt to get it to produce results. Here are a few cherry-picked conversations where I feel like it does a good job (including the system prompt). A common theme in the system prompts is that I say that this is an "expert-to-expert" conversation, which I found tends to make it include less generic explanatory content and be more willing to dive into the details.

- System prompt 1: https://sharegpt.com/c/osmngsQ

- System prompt 2: https://sharegpt.com/c/9jAIqHM

- System prompt 3: https://sharegpt.com/c/cTIqAil Note: I had to nudge ChatGPT on this one.

All of this is anecdotal, but perhaps this style of prompting would be useful to benchmark.

Re: Memory and new controls for ChatGPT

#194
post #143

Earlier quoted context omitted.

I have some success by telling it to not speak to me unless it's in code comments. If it must explain anything, do it it in a code comment.

I’ve been telling it I don’t have any fingers and so can’t type. It’s been pretty empathetic and finishes functions

So already humans need to get down on their metaphorical knees and beg the AI for mercy, just for some chance of convincing it to do its job.

Re: Memory and new controls for ChatGPT

#195
post #78

Often I’ll play dumb and withhold ideas from ChatGPT because I want to know what it thinks. If I give it too many thoughts of mine, it gets stuck in a rut towards my tentative solution. I worry that the memory will bake this problem in.

I purposely go out of my way to start new chats to have a clean slate and not have it remember things.

[deleted]

Re: Memory and new controls for ChatGPT

#196

This is a bit off topic to the actual article, but I see a lot of top ranking comments complaining that ChatGPT has become lazy at coding. I wanted to make two observations: 1. Yes, GPT-4 Turbo is quantitatively getting lazier at coding. I benchmarked the last 2 updates to GPT-4 Turbo, and it got lazier each time. 2. For coding, asking GPT-4 Turbo to emit code changes as unified diffs causes a 3X reduction in lazy co…

Lazy coding is a feature not a bug. My guess is that it breaks aider automation, but by analyzing the AST that wouldn't be a problem. My experience with lazy coding, is it omits the irrelevant code, and focuses on the relevant part. That's good! As a side note, i wrote a very simple small program to analyze Rust syntax, and single out functions and methods using the syn crate [1]. My purpose was exactly to make it ig…

It sounds like you've been extremely lucky and only had GPT "omit the irrelevant code". That has not been my experience working intensively on this problem and evaluating numerous solutions through quantitative benchmarking. For example, GPT will do things like write a class with all the methods as simply stubs with comments describing their function.

Your link appears to be ~100 lines of code that use rust's syntax parser to search rust source code for a function with a given name and count the number of AST tokens it contains.

Your intuitions are correct, there are lots of ways that an AST can be useful for an AI coding tool. Aider makes extensive use of tree-sitter, in order to parse the ASTs of a ~dozen different languages [0].

But an AST parser seems unlikely to solve the problem of GPT being lazy and not writing the code you need.

[0] https://aider.chat/docs/repomap.html

Re: Memory and new controls for ChatGPT

#197

Haha of course this news comes just after I wrote a parser for my ChatGPT dump and generate offline embeddings for it with Phi 2 to help generate conversation metadata.

so far you can't search your whole conversation history, so your tool is relevant for a few more weeks. is it open source?

I'll share the core bit that took a while to figure out the right format, my main script is a hot mess using embeddings with SentenceTransformer, so I won't share that yet. E.g: last night I did a PR for llama-cpp-python that shows how Phi might be used with JSON only for the author to write almost exactly the same code at pretty much the same time. https://github.com/abetlen/llama-cpp-python/pull/1184 But you can see how that might work. Here is the core parser code: https://gist.github.com/lukestanley/eb1037478b1129a5ca0560ee...

Re: Memory and new controls for ChatGPT

#200
How does this technically work? Is it just a natural language shortcut for prepending text to your context window, or does it pull information as needed as inferred from the prompt? E.g. the meeting note formatting "memory" gets retrieved when prompting to summarise meeting notes.
Post reply on HN