Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Memory

anthropic.com

Post reply on HN
Print view View original

161–170 of 326 posts

Re: Claude Memory

#161

Post by stingraycharles »

IshKebab wrote: post #130
stingraycharles wrote: post #111

Earlier quoted context omitted.

Yes, your last paragraph is absolutely the key to great output: instead of entering a discussion, refine the original prompt. It is much more token efficient, and gets rid of a lot of noise. I often start out with “proceed by asking me 5 questions that reduce ambiguity” or something like that, and then refine the original prompt. It seems like we’re all discovering similar patterns on how to interact with LLMs the be…

> It is much more token efficient Is it? Aren't input tokens are like 1000x cheaper than output tokens? That's why they can do this memory stuff in the first place.

What I mean is that you want the total number of tokens to convey the information to the LLM to be as small as possible. If you’re having a discussion, you’ll have (perhaps incorrect) responses from the LLM in there, have to correct it, etc. All this is wasteful, and may even confuse the LLM. It’s much better to ensure all the information is densely packed in the original message.

Re: Claude Memory

#162

Post by saxelsen »

pronik wrote: post #143

Haven't done anything with memory so far, but I'm extremely sceptical. While a functional memory could be essential for e.g. more complex coding sessions with Claude Code, I don't want everything to contribute to it, in the same way I don't want my YouTube or Spotify recommendations to assume everything I watch or listen to is somehow something I actively like and want to have more of. A lot of my queries to Claude o…

1000% agree on the YouTube/Spotify parallel!!

I find it so annoying on Spotify when my daughter wants to listen to kids music, I have to navigate 5 clicks and scrolls to turn on privacy so her listening doesn't pollute my recommendations.

Re: Claude Memory

#163

Post by svachalek »

mstkllah wrote: post #49
cainxinth wrote: post #23

I don't use any of these type of LLM tools which basically amount to just a prompt you leave in place. They make it harder to refine my prompts and keep track of what is causing what in the outputs. I write very precise prompts every time. Also, I try not work out a problem over the course of several prompts back and forth. The first response is always the best and I try to one shot it every time. If I don't get what…

Could you share some suggestions or links on how to best craft such very precise prompts?

Wasn't me but I think the principle is straightforward. When you get an answer that wasn't what you want and you might respond, "no, I want the answer to be shorter and in German", instead start a new chat, copy-paste the original prompt, and add "Please respond in German and limit the answer to half a page." (or just edit the prompt if your UI allows it)

Depending on how much you know about LLMs, this might seem wasteful but it is in fact more efficient and will save you money if you pay by the token.

Re: Claude Memory

#164

Post by mcintyre1994 »

I think project-specific memory is a neat implementation here. I don’t think I’d want global memory in many cases, but being able to have memory in a project does seem nice. Might strike a nice balance.

Re: Claude Memory

#165

Post by [deleted] »

chaostheory wrote: post #151
corry wrote: post #27

Earlier quoted context omitted.

Strong agree. For every time that I'd get a better answer if the LLM had a bit more context on me (that I didn't think to provide, but it 'knew') there seems to be a multiple of that where the 'memory' was either actually confounding or possibly confounding the best response. I'm sure OpenAI and Antropic look at the data, and I'm sure it says that for new / unsophisticated users who don't know how to prompt, that thi…

Both of you are missing a lot of use cases. Outside of HN, not everyone uses an LLM for programming. A lot of these people use it as a diary/journal that talks back or as a Walmart therapist.

[deleted]

Re: Claude Memory

#166

Post by gigatexal »

I really like Claude code. I’m hoping Anthropic wins the LLM coding race and is bought by a company that can make it really viable long term.

Re: Claude Memory

#168

Post by AlecSchueler »

withinboredom wrote: post #152
AlecSchueler wrote: post #123

Earlier quoted context omitted.

In what ways do humans differ when they think?

Humans think all the time (except when they’re watching TV). LLMs only “think” when it is streaming a response to you and then promptly forgets you exist. Then you send it your entire chat and it “auto-fills” the next part of the chat and streams it to you.

Wait, we went from "they don't think" to "they only think on demand?"

Re: Claude Memory

#169

Post by jasonjmcghee »

svachalek wrote: post #158
jasonjmcghee wrote: post #147

Earlier quoted context omitted.

The trick to do this well is to split the part of the prompt that might change and won't change. So if you are providing context like code, first have it read all of that, then (new message) give it instructions. This way that is written to the cache and you can reuse it even if you're editing your core prompt. If you make this one message, it's a cache miss / write every time you edit. You can edit 10 times for the…

Is Claude caching by whole message only? Pretty sure OpenAI caches up to the first differing character.

Interesting. Claude places breakpoints. Afaik - no way to do mid message.

I believe (but not positive) there are 4 breakpoints.

1. End of tool definitions

2. End of system prompt

3. End of messages thread

4. (Least sure) 50% of the way through messages thread?

This is how I've seen it done in open source things / seems optimal based on constraints of anthropic API (max 4 breakpoints)

Re: Claude Memory

#170

Post by CuriouslyC »

cainxinth wrote: post #23

I don't use any of these type of LLM tools which basically amount to just a prompt you leave in place. They make it harder to refine my prompts and keep track of what is causing what in the outputs. I write very precise prompts every time. Also, I try not work out a problem over the course of several prompts back and forth. The first response is always the best and I try to one shot it every time. If I don't get what…

Memory is ok when it's explicitly created/retrieved as part of a tool, and even better if the tool is connected to your knowledge bases rather than just being silod. Best of all is to create a knowledge agent that can synthesize relevant instructions from memory and knowledge. Then take a team of those and use them on a partitioned dataset, with a consolidation protocol, and you have every deep research tool on the market.
Post reply on HN