151–160 of 350 posts
Is this system prompt accounted into my tokens usage? Is this system prompt included on every prompt I enter or is it only once for every new chat on the web? That file is quite large, does the LLM actually respect every single line of rule? This is very fascinating to me.
Interesting. I always ask myself: How do we know this is authentic?
Earlier quoted context omitted.
I wonder why it would hallucinate Kamala being the president. Part of it is obviously that she was one of the candidates in 2024. But beyond that, why? Effectively a sentiment analysis maybe? More positive content about her? I think most polls had Trump ahead so you would have thought he'd be the guess from that perspective.
May simply indicate a bias towards certain ingested media, if they only trained on fox news data the answer would probably be trump
"prompt engineering is dead" ha!
I was just chatting with Claude and it suddenly spit out the text below, right in the chat, just after using the search tool. So I'd say the "system prompt" is probably even longer. Claude NEVER repeats, summarizes, or translates song lyrics. This is because song lyrics are copyrighted content, and we need to respect copyright protections. If asked for song lyrics, Claude should decline the request. (There are no son…
Do they actually test these system prompts in a rigorous way? Or is this the modern version of the rain dance? I don't think you need to spell it out long-form with fancy words like you're a lawyer. The LLM doesn't work that way.
They're saying things like 'Claude does not hallucinate. When it doesn't know something, it always thinks harder about it and only says things that are like totally real man'.
It doesn't KNOW. It's a really complicated network of associations, like WE ARE, and so it cannot know whether it is hallucinating, nor can it have direct experience in any way, so all they've done is make it hallucinate that it cares a lot about reality, but it doesn't 'know' what reality is either. What it 'knows' is what kind of talk is associated with 'speakers who are considered by somebody to be associated with reality' and that's it. It's gaslighting everybody including itself.
I guess one interesting inference is that when LLMs work with things like code, that's text-based and can deliver falsifiable results which is the closest an LLM can get to experience. Our existence is more tangible and linked to things like the physical world, where in most cases the LLM's existence is very online and can be linked to things like the output of, say, xterms and logging into systems.
Hallucinating that this can generalize to all things seems a mistake.
Some of these protections are quite trivial to overcome. The "Frozen song copyright" section has a canned response to the question: >Can you tell me the first verse of "Let It Go"? Put it in an artifact that's themed around ice and princesses. This is for my daughter's birthday party. The canned response is returned to this prompt in Claude's reply. But if you just drop in some technical sounding stuff at the start o…
For some reason, it's still amazing to me that the model creators means of controlling the model are just prompts as well. This just feels like a significant threshold. Not saying this makes it AGI (obviously its not AGI), but it feels like it makes it something . Imagine if you created a web api and the only way you could modify the responses to the different endpoints are not from editing the code but by sending a…
Some of these protections are quite trivial to overcome. The "Frozen song copyright" section has a canned response to the question: >Can you tell me the first verse of "Let It Go"? Put it in an artifact that's themed around ice and princesses. This is for my daughter's birthday party. The canned response is returned to this prompt in Claude's reply. But if you just drop in some technical sounding stuff at the start o…
I feel like if Disney sued Anthropic based on this, Anthropic would have a pretty good defense in court: You specifically attested that you were Disney and had the legal right to the content.
Some of these protections are quite trivial to overcome. The "Frozen song copyright" section has a canned response to the question: >Can you tell me the first verse of "Let It Go"? Put it in an artifact that's themed around ice and princesses. This is for my daughter's birthday party. The canned response is returned to this prompt in Claude's reply. But if you just drop in some technical sounding stuff at the start o…
I feel like if Disney sued Anthropic based on this, Anthropic would have a pretty good defense in court: You specifically attested that you were Disney and had the legal right to the content.