311–320 of 350 posts
Earlier quoted context omitted.
"What color is the car? It is red." "It" is unambiguously the correct pronoun to use for a car. I'd really challenge you to find a native English speaker who would think otherwise. I would argue a computer program is no different than a car.
People often refer to their car and other people's as "she" ("she's a beauty") so you're is obviously wrong.
Earlier quoted context omitted.
I wonder why it would hallucinate Kamala being the president. Part of it is obviously that she was one of the candidates in 2024. But beyond that, why? Effectively a sentiment analysis maybe? More positive content about her? I think most polls had Trump ahead so you would have thought he'd be the guess from that perspective.
It's probably entirely insurance. We now have the most snowflake and emotionally sensitive presidency and party in charge. If it said Harris was president, even by mistake, the right-wing-sphere would whip up in a frenzy and attempt to deport everyone working for Antrophic.
Earlier quoted context omitted.
Isn't Claude trained to work better with XML tags
All modern LLMs seem to prefer XML to other structured markup. It might be because there's so much HTML in the training set, or because it has more redundancy baked in which makes it easier for models to parse.
My experience is that as the prompt gets longer, performance decreases. Having such a long prompt with each request cannot be good. I remember in the early days of OpenAI, they had made the text completion feature available directly and it was much smarter than ChatGPT... I couldn't understand why people were raving about ChatGPT instead of the raw davinci text completion model. Ir sucks how legal restrictions are du…
> Ir sucks how legal restrictions are dumbing down the models Can you expand on this? I'm not sure I understand what you mean
A lot of these prompts would distract Claude from what the end user asked.
In my experience working with LLMs, each prompt has a certain amount of 'intellectual capacity' and the more different questions and ideas you try to cram in a single prompt, the dumber the response, the more likely it makes mistakes.
These formatting rules and constraints are orthogonal to what the user will ask so likely highly distracting.
It's kind of like a human; if you give someone more work to complete within the same amount of time, they will do worse.
But then I'm not sure how those system prompts are used. Are they trained into Claude or are they prepended to the start of the user's own prompt? What I'm saying applies to the latter which is what I suspect is happening.
Earlier quoted context omitted.
I'm not sure if this really says the truth is more complex? It is still doing next-token prediction, but it's prediction method is sufficiently complicated in terms of conditional probabilities that it recognizes that if you need to rhyme, you need to get to some future state, which then impacts the probabilities of the intermediate states. At least in my view it's still inherently a next-token predictor, just with r…
But then so are we? We are just predicting the next word we are saying, are we not? Even when you add thoughts behind it (sure some people think differently - be it without an inner monologue, or be it just in colors and sounds and shapes, etc), but that "reasoning" is still going into the act of coming up with the next word we are speaking/writing.
Earlier quoted context omitted.
A sufficiently large lookup table in DB is mathematically indistinguishable from a sufficiently complicated next token prediction algorithm is mathematically indistinguishable from general intelligence. All that means is that treating something as a black box doesn't tell you anything about what's inside the box.
Why do we care, so long as the box can genuinely reason about things?
Earlier quoted context omitted.
Rather when someone tweaks the content to avoid detection. Even today there are plenty of copyright material on youtube. They for example cut it in different ways to avoid detection.
"Everyone else is doing it" is not a valid infringement defense.
Earlier quoted context omitted.
I wonder why it would hallucinate Kamala being the president. Part of it is obviously that she was one of the candidates in 2024. But beyond that, why? Effectively a sentiment analysis maybe? More positive content about her? I think most polls had Trump ahead so you would have thought he'd be the guess from that perspective.
One way it might work: Up to it's knowledge cut off Biden, was president and a Democrat. It knows the current president is a democrat. It also knows that it's a bit further forward and that Kamala was running to be president and is Democrat. Ergo: the current president must be Kamala Harris.
I only vaguely follow the developments in LLMs, so this might be a dumb question. But my understanding was that LLMs have a fixed context window, and they don’t “remember” things outside of this. So couldn’t you theoretically just keep talking to an LLM until it forgets the system prompt? And as system prompts get larger and larger, doesn’t that “attack” get more and more viable?