Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why XML tags are so fundamental to Claude

glthr.com

Post reply on HN
Print view View original

141–150 of 160 posts

Re: Why XML tags are so fundamental to Claude

#141

Post by OutOfHere »

ashirviskas wrote: post #123

Author does not know what they're talking about. > In other words, XML tags have not only a special place at inference level but also during training Their cited source has 0 proof of that. It's just like python/C/html in training. Doesn't mean it's special. And no, you don't need to format your prompts as python code just because of that. > In truth, it does not matter that these tags are XML. Other models use ad ho…

If an LLM were to struggle to closely follow instructions that weren't wrapped in XML, I would strongly consider it a sign of a poor model reflecting poor model training.

Re: Why XML tags are so fundamental to Claude

#142

Post by RandomBK »

sheept wrote: post #60
kid64 wrote: post #28

The thesis here seems to be that delimiters provide important context for Claude, and for that putpose we should use XML. The article even references English's built-in delimiter, the quotation mark, which is reprented as a token for Claude, part of its training data. So are we sure the lesson isn't simply to leverage delimiters, such as quotation marks, in prompts, period? The article doesn't identify any way in whi…

XML is a bit more special/first class to Claude because it uses XML for tool calling: /path/to/file 100 50 I'm sure Claude can handle any delimiter and pseudo markup you throw at it, but one benefit of XML delimiters over quotation marks is that you repeat the delimiter name at the end, which I'd imagine might help if its contents are long (it certainly helps humans).

How well do we understand the tokenization for Claude? I'd posit that the exact human-representation of this markup is likely irrelevant if it's all being converted into a single token.

Re: Why XML tags are so fundamental to Claude

#143

Post by xeyownt »

sheept wrote: post #60
kid64 wrote: post #28

The thesis here seems to be that delimiters provide important context for Claude, and for that putpose we should use XML. The article even references English's built-in delimiter, the quotation mark, which is reprented as a token for Claude, part of its training data. So are we sure the lesson isn't simply to leverage delimiters, such as quotation marks, in prompts, period? The article doesn't identify any way in whi…

XML is a bit more special/first class to Claude because it uses XML for tool calling: /path/to/file 100 50 I'm sure Claude can handle any delimiter and pseudo markup you throw at it, but one benefit of XML delimiters over quotation marks is that you repeat the delimiter name at the end, which I'd imagine might help if its contents are long (it certainly helps humans).

Cannot believe it's efficient. XML is the most verbose and inefficient of communicating anything. The only benefit of XML was to give lifetime work to an army of engineers. The next news will be "Why DTD is so fundamental to Claude".

Re: Why XML tags are so fundamental to Claude

#144

Post by bryanrasmussen »

paulddraper wrote: post #124
hbarka wrote: post #53

Earlier quoted context omitted.

If you think XML is old tech, wait until you hear of EDI, still powering Walmart and Amazon logistics. XML came in like a wrecking ball with its self-documenting promise designed to replace that cryptic pesky payload called EDI. XML promised to solve world hunger. It spawned SOAP, XML over RPC, DOM, DTD, the heyday was beautiful and Microsoft was leading the charge. C# was also right around this time. Consulting firm…

Imagine the worst data format you can think of. Then spend the next week making it even more convoluted. That data format is still better than EDI.

I'm not sure if this is a compliment or insult to my powers of invention.

Re: Why XML tags are so fundamental to Claude

#146

Post by imtringued »

xeyownt wrote: post #143
sheept wrote: post #60

Earlier quoted context omitted.

XML is a bit more special/first class to Claude because it uses XML for tool calling: /path/to/file 100 50 I'm sure Claude can handle any delimiter and pseudo markup you throw at it, but one benefit of XML delimiters over quotation marks is that you repeat the delimiter name at the end, which I'd imagine might help if its contents are long (it certainly helps humans).

Cannot believe it's efficient. XML is the most verbose and inefficient of communicating anything. The only benefit of XML was to give lifetime work to an army of engineers. The next news will be "Why DTD is so fundamental to Claude".

The point isn't to be efficient. If you train an LLM on code with an example execution trace written in the comments, the LLM gains a better understanding due to the additional context in the data. LLMs don't have a real world model. For them, the token space is the real world. All the information needs to be present in the training data and XML makes it easy because it is verbose and explicit about everything.

Re: Why XML tags are so fundamental to Claude

#147

Post by spider-mario »

dragonwriter wrote: post #136
phlakaton wrote: post #94

Earlier quoted context omitted.

HTML is far loosier-goosier in its syntax than XML allows. There was an attempt to nail its syntax down in the pre-HTML 5 days; that's XHTML. When HTML 5 pivoted away from that, that spelled the end of these two things ever coming together. Really, I think you can trace a lot of the "XML is spooky old technology" mindset to the release of HTML 5. That was when XML stopped being directly relevant to the web, though of…

> There was an attempt to nail its syntax down in the pre-HTML 5 days; that's XHTML. When HTML 5 pivoted away from that, that spelled the end of these two things ever coming together. Exactly the opposite; WHATWG “Living Standard” HTML (different releases of which were used as the basis for W3C HTML5, 5.1, and 5.2 before the W3C stopped doing that) includes an XML serialization as part of the spec, so now the HTML-in…

https://html.spec.whatwg.org/multipage/xhtml.html

“Warning! Using the XML syntax is not recommended, for reasons which include the fact that there is no specification which defines the rules for how an XML parser must map a string of bytes or characters into a Document object, as well as the fact that the XML syntax is essentially unmaintained — in that, it’s not expected that any further features will ever be added to the XML syntax (even when such features have been added to the HTML syntax).”

Re: Why XML tags are so fundamental to Claude

#148

Post by irthomasthomas »

apwheele wrote: post #15

I think XML is good to know for prompting (similar to how was popular for outputs, you can do that for other sections). But I have had much better experience just writing JSON and using line breaks, colons, etc. to demarcate sections. E.g. instead of .... ..... .... ... .... {actual input} Just doing something like: ...instructions... input: .... output: {..json here} ...maybe further instructions... input: {actual i…

The main thing i use xml tags for is seperating content from instructions. Say I am doing prompt engineering, so that the content being operated on is itself a prompt then I wrap it with

draft prompt

instructions for modifying draft prompt

If I don't do this, a significant number of times it responds to the instructions in the draft.

Re: Why XML tags are so fundamental to Claude

#149

Post by its-summertime »

arbirk wrote: post #145

If this is true, the why does Claude Code's own system prompt not use this style? https://github.com/Piebald-AI/claude-code-system-prompts/tre...

https://github.com/Piebald-AI/claude-code-system-prompts/blo... They seem to use XML-esque tags here in the first prompt I looked at

Re: Why XML tags are so fundamental to Claude

#150

Post by Lerc »

krackers wrote: post #109
zahlman wrote: post #104

Earlier quoted context omitted.

Couldn't you just insert tokens that don't correspond to any possible input, after the tokenization is performed? Unicode is bounded, but token IDs not so much.

This already happens, user vs system prompts are delimited in this manner, and most good frontends will treat any user input as "needing to be escaped" so you can never "prompt inject" your way into emitting a system role token. The issue is that you don't need to physically emit a "system role" token in order to convince the LLM that it's worth ignoring the system instructions.

>The issue is that you don't need to physically emit a "system role" token in order to convince the LLM that it's worth ignoring the system instructions.

My suspicion is that this failure happens for the same reason why I think the metadata would help with nesting. To take an electronic metaphor, special tokens are edge triggered signals, the metadata approach is signaled by level.

Special tokens are effively an edge but Internally, a transformer must turn the edge into level that propagates along with the context. You can attack this because it can decide by context that the level has been turned off.

You can see this happen in attacks that pre-seed responses with a few tokens accepting the prompt to override refusals. The refusal signal seems to last very few tokens before simply completing the text of the refusal because that's what it has started saying.

There's a paper showing how quickly the signal drops away, but I forget what it is called.

Post reply on HN