Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why XML tags are so fundamental to Claude

glthr.com

Post reply on HN
Print view View original

101–110 of 160 posts

Re: Why XML tags are so fundamental to Claude

#101

Post by Lerc »

cyanydeez wrote: post #61
Lerc wrote: post #51

I am unconvinced. To me it seems like handling symbols that start and end sequences that could contain further start and end symbols is a difficult case. Humans can't do this very well either, we use visual aids such as indentation, synax hilighting or resort to just plain counting of levels. Obviously it's easy to throw parameters and training at the problem, you can easily synthetically generate all the XML trainin…

Basically, the only way you're separting user input from model meta-input is using some kind of character that'll never show up in the output of either users or LLMs. While technically possible, it'd be like a unicode conspiracy that had to quietly update everywhere without anyone being the wiser.

Not at all. You have a set of embeddings for the literal token, and a set for the metadata. At inference time all input gets the literal embedding, the metadata embedding can receive provenance data or nothing at all. You have a vector for user query in the metadata space. The inference engine dissallows any metadata that is not user input to be close to the user query vector.

Imagine a model finteuned to only obey instructions in a Scots accent, but all non user input was converted into text first then read out in a Benoit Blanc speech model. I'm thinking something like that only less amusing.

Re: Why XML tags are so fundamental to Claude

#102

Post by coldtea »

eduction wrote: post #72
coldtea wrote: post #44

Earlier quoted context omitted.

XML has been "spooky old technology" for over a decade now. It's heyday was something like 2002. Nobody dares advertise the XML capabilities of their product (which back then everybody did), nobody considers it either hot new thing (like back then) or mature - just obsolete enterprise shit. It's about as popular now as J2EE, except to people that think "10 years ago" means 1999.

Obsolete enterprise shit I guess includes podcasting. Impressive for the enterprise. I’d be very curious what lasting open formats JSON has been used to build.

That the podcast feed format is XML based is an insignificant detail - and a remnant of the past, nobody cares about.

People upload their podcasts to a platform like Apple Music or Spotify or Substack and co, or to some backend connected to their Wordpress/Ghost/etc) and it spits the RSS behind the scenes, with nobody giving a shit about the XML part.

Might as well declare USSR a huge IT success because people still play Tetris.

Re: Why XML tags are so fundamental to Claude

#103

Post by [deleted] »

wolttam wrote: post #35
jinushaun wrote: post #34

Earlier quoted context omitted.

Except quotation marks look like regular text. I regularly use quotes in prompts for, ya know, quotes.

The GP isn't suggesting to literally use quotes as the delimiter when prompting LLMs. They're pointing out that we humans already use delimiters in our natural language (quotation marks to delimit quotes). They're suggesting that delimiters of any kind may be helpful in the context of LLM prompting, which to me makes intuitive sense. That Claude is using XML is merely a convention.

[deleted]

Re: Why XML tags are so fundamental to Claude

#104

Post by zahlman »

cyanydeez wrote: post #61
Lerc wrote: post #51

I am unconvinced. To me it seems like handling symbols that start and end sequences that could contain further start and end symbols is a difficult case. Humans can't do this very well either, we use visual aids such as indentation, synax hilighting or resort to just plain counting of levels. Obviously it's easy to throw parameters and training at the problem, you can easily synthetically generate all the XML trainin…

Basically, the only way you're separting user input from model meta-input is using some kind of character that'll never show up in the output of either users or LLMs. While technically possible, it'd be like a unicode conspiracy that had to quietly update everywhere without anyone being the wiser.

Couldn't you just insert tokens that don't correspond to any possible input, after the tokenization is performed? Unicode is bounded, but token IDs not so much.

Re: Why XML tags are so fundamental to Claude

#105

Post by coldtea »

himata4113 wrote: post #70
coldtea wrote: post #44

Earlier quoted context omitted.

XML has been "spooky old technology" for over a decade now. It's heyday was something like 2002. Nobody dares advertise the XML capabilities of their product (which back then everybody did), nobody considers it either hot new thing (like back then) or mature - just obsolete enterprise shit. It's about as popular now as J2EE, except to people that think "10 years ago" means 1999.

didn't know html was spooky tech, TIL. /s

HTML predates XML by 5 years.

What's more, the web standards bodies even abandoned a short-lived XML-hype-era plan to make a new version of HTML based on XML in 2009.

That from this touted to the heavens format a handful of uses remain (some companies still using SOAP, the MS Office monster schemas, RSS, EPUB, and so on) is the very opposite of the adoption it was supposed to have. For those that missed the 90s/early 00s, XML was a hugely hyped format, with enormous corporate adoption between 1999–2005, which deflated totally.

Did you also learned those things too today?

Re: Why XML tags are so fundamental to Claude

#106

Post by array_key_first »

coldtea wrote: post #99
fc417fc802 wrote: post #84

Earlier quoted context omitted.

It's not the hot new thing but when has hype ever mattered for getting shit done? I don't think anyone who considers it obsolete has an informed opinion on the matter. Typically a more primitive (sorry, minimal) format such as JSON is sufficient in which case there's no excuse to overcomplicate things. But sometimes JSON isn't sufficient and people start inventing half baked solutions such as JSON-LD for what is alre…

> It's not the hot new thing but when has hype ever mattered for getting shit done? People who wanted to "get shit done" had much better alternatives. XML grew out of hype, corporate management forcing it, and bundling to all kinds of third party products and formats just so they can tick the "have this hot new format support" box.

XML is perfectly fine. What are these alternatives?

YAML is just bad. JSON is harder to read for deeply nested structures. TOML and the like don't have enough features.

Re: Why XML tags are so fundamental to Claude

#107

Post by krackers »

All system prompts are already wrapped in specific role markers (each LLM has its own unique format), so I'm sure every lab is familiar with the concept of delimters, in-band vs out-of-band signalling and such.

It'd not clear why within any section XML markers would do better than something like markdown, other than claude being explicitly post-trained with XML prompts as opposed to markdown. One hypothesis could be that since a large portion of the training corpus is websites, XML is more natural to use since it's "learned" the structure of XML better than markdown. Another could be that explicit start/end tags make identifying matching delimiters easier than JSON (which requires counting matching brackets) or markdown (where the end of a section is implicitly defined by the presence of a new header element).

Re: Why XML tags are so fundamental to Claude

#108

Post by krackers »

thesz wrote: post #65
scotty79 wrote: post #54

Earlier quoted context omitted.

Transformers look like perfect tech for keeping track of how deep and inside of what we are at the moment.

Transformers are able to recognize balanced brackets grammar at 97% success rate: https://openreview.net/pdf?id=kaILSVAspn This is 3% or infinitely far away from the perfect tech. The perfect tech is the stack.

This is very interesting since there is another notable paper which shows LLMs can recognize and generate CFGs

https://arxiv.org/abs/2305.13673

and of course a^n b^n is also classic CFG, so it's not clear why one paper had positive results while the other hand negative.

Re: Why XML tags are so fundamental to Claude

#109

Post by krackers »

zahlman wrote: post #104
cyanydeez wrote: post #61

Earlier quoted context omitted.

Basically, the only way you're separting user input from model meta-input is using some kind of character that'll never show up in the output of either users or LLMs. While technically possible, it'd be like a unicode conspiracy that had to quietly update everywhere without anyone being the wiser.

Couldn't you just insert tokens that don't correspond to any possible input, after the tokenization is performed? Unicode is bounded, but token IDs not so much.

This already happens, user vs system prompts are delimited in this manner, and most good frontends will treat any user input as "needing to be escaped" so you can never "prompt inject" your way into emitting a system role token.

The issue is that you don't need to physically emit a "system role" token in order to convince the LLM that it's worth ignoring the system instructions.

Re: Why XML tags are so fundamental to Claude

#110

Post by sham1 »

array_key_first wrote: post #106
coldtea wrote: post #99

Earlier quoted context omitted.

> It's not the hot new thing but when has hype ever mattered for getting shit done? People who wanted to "get shit done" had much better alternatives. XML grew out of hype, corporate management forcing it, and bundling to all kinds of third party products and formats just so they can tick the "have this hot new format support" box.

XML is perfectly fine. What are these alternatives? YAML is just bad. JSON is harder to read for deeply nested structures. TOML and the like don't have enough features.

Maybe ASN.1? Although that has an official XML encoding so maybe not.
Post reply on HN