Live data from Hacker News

PEP 750: Tag Strings for Writing Domain-Specific Languages

discuss.python.org

91–99 of 99 posts

Re: PEP 750: Tag Strings for Writing Domain-Specific Languages

#91
post #87

Earlier quoted context omitted.

Why could you not know these things without a language feature? > ...other stuff that happens before evaluation... A greet(string) function could parse the string and resolve the names itself: parsed = parser(string) resolved = resolver(parsed) return formatter(resolved) If you hate boilerplate, make the first two steps into a decorator. A PEP introducing a grand unified theory of magic (tag strings) isn't inherently…

If the string is an f-string, it is immediately evaluated and you no longer have access to the interpolation info for a resolver. If the string is not an f-string, you get no help from Python tooling. In both cases, you have to use frame hacks to get back to the scope, which has negative consequences.

> If the string is an f-string, it is immediately evaluated and you no longer have access to the interpolation info for a resolver.

So? It's been evaluated successfully. What more is there to do?

> If the string is not an f-string, you get no help from Python tooling.

Expose that tooling via the standard library. It's just pure functions.

> In both cases, you have to use frame hacks to get back to the scope, which has negative consequences.

What consequences? Isn't CPython forced to do all the nasty stuff anyhow when it's a language feature?

Re: PEP 750: Tag Strings for Writing Domain-Specific Languages

#92

Earlier quoted context omitted.

That would be an easy mental jump for skilled python users, but I think that it would be surprising for many less experienced users. Just giving it a function call syntax would be more in line with the principles established in PEP-20 IMO.

If I understand correctly, you'd prefer what other commenters have said for `html(i"Hello {name}")`?

Something like that, yeah.

Re: PEP 750: Tag Strings for Writing Domain-Specific Languages

#93

I LOVE tagged templates in JavaScript. But in Python I could also imagine YET ANOTHER constant prefix, like t"", that returns a "template" object of some sort, and then you could do html(t"") or whatever. That is, it would be just like f"" but return an object and not a string so you could get at the underlying values. Like in JavaScript the ability to see the original backslash escaping would be nice, and as an impr…

Cool to see you jump in, Ian. I don't particularly mind the prefix thing. It came up in the PEP discussion, as did choice of backticks to indicate this is different. But JS template literals -> tagged template literals shows, you can get from A to B without a fundamental change. I'm very interested though in the deferred part. I agree that there is complexity. I weigh that, though, against the complexity of existing…

Hey Paul!

I think JSX is an example of the somewhat crude but practical use of simple execution patterns. For instance if you have a loop you do:

    return 
      {items.map((item, i) => {item})}
    ;
Which isn't really templating at all, but just the ability to use inline expressions and easily construct objects does get the job done.

Or in a SQL builder with JavaScript tagged templates, I do:

    exec(sql`
      SELECT * FROM a_table
      WHERE category = ${category}
        ${subcategory ? sql`AND subcategory=${subcategory}` : sql``}
    `)
That is, I nest tagged templates to handle different logic conditions and loops.

If there's deferred execution, it's done with ?: and .map() – though these very long expressions don't work nearly as well in Python. (List comprehension is in some ways better than .map()/.filter(), but not for very large expressions like in a JSX template.)

Re: PEP 750: Tag Strings for Writing Domain-Specific Languages

#94
post #69

Earlier quoted context omitted.

There already is a syntax for writing log messages where the arguments are only evaluated when they are needed. logger.debug("bla: %s" % myvar).

Actually, the % syntax eagerly evaluates the log string, you need to pass the variables as arguments to the logging function like this: logger.debug("bla: %s", myvar). It's such a subtle difference that I only notice it when my IDE underlines it :/

You are right! I mixed them up as well. Here is a good source: https://blog.pilosus.org/posts/2020/01/24/python-f-strings-i... (not mine)

Re: PEP 750: Tag Strings for Writing Domain-Specific Languages

#95

Earlier quoted context omitted.

For a practical example of this technique used in JS take a look at libraries like htm and lit-html: https://github.com/developit/htm

(PEP co-author here) The htm folks did a Python implementation: https://pypi.org/project/htm/ It required a janky workaround for the absence of this PEP: https://pypi.org/project/tagged/ I used these to investigate ideas about component-driven development: https://viewdom.readthedocs.io/en/latest/

How is that a Janky workaround? It's not particularly ugly or verbose, nor does it seem to violate any python tenets.

Re: PEP 750: Tag Strings for Writing Domain-Specific Languages

#96
post #91

Earlier quoted context omitted.

If the string is an f-string, it is immediately evaluated and you no longer have access to the interpolation info for a resolver. If the string is not an f-string, you get no help from Python tooling. In both cases, you have to use frame hacks to get back to the scope, which has negative consequences.

> If the string is an f-string, it is immediately evaluated and you no longer have access to the interpolation info for a resolver. So? It's been evaluated successfully. What more is there to do? > If the string is not an f-string, you get no help from Python tooling. Expose that tooling via the standard library. It's just pure functions. > In both cases, you have to use frame hacks to get back to the scope, which ha…

Frame hacks with sys._getframe necessarily imply dynamic scope not lexical scope. Dynamic scope does not work with nested functions, including comprehensions. See this issue with the htm library, https://github.com/jviide/htm.py/issues/11

Re: PEP 750: Tag Strings for Writing Domain-Specific Languages

#97

Yikes. Don't get me wrong, I totally understand the reasoning why this would be useful (though I violently disagree with the idea of deferring the evaluation of the contained expressions), but it's also so very kitchensinky and adds so little over just calling a function (which doesn't require a 20-page explainer, as everyone already knows how function calls work). It also promotes using what looks like string interp…

> This seems really awkward to me for building stuff like the suggested use cases of XML/HTML or SQL templating. Compared to what? At the end of the day you're still doing string formatting, if you want parsing, then you'd feed the item into a parser, which this doesn't preclude. The interface sounds a lot better than JS's anyway, as that completely separates the literal strings and the interpolations so you have to…

The intent here is to support the following approach for tag function authors:

1. Parse to an AST, generally using an off-the-shelf parser. In practice, it's possible to rewrite interpolations with a placeholder suitable for a given language, eg x$Nx for HTML. Of course if that doesn't actually work, you might have to write/modify an existing parser. Hopefully we can cleverly avoid this extra work.

2. Walk/compile the AST, filling interpolations, but taking in account the context. This can for example take the form of building appropriate query strings that avoid Bobby Tables SQL injection, whether by mapping to SQL placeholders or with appropriate quoting (such as for a column or table name).

3. Memoize these steps, much as we see with builtin DSLs in Python, like the re module; see https://github.com/python/cpython/blob/3.12/Lib/re/__init__.... We do plan to make this easier/faster by supporting getting the original source of the template string (Template.source), vs the *args approach we show in the PEP at the start of this discussion (this will become Template.args instead; Template here is the proposed protocol of the object passed in).

Related is my post here: https://discuss.python.org/t/pep-750-tag-strings-for-writing...

Re: PEP 750: Tag Strings for Writing Domain-Specific Languages

#98

I tried skimming the PEP while I could, but it seems like this might be missing a couple of the features that make JS tagged template literals work so well: - tags get a strings array that's referentially stable across invocations. This can function as a cache key to cache strings parsing work. - tags can return any kind of value, not just a string. Often you need to give structured data to another cooperating API. D…

Thanks to the feedback in the Discourse thread, we will be changing the signature such that a tag function will be passed an object implementing a Template protocol, with attributes source, for the original source string as written; and attribute args for what was *args in the starting version of the PEP.

Template.source can act as a memoization key.

Re: PEP 750: Tag Strings for Writing Domain-Specific Languages

#99
post #56

... is this any different than a function like this: greet("hello {world}") which walks the call stack to find the variables, and uses them as locals / arguments? If so: why not just do that? I would expect the performance to be kinda terrible compared to a language-intrinsic, but this hardly seems like a thing worth truly optimizing. And if it's too costly at runtime, someone can implement a parse-time AST rewriter,…

Walking the call stack implies using dynamic scope, which has hard edges, vs lexical scope. See my answer earlier in this thread https://news.ycombinator.com/item?id=41272285

It's been nearly 23 years, but Python 2.2 fixed this issue (https://docs.python.org/3/whatsnew/2.2.html#pep-227-nested-s...), and it's also why JavaScript added let (and const). f-string support also uses lexical scope, and it's an important part of its success.

Post reply on HN