As for C#'s triple-quoted strings, they actually came from Java before and C# ended up adopting the same or almost the same semantics. Including stripping leading whitespace.
Weird Lexical Syntax
21–30 of 234 posts
Re: Weird Lexical Syntax
#22Some random things that the author seem to have missed:
> but TypeScript, Swift, Kotlin, and Scala take string interpolation to the furthest extreme of encouraging actual code being embedded inside strings
Many more languages support that:
C# $"{x} plus {y} equals {x + y}"
Python f"{x} plus {y} equals {x + y}"
JavaScript `${x} plus ${y} equals ${x + y}`
Ruby "#{x} plus #{y} equals #{x + y}"
Shell "$x plus $y equals $(echo "$x+$y" | bc)"
Make :) echo "$(x) plus $(y) equals $(shell echo "$x+$y" | bc)"
Tcl is funny because comments are only recognized in code, and since it's a homoiconic, it's very hard to distinguish code and data. { } are just funny string delimiters. E.g.:
xyzzy {#hello world}
> SQL
PostgreSQL has the very convenient dollar-quoted strings: https://www.postgresql.org/docs/current/sql-syntax-lexical.h... E.g. these are equivalent:
'Dianne''s horse'
$$Dianne's horse$$
$SomeTag$Dianne's horse$SomeTag$Re: Weird Lexical Syntax
#23In the C# multiquoted strings, how does it know this: Console.WriteLine(""""""); Console.WriteLine(""""""); Are 2 triplequoted empty strings and not one "\nConsole.WriteLine(" sixtuplequoted string?
The former, I'd say.
https://learn.microsoft.com/en-us/dotnet/csharp/programming-...
For a multi-line string the quotes have to be on their own line.
Re: Weird Lexical Syntax
#24In the C# multiquoted strings, how does it know this: Console.WriteLine(""""""); Console.WriteLine(""""""); Are 2 triplequoted empty strings and not one "\nConsole.WriteLine(" sixtuplequoted string?
If the opening quotes are followed by anything that is not a whitespace before the next new-line (or EOF), then it's a single-line string.
I imagine implementing those things took several iterations :)
Re: Weird Lexical Syntax
#25As soon as I saw this was part of llamafile I was hoping that it would be used to limit LLM output to always be "valid" code as soon as it saw the backticks, but I suppose most LLMs don't have problems with that anyway. And I'm not sure you'd want something like that automatically forcing valid code anyway
llama.cpp does support something like this -- you can give it a grammar which restricts the set of available next tokens that are sampled over
so in theory you could notice "```python" or whatever and then start restricting to valid python code. (in least in theory, not sure how feasible/possible it would be in practice w/ their grammar format.)
for code i'm not sure how useful it would be since likely any model that is giving you working code wouldn't be struggling w/ syntax errors anyway?
but i have had success experimentally using the feature to drive fiction content for a game from a smaller llm to be in a very specific format.
Re: Weird Lexical Syntax
#26In the C# multiquoted strings, how does it know this: Console.WriteLine(""""""); Console.WriteLine(""""""); Are 2 triplequoted empty strings and not one "\nConsole.WriteLine(" sixtuplequoted string?
It's a syntax error!
Unterminated raw string literal.
Re: Weird Lexical Syntax
#27Some random things that the author seem to have missed: > but TypeScript, Swift, Kotlin, and Scala take string interpolation to the furthest extreme of encouraging actual code being embedded inside strings Many more languages support that: C# $"{x} plus {y} equals {x + y}" Python f"{x} plus {y} equals {x + y}" JavaScript `${x} plus ${y} equals ${x + y}` Ruby "#{x} plus #{y} equals #{x + y}" Shell "$x plus $y equals $…
[deleted]
Re: Weird Lexical Syntax
#28Meanwhile NeoVim doesn’t syntax highlight my commit message properly if I have messed with "commit cleanup" enough. The comment character in Git commit messages can be a problem when you insist on prepending your commits with some "id" and the id starts with `#`. One suggestion was to allow backslash escapes in commit messages since that makes sense to a computer scientist.[1] But looking at all of this lexical stuff…
> The comment character in Git commit messages can be a problem when you insist on prepending your commits with some "id" and the id starts with `#` https://git-scm.com/docs/git-commit#Documentation/git-commit...
See "commit cleanup".
There’s surprising layers to this. That the reporter in that thread says that git-commit will “happily” accept `#` in commit messages is half-true: it will accept it if you don’t edit the message since the `default` cleanup (that you linked to) will not remove comments if the message is given through things like `-m` and not an editing session. So `git commit -m'#something' is fine. But then try to do rebase and cherry-pick and whatever else later, maybe get a merge commit message with a commented "conflicted" files. Well it can get confusing.
Re: Weird Lexical Syntax
#29I’d be interested to see a re-usable implementation of joe's[0] syntax highlighting.[1] The format is powerful enough to allow for the proper highlighting of Python f-strings.[2] 0. https://joe-editor.sf.net/ 1. https://github.com/cmur2/joe-syntax/blob/joe-4.4/misc/HowItW... 2. https://gist.github.com/irdc/6188f11b1e699d615ce2520f03f1d0d...
Interestingly, python f-strings changed their syntax at version 3.12, so highlighting should depend on the version.
Re: Weird Lexical Syntax
#30I’d be interested to see a re-usable implementation of joe's[0] syntax highlighting.[1] The format is powerful enough to allow for the proper highlighting of Python f-strings.[2] 0. https://joe-editor.sf.net/ 1. https://github.com/cmur2/joe-syntax/blob/joe-4.4/misc/HowItW... 2. https://gist.github.com/irdc/6188f11b1e699d615ce2520f03f1d0d...
Interestingly, python f-strings changed their syntax at version 3.12, so highlighting should depend on the version.
It’s just that nesting them arbitrarily is now allowed, right? That shouldn’t matter much for a mere syntax highlighter then. And one could even argue that code that relies on this too much is not really for human consumption.