Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Weird Lexical Syntax

justine.lol

Post reply on HN
Print view View original

111–120 of 234 posts

Re: Weird Lexical Syntax

#111

Post by jonahx »

panzi wrote: post #96
pdw wrote: post #22

Some random things that the author seem to have missed: > but TypeScript, Swift, Kotlin, and Scala take string interpolation to the furthest extreme of encouraging actual code being embedded inside strings Many more languages support that: C# $"{x} plus {y} equals {x + y}" Python f"{x} plus {y} equals {x + y}" JavaScript `${x} plus ${y} equals ${x + y}` Ruby "#{x} plus #{y} equals #{x + y}" Shell "$x plus $y equals $…

Is this a bash-ism? "$x plus $y equals $((x+y))"

This works in "sh" as well for me.

Re: Weird Lexical Syntax

#112

Post by jwilk »

panzi wrote: post #96
pdw wrote: post #22

Some random things that the author seem to have missed: > but TypeScript, Swift, Kotlin, and Scala take string interpolation to the furthest extreme of encouraging actual code being embedded inside strings Many more languages support that: C# $"{x} plus {y} equals {x + y}" Python f"{x} plus {y} equals {x + y}" JavaScript `${x} plus ${y} equals ${x + y}` Ruby "#{x} plus #{y} equals #{x + y}" Shell "$x plus $y equals $…

Is this a bash-ism? "$x plus $y equals $((x+y))"

No, it's portable shell syntax.

Re: Weird Lexical Syntax

#113

Post by murkt »

Author hasn’t tried to highlight TeX. Which is good for their mental health, I suppose, as it’s generally impossible to fully highlight TeX without interpreting it.

Even parsing is not enough, as it’s possible to redefine what each character does. You can make it do things like “and now K means { and C means }”.

Yes, you can find papers on arXiv that use this god-forsaken feature.

Re: Weird Lexical Syntax

#114

Post by jart »

murkt wrote: post #113

Author hasn’t tried to highlight TeX. Which is good for their mental health, I suppose, as it’s generally impossible to fully highlight TeX without interpreting it. Even parsing is not enough, as it’s possible to redefine what each character does. You can make it do things like “and now K means { and C means }”. Yes, you can find papers on arXiv that use this god-forsaken feature.

I wrote https://github.com/Mozilla-Ocho/llamafile/blob/main/llamafil... and it does a reasonable job highlighting without breaking for all the .tex files I could find on my hard drive. My goal is to hopefully cover 99.9% of real world usage, since that'll likely cover everything an LLM might output. Esoteric syntax also usually isn't a problem, so long as it doesn't cause strings and comments to extend forever, eclipsing the rest of the source code in a file.

Re: Weird Lexical Syntax

#115

Post by therein »

pdw wrote: post #22

Some random things that the author seem to have missed: > but TypeScript, Swift, Kotlin, and Scala take string interpolation to the furthest extreme of encouraging actual code being embedded inside strings Many more languages support that: C# $"{x} plus {y} equals {x + y}" Python f"{x} plus {y} equals {x + y}" JavaScript `${x} plus ${y} equals ${x + y}` Ruby "#{x} plus #{y} equals #{x + y}" Shell "$x plus $y equals $…

> PostgreSQL has the very convenient dollar-quoted strings

I did not know that. Today I learned.

Re: Weird Lexical Syntax

#117

Post by panzi »

Tarean wrote: post #102
panzi wrote: post #98

Earlier quoted context omitted.

Indeed in some of the listed languages you can nest it like that, but in others (e.g. Python) you can't. I would guess they deliberately don't want to enable that and it's not a problem in their parser or something.

As of python 3.6 you can nest fstrings. Not all formatters and highlighters have caught up, though. Which is fun, because correct highlighting depends on language version. Haskell has similar problems where different compiler flags require different parsers. Close enough is sufficient for syntax highlighting, though. Python is also a bit weird because it calls the format methods, so objects can intercept and react to…

I didn't mean nested f-strings. I mean this is a syntax error:

    >>> print(f"foo {"bar"}")
    SyntaxError: f-string: expecting '}'
Only this works:

    >>> print(f"foo {'bar'}")
    foo bar

Re: Weird Lexical Syntax

#118

Post by Izkata »

autarch wrote: post #63
pdw wrote: post #22

Some random things that the author seem to have missed: > but TypeScript, Swift, Kotlin, and Scala take string interpolation to the furthest extreme of encouraging actual code being embedded inside strings Many more languages support that: C# $"{x} plus {y} equals {x + y}" Python f"{x} plus {y} equals {x + y}" JavaScript `${x} plus ${y} equals ${x + y}` Ruby "#{x} plus #{y} equals #{x + y}" Shell "$x plus $y equals $…

Perl lets you do this too: my $foo = 5; my $bar = 'x'; my $quux = "I have $foo $bar\'s: @{[$bar x $foo]}"; print "$quux\n"; This prints out: I have 5 x's: xxxxx The "@{[...]}" syntax is abusing Perl's ability to interpolate an _array_ as well as a scalar. The inner "[...]" creates an array reference and the outer "@{...}" dereferences it. For reasons I don't remember, the Perl interpreter allows arbitrary code in the…

> For reasons I don't remember, the Perl interpreter allows arbitrary code in the inner "[...]" expression that creates the array reference.

...because it's an array value? Aside from how the languages handle references, how is that part any different from, for example, this in python:

  >>> [5 * 'x']
  ['xxxxx']
You can put (almost) anything there, as long as it's an expression that evaluates to a value. The resulting value is what goes into the array.

Re: Weird Lexical Syntax

#120

Post by quietbritishjim »

legobmw99 wrote: post #105
layer8 wrote: post #100

The author may have missed that lexing C is actually context-sensitive, i.e. you need a symbol table: https://en.wikipedia.org/wiki/Lexer_hack Of course, for syntax highlighting this is only relevant if you want to highlight the multiplication operator differently from the dereferencing operator, or declarations differently from expressions. More generally, however, I find it useful to highlight (say) types different…

I’d be shocked if jart didn’t know this, but it seems unlikely that an LLM would generate one of these most vexing parses, unless explicitly asked

I think you're thinking of something different to the issue in the parent comment. The most vexing parse is, as the name suggests, a problem at the parsing stage rather than the earlier lexing phase. Unlike the referenced lexing problem, it does't require any hack for compilers to deal with it. That's because it's not really a problem for the compiler; it's humans that find it surprising.
Post reply on HN