Live data from Hacker News

Weird Lexical Syntax

justine.lol

51–60 of 234 posts

Re: Weird Lexical Syntax

#51
Justine gets very close to the hairiest parsing issue in any language without encountering it:

Perl's syntax is undecidable, because the difference between treating some characters as a comment or as a regex can depend on the type of a variable that is only determined e.g. based on whether a search for a Collatz counterexample terminates, or just, you know, user input.

https://perlmonks.org/?node_id=663393

C++ templates have a similar issue, I think.

Re: Weird Lexical Syntax

#52
post #15

> TypeScript, Swift, Kotlin, and Scala take string interpolation to the furthest extreme of encouraging actual code being embedded inside strings. So to highlight a string, one must count curly brackets and maintain a stack of parser states. Presumably this is also true in Python - IIRC the brace-delimited fields within f-strings may contain arbitrary expressions. More generally, this must mean that the lexical gramm…

> Is it even possible to support generalized string interpolation within a strictly regular lexical grammar?

Almost certainly not, a fun exercise is to attempt to devise a Pumping tactic for your proposed language. If it doesn’t exist, it’s not regular.

https://en.m.wikipedia.org/wiki/Pumping_lemma_for_regular_la...

Re: Weird Lexical Syntax

#53

I think my favorite C trigraph was something like do_action() ??!??! handle_error() It almost looks like special error handling syntax but still remains satisfying once you realize it's an || logical-or statement and it's using short circuiting rules to execute handle error if the action returns a non-zero value.

Did you choose the legacy C trigraphs over || for aesthetic purposes?

Re: Weird Lexical Syntax

#54

This was a fun read, but it left me a bit more sympathetic to the lisp perspective, which (if I've understood it) is that syntax, being not an especially important part of a language, is more of a hurdle than a help, and should be as simple and uniform as possible so we can focus on other things. Which is sort of ironic because learning how to do structural editing on lisps has absolutely been more hurdle than help s…

I am surprised to hear that structural editing has been a hurdle for you, and I think I can offer a piece of advice. I also used to be terrified by its apparent complexity, but later found out that one just needs to use parinfer and to know key bindings for only three commands: slurp, barf, and raise.

With just these four things you will be 95% there, enjoying the fruits of paredit without any complexity — all the remaining tricks you can learn later when you feel like you’re fluent.

Re: Weird Lexical Syntax

#55

This was a fun read, but it left me a bit more sympathetic to the lisp perspective, which (if I've understood it) is that syntax, being not an especially important part of a language, is more of a hurdle than a help, and should be as simple and uniform as possible so we can focus on other things. Which is sort of ironic because learning how to do structural editing on lisps has absolutely been more hurdle than help s…

I am surprised to hear that structural editing has been a hurdle for you, and I think I can offer a piece of advice. I also used to be terrified by its apparent complexity, but later found out that one just needs to use parinfer and to know key bindings for only three commands: slurp, barf, and raise. With just these four things you will be 95% there, enjoying the fruits of paredit without any complexity — all the re…

Thanks very much for the advice, it's timely.

It's not so much the editing itself but the unfamiliarity of the ecosystem. It seems it's a square-peg I've been crafting a round hole of habits for it:

I guess I should use emacs? How to even configure it such that these actions are available? Or maybe I should write a plugin for helix so that I can be in a familiar environment. Oh, but the helix plugin language is a scheme, so I guess I'll use emacs until I can learn scheme better and then write that plugin. Oh but emacs keybinds are conflicting with what I've configured for zellij, maybe I can avoid conflicts by using evil mode? Oh ok, emacs-lisp, that's a thing. Hey symex seems like it aligns with my modal brain, oh but there goes another afternoon of fussing with emacs. Found and reported a symex "bug" but apparently it only appears in nix-governed environments so I guess I gotta figure out how to report the packaging bug (still todo). Also, I guess I might as well figure out how to get emacs to evaluate expressions based on which ones are selected, since that's one of the fun things you can do in lisps, but there's no plugin for the scheme that helix is using for its plugin language (which is why I'm learning scheme in the first place), but it turns out that AI is weirdly good at configuring emacs so now my emacs config contains most that that plugin would entail. Ok, now I'm finally ready to learn scheme, I've got this big list of new actions to learn: https://countvajhula.com/2021/09/25/the-animated-guide-to-sy.... Slurp, barf, and raise you say? excellent, I'll focus on those.

I'm not actually trying to critique the unfamiliar space. These are all self inflicted wounds: me being persnickety about having it my way. It's just usually not so difficult to use something new and also have it my way.

Re: Weird Lexical Syntax

#56
post #7

Another syntax oddity (not mentioned here) that breaks most highlighters: In Java, unicode escapes can be anywhere, not just in strings. For example, the following is a valid class: class Foo\u007b} and this assert will not trigger: assert // String literals can have unicode escapes like \u000A! "Hello World".equals("\u00E4");

I also argue that failing to syntax highlight this correctly is a security issue. You can terminate block comments with Unicode escapes, so if you wanted to hide some malicious code in a Java source file, you just need an excuse for there to be a block of Unicode escapes in a comment. A dev who doesn’t know about this quirk is likely to just skip over it, assuming it’s commented out.

Re: Weird Lexical Syntax

#57
> Perl also has this goofy convention for writing man pages in your source code

The world corpus of software would be much better documented if everywhere else had stolen this from Perl. Inline POD is great.

Re: Weird Lexical Syntax

#58
post #36

> Every C programmers (sic) knows you can't embed a multi-line comment in a multi-line comment. And every Standard ML programmer might find this to be a surprising limitation. The following is a valid Standard ML program: (* (* Nested (**) *) comment *) val _ = print "hello, world\n" Here is the output: $ sml Given how C was considered one of the "expressive" languages when it arrived, it's curious that nested commen…

Well there is one way to nest comments in C, and that's by using #if 0:

  #if 0
  This is a
  #if 0
  nested comment!
  #endif
  #endif

Re: Weird Lexical Syntax

#59

> Of all the languages, I've saved the best for last, which is Ruby. Now here's a language whose syntax evades all attempts at understanding. TeX with its arbitrarily reprogrammable lexer: how adorable

Lisp reader macros allow you to program its lexer too.

Re: Weird Lexical Syntax

#60

This was a fun read, but it left me a bit more sympathetic to the lisp perspective, which (if I've understood it) is that syntax, being not an especially important part of a language, is more of a hurdle than a help, and should be as simple and uniform as possible so we can focus on other things. Which is sort of ironic because learning how to do structural editing on lisps has absolutely been more hurdle than help s…

Lisp has reader macros which allow you to reprogram its lexer. Lisp macros allow you to program the translation from the visible structure to the parse tree.

For example, https://pyret.org/

It really isn’t simple or necessarily uniform.

Post reply on HN