Live data from Hacker News

Dear sir, you have built a compiler

rachitnigam.com

21–30 of 177 posts

Re: Dear sir, you have built a compiler

#21

I'm always surprised by the lack of "easy" parser/compiler toolkit for more or less complete DSL... It looks like there's only - either full blown programming language for IT guys - "natural language" AI toolkits (not really usable yet for just simple automation by users) - graphical language like State Machine or "no code", missing loops and requiring mouse and boxes However, most of the time, user simply need some…

Sounds like a gap TCL could fill. The syntax is simple and consistent, a developer could create packages that a non-developer end user can use as commands, and it could be wrapped with GUI code that runs on Windows, Mac, or Linux.

Re: Dear sir, you have built a compiler

#22

I'm always surprised by the lack of "easy" parser/compiler toolkit for more or less complete DSL... It looks like there's only - either full blown programming language for IT guys - "natural language" AI toolkits (not really usable yet for just simple automation by users) - graphical language like State Machine or "no code", missing loops and requiring mouse and boxes However, most of the time, user simply need some…

Racket is often heralded as the "programming language programming language" and comes with tons of features to build full blown languages: https://racket-lang.org/ The most complete language/ecosystem that really showcases Racket's capabilities is Rosette IMO: https://github.com/emina/rosette/

Also worth noting that language that runs this very website was implemented in Racket.

Re: Dear sir, you have built a compiler

#23
Seems like the idea of exposing a programming language to the end-user should have better support so that it's more routine to create a restricted subset of something existing for users to put arbitrary code in. Something with both standardized textual interface and standardized structured editor interface would be great. I guess this was always the dream for LISP but it never got there.

Re: Dear sir, you have built a compiler

#24

I'm always surprised by the lack of "easy" parser/compiler toolkit for more or less complete DSL... It looks like there's only - either full blown programming language for IT guys - "natural language" AI toolkits (not really usable yet for just simple automation by users) - graphical language like State Machine or "no code", missing loops and requiring mouse and boxes However, most of the time, user simply need some…

There's Prolog and its Definite Clause Grammars (DCG) formalism. Here's a DCG for a tiny subset of natural English (copied from wikipedia [1]):

  sentence --> noun_phrase, verb_phrase.
  noun_phrase --> det, noun.
  verb_phrase --> verb, noun_phrase.
  det --> [the].
  det --> [a].
  noun --> [cat].
  noun --> [bat].
  verb --> [eats].
The syntax is just like BNF. "-->" is "::=", terms in []'s are terminals and the rest are nonterminals. Nonterminals can have arguments and there's special syntax to include entire Prolog programs in the body of rules.

I discovered DCGs at the end of my CS degree when I was working on my graduation project, a Magic: the Gathering expert system written in Prolog. I needed a way to script cards for my rules engine, to translate expressions in the M:tG ability text language to function calls in the engine. I really wanted to write a parser using the standard parser generator tools, but I thought it would be too much work for a graduate project. Then I found out about DCGs. With DCGs I could write grammar rules like this:

  tap(X,Y) --> (['Tap'];[tap]), target(X,Y), (condition ; []), {permanent_Type(Y)}.
  tap(X,Y) --> (['Tap'];[tap]), all(X,Y), (condition ; []).
And that would not only parse, but also _generate_ ability text like "Tap target Kavu" or "Tap all Goblins target player controls" etc. So as a side effect of adding a bunch of abilities to my engine, I also had a generator for new M:tG cards.

[Edit: to be clear, the grammar rules above are already, by themselves, a parser for abilities that start with "tap". The Prolog interpreter executes those grammar rules as a program and instantiates their variables in the process. To integrate them with the engine all I had to do was write engine functions that called the nonterminals in the grammar and read the values of variables like X and Y in tap(X,Y).]

Admittedly, my grammar was limited because I didn't have the time to hand-code a grammar for the entirety of M:tG ability text, as it was at the time (I graduated 2011). In any case my rules engine was also limited (I had no planeswalkers and static effects were only partially implemented, if I remember correctly; because the rules for static effects are a bit arcane). But I managed to cover a good subset of the rules and cards, so it was possible to play a basic game with creatures with activated and triggered abilities and with non-creature spells [2].

The important detail is that DCGs are syntactic sugar for ordinary Prolog, so I could code my parser in the same language as my engine and my player. My biggest problem was really to draw an imaginary line between the parser and the other two components, because the easiest thing would be to allow them to know about each other and be infinitely entangled in a big ball of mess.

Anyway, yeah, there's something out there that is perfectly suited for creating DSLs, if you have the patience [3]. The tradeoff is that you have to learn Prolog. I suspect this is a cost too high for many who just want an easy-to-use tool.

Btw, you can't easily run my degree project unfortunately. Like an idiot I wrote it for a proprietary Prolog system and although I tried to port it to the free and open-source SWI-Prolog, it's been a nightmare of incompatibilities. The ported project is here but it breaks unexpectedly all the time:

https://github.com/stassa/Gleemin

________

[1] https://en.wikipedia.org/wiki/Definite_clause_grammar#Exampl...

[2] I also had an alpha-beta minimax AI player. Not that good though. I wanted to program a player with background knowledge of M:tG but I didn't have the time.

[3] Or you can learn them from examples... See Experiment 3 on page 19 of the pdf, here:

https://link.springer.com/article/10.1007/s10994-020-05945-w

Re: Dear sir, you have built a compiler

#25
post #18

YAGNI is a good principle here. Whenever I've found myself thinking about reaching for a parser library, I was over-complicating or over-generalizing the problem. Write the code you need to solve the problem you actually have.

I agree completely. Anytime I’m looking at a parser library I just shake my head and close that browser tab. I’m invariably going to want a hand-rolled recursive descent parser two weeks later, so let’s just get on it.

Re: Dear sir, you have built a compiler

#26

I'm always surprised by the lack of "easy" parser/compiler toolkit for more or less complete DSL... It looks like there's only - either full blown programming language for IT guys - "natural language" AI toolkits (not really usable yet for just simple automation by users) - graphical language like State Machine or "no code", missing loops and requiring mouse and boxes However, most of the time, user simply need some…

Just use flex / bison?

Looks like these are grossly underappreciated these days. They are wonderful, easy to use tools.

Re: Dear sir, you have built a compiler

#27
post #12

I think you should almost always build a compiler when presented with a parse then execute problem. It is so much faster to create a small machine that is fed data to specific the program, and a test harness that also drives that machine, then it is to hand code every little case repetitively. Or maybe I like building little compilers.

I completely agree with the sentiment but also feel like it is just my bias because I enjoy writing compilers.

Re: Dear sir, you have built a compiler

#28
post #21

I'm always surprised by the lack of "easy" parser/compiler toolkit for more or less complete DSL... It looks like there's only - either full blown programming language for IT guys - "natural language" AI toolkits (not really usable yet for just simple automation by users) - graphical language like State Machine or "no code", missing loops and requiring mouse and boxes However, most of the time, user simply need some…

Sounds like a gap TCL could fill. The syntax is simple and consistent, a developer could create packages that a non-developer end user can use as commands, and it could be wrapped with GUI code that runs on Windows, Mac, or Linux.

Tcl is good, but non-developers could really benefit from more write-time checks. Something that can tell them "there's no command called moveRight, the nearest match is move-right" before their script is executed.

Re: Dear sir, you have built a compiler

#29
post #25
post #18

YAGNI is a good principle here. Whenever I've found myself thinking about reaching for a parser library, I was over-complicating or over-generalizing the problem. Write the code you need to solve the problem you actually have.

I agree completely. Anytime I’m looking at a parser library I just shake my head and close that browser tab. I’m invariably going to want a hand-rolled recursive descent parser two weeks later, so let’s just get on it.

So much this. I've used a parser generator successfully exactly once, and even then I simply didn't care about proper error reporting.

I wouldn't mind something reusable that generated red-green trees for me a la Roslyn. I can bang out a recursive descent parser in a day, but not one that can be run after every keystroke.

Re: Dear sir, you have built a compiler

#30
This happens to so many people, it’s a tale as old as programming languages. If you ever find yourself saying “wouldn’t it be neat…” or “if I could just only…”, stop yourself immediately — you may be about to accidentally spend the next 3-5 years writing a compiler. So many compilers started out looking to implement small feature X, and then ended up spending ungodly amounts of time writing an entire compiler instead.

The scariest thing about compilers is by writing one, it induces a kind of amnesia where you can’t remember why you started writing it in the first place! Soon enough the point of the compiler is to write the compiler, and feature X goes unimplemented. Eventually it will be possible once the compiler is done. But the compiler is never done… never… done…

Post reply on HN