I'm always surprised by the lack of "easy" parser/compiler toolkit for more or less complete DSL... It looks like there's only - either full blown programming language for IT guys - "natural language" AI toolkits (not really usable yet for just simple automation by users) - graphical language like State Machine or "no code", missing loops and requiring mouse and boxes However, most of the time, user simply need some…
Dear sir, you have built a compiler
21–30 of 177 posts
Re: Dear sir, you have built a compiler
#22I'm always surprised by the lack of "easy" parser/compiler toolkit for more or less complete DSL... It looks like there's only - either full blown programming language for IT guys - "natural language" AI toolkits (not really usable yet for just simple automation by users) - graphical language like State Machine or "no code", missing loops and requiring mouse and boxes However, most of the time, user simply need some…
Racket is often heralded as the "programming language programming language" and comes with tons of features to build full blown languages: https://racket-lang.org/ The most complete language/ecosystem that really showcases Racket's capabilities is Rosette IMO: https://github.com/emina/rosette/
Re: Dear sir, you have built a compiler
#23Re: Dear sir, you have built a compiler
#24I'm always surprised by the lack of "easy" parser/compiler toolkit for more or less complete DSL... It looks like there's only - either full blown programming language for IT guys - "natural language" AI toolkits (not really usable yet for just simple automation by users) - graphical language like State Machine or "no code", missing loops and requiring mouse and boxes However, most of the time, user simply need some…
sentence --> noun_phrase, verb_phrase.
noun_phrase --> det, noun.
verb_phrase --> verb, noun_phrase.
det --> [the].
det --> [a].
noun --> [cat].
noun --> [bat].
verb --> [eats].
The syntax is just like BNF. "-->" is "::=", terms in []'s are terminals and the
rest are nonterminals. Nonterminals can have arguments and there's special
syntax to include entire Prolog programs in the body of rules.I discovered DCGs at the end of my CS degree when I was working on my graduation project, a Magic: the Gathering expert system written in Prolog. I needed a way to script cards for my rules engine, to translate expressions in the M:tG ability text language to function calls in the engine. I really wanted to write a parser using the standard parser generator tools, but I thought it would be too much work for a graduate project. Then I found out about DCGs. With DCGs I could write grammar rules like this:
tap(X,Y) --> (['Tap'];[tap]), target(X,Y), (condition ; []), {permanent_Type(Y)}.
tap(X,Y) --> (['Tap'];[tap]), all(X,Y), (condition ; []).
And that would not only parse, but also _generate_ ability text like "Tap target
Kavu" or "Tap all Goblins target player controls" etc. So as a side effect of
adding a bunch of abilities to my engine, I also had a generator for new M:tG
cards.[Edit: to be clear, the grammar rules above are already, by themselves, a parser for abilities that start with "tap". The Prolog interpreter executes those grammar rules as a program and instantiates their variables in the process. To integrate them with the engine all I had to do was write engine functions that called the nonterminals in the grammar and read the values of variables like X and Y in tap(X,Y).]
Admittedly, my grammar was limited because I didn't have the time to hand-code a grammar for the entirety of M:tG ability text, as it was at the time (I graduated 2011). In any case my rules engine was also limited (I had no planeswalkers and static effects were only partially implemented, if I remember correctly; because the rules for static effects are a bit arcane). But I managed to cover a good subset of the rules and cards, so it was possible to play a basic game with creatures with activated and triggered abilities and with non-creature spells [2].
The important detail is that DCGs are syntactic sugar for ordinary Prolog, so I could code my parser in the same language as my engine and my player. My biggest problem was really to draw an imaginary line between the parser and the other two components, because the easiest thing would be to allow them to know about each other and be infinitely entangled in a big ball of mess.
Anyway, yeah, there's something out there that is perfectly suited for creating DSLs, if you have the patience [3]. The tradeoff is that you have to learn Prolog. I suspect this is a cost too high for many who just want an easy-to-use tool.
Btw, you can't easily run my degree project unfortunately. Like an idiot I wrote it for a proprietary Prolog system and although I tried to port it to the free and open-source SWI-Prolog, it's been a nightmare of incompatibilities. The ported project is here but it breaks unexpectedly all the time:
https://github.com/stassa/Gleemin
________
[1] https://en.wikipedia.org/wiki/Definite_clause_grammar#Exampl...
[2] I also had an alpha-beta minimax AI player. Not that good though. I wanted to program a player with background knowledge of M:tG but I didn't have the time.
[3] Or you can learn them from examples... See Experiment 3 on page 19 of the pdf, here:
https://link.springer.com/article/10.1007/s10994-020-05945-w
Re: Dear sir, you have built a compiler
#25YAGNI is a good principle here. Whenever I've found myself thinking about reaching for a parser library, I was over-complicating or over-generalizing the problem. Write the code you need to solve the problem you actually have.
Re: Dear sir, you have built a compiler
#26I'm always surprised by the lack of "easy" parser/compiler toolkit for more or less complete DSL... It looks like there's only - either full blown programming language for IT guys - "natural language" AI toolkits (not really usable yet for just simple automation by users) - graphical language like State Machine or "no code", missing loops and requiring mouse and boxes However, most of the time, user simply need some…
Just use flex / bison?
Re: Dear sir, you have built a compiler
#27I think you should almost always build a compiler when presented with a parse then execute problem. It is so much faster to create a small machine that is fed data to specific the program, and a test harness that also drives that machine, then it is to hand code every little case repetitively. Or maybe I like building little compilers.
Re: Dear sir, you have built a compiler
#28I'm always surprised by the lack of "easy" parser/compiler toolkit for more or less complete DSL... It looks like there's only - either full blown programming language for IT guys - "natural language" AI toolkits (not really usable yet for just simple automation by users) - graphical language like State Machine or "no code", missing loops and requiring mouse and boxes However, most of the time, user simply need some…
Sounds like a gap TCL could fill. The syntax is simple and consistent, a developer could create packages that a non-developer end user can use as commands, and it could be wrapped with GUI code that runs on Windows, Mac, or Linux.
Re: Dear sir, you have built a compiler
#29YAGNI is a good principle here. Whenever I've found myself thinking about reaching for a parser library, I was over-complicating or over-generalizing the problem. Write the code you need to solve the problem you actually have.
I agree completely. Anytime I’m looking at a parser library I just shake my head and close that browser tab. I’m invariably going to want a hand-rolled recursive descent parser two weeks later, so let’s just get on it.
I wouldn't mind something reusable that generated red-green trees for me a la Roslyn. I can bang out a recursive descent parser in a day, but not one that can be run after every keystroke.
Re: Dear sir, you have built a compiler
#30The scariest thing about compilers is by writing one, it induces a kind of amnesia where you can’t remember why you started writing it in the first place! Soon enough the point of the compiler is to write the compiler, and feature X goes unimplemented. Eventually it will be possible once the compiler is done. But the compiler is never done… never… done…