The language of languages
matt.might.net
The language of languages
1–10 of 20 posts
Re: The language of languages
#2Good introduction to Backus-Naur Form (BNF), Extended BNF (EBNF), Augmented BNF (ABNF, a la RFC 5234) and the typical extensions. Not earth shattering, but a good reference.
Re: The language of languages
#3I was thrown off by Might stating that "a grammar defines a language," which is not nearly as useful or factual as saying that it "describes" a language, the wording that he relies on throughout the rest of the article. That is the difference between me being able to make a dog or identify a dog based on a set of characteristics.
Grammars are only one part of understanding a language, hardly the "language of languages". In natural languages, grammars are one subset of linguistics. It would be just as valid to say vocabularies or phonology are the language of languages as it would be to say grammars are.
Other than these overly broad arguments and attempts to define natural languages in the same way that formal languages can be defined, this is a nice general introduction to some specific notation techniques for computer languages.
Of course, I might not have read it at all if it were titled "An Introduction to Backus-Naur Form, Extendend BNF, and Augmented BNF Notation Techniques".
Re: The language of languages
#4I just worked with a mathematician who understood grammars, and could write one, but could not use the theory in any practical way. Parsing is not just recognition. You need to pass info up the parse tree so that after parsing, you have a new data structure with everything nicely structured ready to ACT UPON. People write in specific languages to achieve something, not to simply create syntactically correct bodies of text. So understanding grammars is 50% of being able to use them as tools.
The other understanding you need to be able to put them to use is having a mental model of how a bottom up parser processes the tokens. You need to be able to insert actions for each grammar rule at appropriate places, which allows the information to flow bottom up too, and be processed appropriately at the same time. I have found this second bit to language processing is actually the harder bit, and its worth rewriting your grammar to make it simpler.
Re: The language of languages
#5Lisp is the language of languages.
Re: The language of languages
#6If you want to learn to apply grammars and parsers in practice I recommend the antlr parser-generator:
It can not only do the basic text->tree parsing from a file describing the grammar, but will also allow to specify additional grammars for traversing the generated tree and executing arbitrary code in your language of choice as particular nodes are recognized. I built a little compiler in it some years ago, I had a Xpl.g grammar file for parsing the program text and creating the abstract syntax tree, a SemanticAnalysis.g grammar file for doing a first pass through the tree, annotating it with additional information, filling the symbol table, checking semantic correctness and then finally CodeGeneration.g for emitting JVM bytecode using the annotated tree. The code for this is still on Github:
https://github.com/jaroslawr/xpl/tree/509120e66e23aac8493414...
There is a simpler example using the same functionalities described on the ANTLR wiki:
http://www.antlr.org/wiki/display/ANTLR3/Simple+tree-based+i...
I used the ANTLR reference book when learning it, but now there is also a real introductory manual:
http://pragprog.com/book/tpantlr2/the-definitive-antlr-4-ref... http://pragprog.com/book/tpdsl/language-implementation-patte...
I also used the Dragon Book and "Programming Language Pragmatics" for theory, both great books and PLP certainly deserves to be better known.
Re: The language of languages
#7I was thrown off by Might stating that "a grammar defines a language," which is not nearly as useful or factual as saying that it "describes" a language, the wording that he relies on throughout the rest of the article. That is the difference between me being able to make a dog or identify a dog based on a set of characteristics. Grammars are only one part of understanding a language, hardly the "language of language…
In formal language theory, '[formal] language' and '[formal] grammar' are well-defined mathematical terms, and it's indeed appropriate to say that a grammar defines language in that context.
Similarly, the 'language of languages' is also appropriate given that BNF is defined with a grammar, and is used to specify grammars.
Furthermore, formal grammars were originally invented for purposes of exploring natural languages.
Sorry if you didn't enjoy the article, but there's nothing wrong with it in the context of formal language theory :)
Re: The language of languages
#8BNFC is a compiler front end generator for labeled bnf grammars that can generate parsers and traversers for various languages. It uses Labaled BNF grammar. I'm not sure what the difference to regular grammars is since I only used this and it was a while ago. Some added expressiveness anyway:
Re: The language of languages
#9I was thrown off by Might stating that "a grammar defines a language," which is not nearly as useful or factual as saying that it "describes" a language, the wording that he relies on throughout the rest of the article. That is the difference between me being able to make a dog or identify a dog based on a set of characteristics. Grammars are only one part of understanding a language, hardly the "language of language…
Ya, grammar only describes the syntax of a language. You still have the semantics and pragmatics to consider!
I'm not even sure if a XBNF is the best way to describe or reason about language syntax. Precedence grammars (with hacks to handle braces) are quite interesting for robust error tolerant parsing, and might more closely mirror how we internal grammars in our head.
Re: The language of languages
#10If you want to learn to apply grammars and parsers in practice I recommend the antlr parser-generator: http://www.antlr.org/ It can not only do the basic text->tree parsing from a file describing the grammar, but will also allow to specify additional grammars for traversing the generated tree and executing arbitrary code in your language of choice as particular nodes are recognized. I built a little compiler in it so…
Thanks for the additional information on antlr. I stumbled upon it last month when doing some prep work for long-term planning at my company. I immediately wished I could rip out most of the custom code my team had written for handling ARFF files (for Weka machine learning toolkit) and replace it with the antlr grammar file attached to the Weka wiki.