Notice that the DNS example is parsing a binary DNS request, not a text file.
:) If I had my way this comment would be closer to the top. Not many grammar-based parsing systems can claim raw DNS parsing.
Colm programming language released: best parser-writer ever
31–40 of 45 posts
Re: Colm programming language released: best parser-writer ever
#32Earlier quoted context omitted.
To have both generalized parsing and context dependent parsing you need to somehow revert changes to the global state when you backtrack. You can do this by restoring it to old versions, which requires copying the state and keeping the history. The Colm approach is to keep only one global state, but store instructions for programmatically reverting the state while you backtrack.
The point of functional data structures is that if you modify, you don't copy the whole structure, you only copy the path to the things that changed. A simple example would be parsing a sequence of characters and storing that in a linked list. When you get an additional character you don't destructively modify the list, instead you create a new list node that points to the existing list. def process(char, state): ret…
Yes I understand these techniques, in fact they are heavily in use in Colm, just not for maintaining the global state that is used for the parsing feedback loop.
I use the term "copy" in the general sense to make discussion easier. Conceptually, persistent data structures are a copy, even if they are optimized heavily to incur very little cost.
Re: Colm programming language released: best parser-writer ever
#33Earlier quoted context omitted.
Where is the grammar-dependent scanning? Note that threading the context through the parse tree while maintaining fully generalized parsing requires keeping all versions of the parsing context in memory. Consider making a C++ parser in that way ... ie every time you modify the structures you build in memory you make a copy of them first.
If your data structures are persistent data structures you don't incur the costs of copying.
Indeed I did not follow what you meant. I missed the "persistent" and saw ... "don't incur the cost of copying." I took that to mean that you were referring to not maintaining any history and using a single, mutable global state. It is with that understanding that I made the previous comment.
To address what you meant ... it's more a matter of what's practical. For example, to parse C++ one needs to do quite a bit of semantic analysis during parsing. A full implementation of the type system is necessary. Then the data structures change with every statement/clause parsed. Achieving this with persistent data structures is far too costly.
Re: Colm programming language released: best parser-writer ever
#34I am more interested in DSNP, how come this project has not received more fame than the infamous Disapora? http://www.complang.org/dsnp/
There are some difficult problems in that space. I've posted to HN and reddit a few times, but mostly I've been working on it quietly so I can focus. Lately, that's starting to change. I'll be talking about it at FSW 11 in Berlin in a few weeks.
My google sense failed me this time around to find information on this FSW 11 in Berlin. Care to explain? Is it a conference, can anybody come?
I am really interested in DSNP and am fairly well versed in GNU/Linux and can do some programming, Java and Python mostly. I work as a web-frontend developer guy. Can I be of some help? Do you need testers, peers, documenters?
Re: Colm programming language released: best parser-writer ever
#35This kind of sounds like Semantic Design's DMS software Reengineering toolkit and the Parlanse language.
Re: Colm programming language released: best parser-writer ever
#36From my quick scan of the thesis, the basic design seems to be a programming language in which you write both the parser and any transformations you want to perform. It's not clear whether there is an easily-accessible parse tree serialization that you can use to load the output into another language, or whether you'd have to invent that yourself. I think it's generally a hard sell if you try to convince people that…
Colm has built-in serialization. There is still some work to do in this area though. Colm will preserve whitespace for minimal disruption of untransformed text, but figuring out what to do at the boundaries between modified and unmodified trees can be tricky. You are right, people want to use general purpose languages for the more complex algorithms. I agree a means of embedding is necessary and I have kept this in m…
Thanks for the info. What is the built-in serialization format?
Re: Colm programming language released: best parser-writer ever
#37From my quick scan of the thesis, the basic design seems to be a programming language in which you write both the parser and any transformations you want to perform. It's not clear whether there is an easily-accessible parse tree serialization that you can use to load the output into another language, or whether you'd have to invent that yourself. I think it's generally a hard sell if you try to convince people that…
What you're describing sounds like the Gold parser: http://www.devincook.com/goldparser/
Re: Colm programming language released: best parser-writer ever
#38Earlier quoted context omitted.
There are some difficult problems in that space. I've posted to HN and reddit a few times, but mostly I've been working on it quietly so I can focus. Lately, that's starting to change. I'll be talking about it at FSW 11 in Berlin in a few weeks.
My google sense failed me this time around to find information on this FSW 11 in Berlin. Care to explain? Is it a conference, can anybody come? I am really interested in DSNP and am fairly well versed in GNU/Linux and can do some programming, Java and Python mostly. I work as a web-frontend developer guy. Can I be of some help? Do you need testers, peers, documenters?
Ya it's currently hard to find. http://d-cent.org/fsw2011/
I need help from people like you actually. What I've done.
1. defined the protocol
2. implmented it in a C++ daemon that
a) talks to other daemons
b) serves the content managers (frontend UIs)
What needs to happen next is step 3 needs to be repeated by other people who know what they are doing. They don't need to understand the details of the protocol, they just need to understand the basic model, which is just message broadcast, distributed agreement, etc.
Email me for more details, will get back to you later tonight.
Re: Colm programming language released: best parser-writer ever
#39This kind of sounds like Semantic Design's DMS software Reengineering toolkit and the Parlanse language.
They are related systems. DMS is much more mature.
Re: Colm programming language released: best parser-writer ever
#40Earlier quoted context omitted.
Colm has built-in serialization. There is still some work to do in this area though. Colm will preserve whitespace for minimal disruption of untransformed text, but figuring out what to do at the boundaries between modified and unmodified trees can be tricky. You are right, people want to use general purpose languages for the more complex algorithms. I agree a means of embedding is necessary and I have kept this in m…
Thanks for the info. What is the built-in serialization format?
Just plain old text as it came in. I see now that is not what you were referring to. You're talking about JSON, XML, etc I now think.
There is also a print_xml function, which puts the tree into XML, but it's mostly used for debugging at this point, not export to other systems. I'm hoping that with time these kinds of features will crop up.