Now, the big thing I'm working on is improving code quality while leaving the language working as it is(which I have a few regression tests to "prove"). It doesn't have a formal grammar at all and works like so:
- Preprocess/Tokenize. At this point it removes whitespace and cuts everything into "tokens", which is basically just a structure containing a string and a rough "hint" as to what the token is (Number, Identifier, Operation, etc) and some debugging info such as line number
- A ScriptingEngine class which takes the list of tokens and actually parses them and executes them
- An "ExpressionEvaluator" class which will take a subset of the tokens list and build a specific tree of operations, values, and then execute operations and such and collapse the tree down into a single value
My engine has the goals of being portable(works everywhere .Net does) and self-contained. So far, this "works", but the code is terrible and I'm pretty sure that I'm going about it the wrong way.
I'm wondering if a formal grammar and everything that goes with it might help
Some benefits I've heard of being more formal with grammar
- Unambiguous specification of the language
- Easier to maintain/change
- More traditional/Bigger community support?
And some of the disadvantages
- Some languages can be very difficult to reduce to a formal grammar, ie Perl.
- A learning curve for someone not in the know(ie, me)
- Generally rely on tools such as yacc and ANTLR, which introduce another step in your workflow and/or add dependencies(which I'd like to avoid)
Although this project is in .Net, it could equally apply to any other implementing language. Should I use a formal grammar? Can someone expand on the pros/cons of both sides?