Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I have a scripting engine I just published as an open source project. It's been sitting on my harddrive waiting for about a year. My engine of course isn't complete in any way, but it does work for simple scripts. It has a javascript-ish feel to it, but I don't wish to abide by the ECMA spec or anything.

Now, the big thing I'm working on is improving code quality while leaving the language working as it is(which I have a few regression tests to "prove"). It doesn't have a formal grammar at all and works like so:

  1. Preprocess/Tokenize. At this point it removes whitespace and cuts everything into "tokens", which is basically just a structure containing a string and a rough "hint" as to what the token is (Number, Identifier, Operation, etc) and some debugging info such as line number
  2. A ScriptingEngine class which takes the list of tokens and actually parses them and executes them
  3. An "ExpressionEvaluator" class which will take a subset of the tokens list and build a specific tree of operations, values, and then execute operations and such and collapse the tree down into a single value

My engine has the goals of being portable(works everywhere .Net does) and self-contained. So far, this "works", but the code is terrible and I'm pretty sure that I'm going about it the wrong way.

I'm wondering if a formal grammar and everything that goes with it might help

Some benefits I've heard of being more formal with grammar

  • Unambiguous specification of the language
  • Easier to maintain/change
  • More traditional/Bigger community support?

And some of the disadvantages

  • Some languages can be very difficult to reduce to a formal grammar, ie Perl.
  • A learning curve for someone not in the know(ie, me)
  • Generally rely on tools such as yacc and ANTLR, which introduce another step in your workflow and/or add dependencies(which I'd like to avoid)

Although this project is in .Net, it could equally apply to any other implementing language. Should I use a formal grammar? Can someone expand on the pros/cons of both sides?

share|improve this question
Perl is an example of why you should have a grammar. The real counterexample is perhaps Tcl, which uses a syntactic approach much like you're doing; its formal grammar is keyword-free. (Of course, the real question is “Why are you making another language? What's your unique hook?”) –  Donal Fellows Mar 25 '13 at 8:49
A formal grammar might help you reveal syntactic ambiguities and pitfals (precedences, etc.) on the other hand you have to keep it in sync with your lexer/parser. –  wirrbel Mar 25 '13 at 8:58
@DonalFellows My unique hook is "because I want to" :) –  Earlz Mar 25 '13 at 14:18

2 Answers 2

First of all, a formal grammar doesn't imply you have to use a parser generator. You can write a recursive descent parser or whatever floats your boat.

Formal grammars are good as documentation. They help programmers decide what foreign code means and tells you what is allowed / mandatory in the language.

For example, in javascript, do I always have to wrap a function literal in parens if I want to call it right away? That is, is this allowed:

function(a1,a2) { ... }(42, "bacon").

Formal grammars are also very helpful when you want to

  1. Make another implementation of the language,
  2. Do code generation targeting the language,
  3. Or for doing static analysis of the language.

Of course, if it's a quick and dirty scripting language these points are less important.

share|improve this answer

Based on what I think your scripting language is like, I don't really see a benefit for you to formalize it.

I think a good practical alternative would be, when you do a new feature, document it first, make sure it is consistent, then write tests scripts for it, then finally implement it.

That should achieve enough consistency for practical purposes. Big part of consistency is library (or comparable built-in stuff), and having a formalized grammar does not help one bit with that anyway.

Whether it makes any sense to develop YASL, I'll not comment on that...

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.