Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I'm thinking of making a Python-JavaScript converter/compile. I've been wondering about the pros and cons of the following methods to parse the Pythonic code, into JavaScript equivalent.

  1. Working with the output of ast.dump(ast.parse(code))
  2. Working with tokens.

I've been trying weigh the pros and cons of these methods, but well I'm unable to think of any major one, as I've never used them..
So, Basically could someone help me by telling me what the pros and cons of the two methods I've mentioned above are?
Also, if there is something better that I can use for this purpose, notify me about it..

PS: I'll be using Python to write the converter..

share|improve this question
add comment

1 Answer

up vote 2 down vote accepted

Big con of tokens: You'll have to parse, and this is both unnecessary and nontrivial.

Big con of ast.dump(ast.parse(code)): It's pointlessly stupid. ast.parse gives you a perfectly good AST, structured in an appropriate data structure. ast.dump gives you a string representation of that data structure, so you'd have to parse that to get back what ast.parse already gave you!

Leaving out ast.dump and using the AST from ast.parse, there are two big pros:

  • You don't have to worry about parsing Python code at all (and doing so is moderately complex).
  • ASTs are really the minimum you need to look at for any compiler-ish project at this scale. While the ast AST may have some complexity you don't need (e.g. line information, or Python features you don't support), I assume that simplifying a Python AST to your custom AST is simpler than generating your custom AST from tokens.
  • A minor benefit: There's already a visitor class built into ast, so you don't have to create one (a simple but somewhat boring task) for easy AST traversal.

Cons? Not so much. Go for ast, but don't be afraid to use additional data structures beyond that.

share|improve this answer
    
So use ast!! Thx.. Any cons? –  Schoolboy May 11 '13 at 14:23
    
As I said, it may have some fluff that gets in your way, so you may want to immediately turn the AST into a different, simpler AST. –  delnan May 11 '13 at 14:25
    
I was thinking of using ast.dump as I wanted to parse a string, maybe implement some functions, and then evaluate the string.. –  Schoolboy May 11 '13 at 14:58
    
Token have flexibility, I can have the user type $, for using jquery.. –  Schoolboy May 12 '13 at 8:20
    
@Schoolboy Yes, although then you aren't parsing Python but a dialect of it, which would have been a useful piece of info to include in the question. Assuming you're using tokenize, you don't get unlimited flexibility either: $ produces an "error" token (and I wouldn't be surprised if the next refactoring of the tokenizer turns it into an actual error), and indentation is checked during tokenizing. Of course, if you don't use tokenize and write your own lexer, you're facing even more work... –  delnan May 12 '13 at 11:58
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.