Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

If one would to write auto-suggest for an IDE/code editor (like IntelliSense), does one generally need to write a parser for any language that should be supported, or do compilers/runtimes provide these things?

Neither the PHP or Python binaries seem to support just parsing files and generating some kind of raw output.

Eclipse seems to roll their own parser, but is this the only way to do it? Using the "native" parser would be awesome on so many levels :)

share|improve this question
    
Some languages do. Take a look at GCC-XML for C++ and maybe C. It might be able to do the other GCC languages too (like Fortran). For other languages, you could do some digging with their source code (like taking the source of the Ruby parser and turning it into a library), but that would be a lot more effort than you're looking for. –  Linuxios Aug 3 '12 at 23:56
    
Actually Python does, see the ast module –  Winston Ewert Aug 4 '12 at 15:28
    
JetBrains IntellJ uses ANTLR to parse all the supported languages for its auto complete. –  Jarrod Roberson Aug 4 '12 at 18:01
add comment

1 Answer

up vote 2 down vote accepted

PHP has token_get_all to parse a file and generate the list of tokens you can use to work with code, like doing static analysis, checking the style or implementing an auto-complete feature.

Python has Abstract Syntax Trees (AST), which seems even more helpful, and has also some neat features like the compile function which compiles the tree itself.

In general, it's a very bad idea to implement your own parser (aside from learning). It's hugely difficult and error prone and will become invalid as soon as the language specification changes. Such changes are unusual with well-designed languages like C#, but are not so unusual in languages like PHP which have lots of flaws and missing features (example: the recent implementation of namespaces in PHP). Also, by reinventing your own parser, you're reinventing the wheel already invented for a compiler: instead of doing exclusively your work (the auto-complete feature), you spend time writing lots of code related to parsing, the code you have to test and maintain later.

Some hints

You may be interested in a term of "compiler as a service". For example, Microsoft is working on a compiler as a service for C#, which will let you to extract programmatically the information from the compiler; such scenario might be helpful for an auto-complete feature.

You may also search for static checkers for the language of your interest. Many are open source, so looking at how they process the code source may give you some suggestions about the parsing.

Finally, some compilers are themselves open sourced. Depending on the license they use and the one you will use for your auto-complete-enabled product, you may be unable to reuse the code, but still it can give some hints too.

share|improve this answer
    
PHP's token_get_all was exactly what I was looking for! Thank you for all suggestions. –  Znarkus Aug 4 '12 at 12:33
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.