Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I am trying to learn Clojure, which seems a good candidate for a successful LISP. I have no problem with the concepts, but now I would like to start actually doing something.

Here it comes my problem. As I mainly do web stuff, I have been looking into existing frameworks, database libraries, templating libraries and so on. Often these libraries are heavily based on macros.

Now, I like very much the possibility of writing macros to get a simpler syntax than it would be possible otherwise. But it definitely adds another layer of complexity. Let me take an example of a migration in Lobos from a blog post:

(defmigration add-posts-table
  (up [] (create clogdb
           (table :posts (integer :id :primary-key )
             (varchar :title 250)
             (text :content )
             (boolean :status (default false))
             (timestamp :created (default (now)))
             (timestamp :published )
             (integer :author [:refer :authors :id] :not-null))))
  (down [] (drop (table :posts ))))

It is very readable indeed. But it is hard to recognize what the structure is. What does the function timestamp return? Or is it a macro?

Having all this freedom of writing my own syntax means that I have to learn other people's syntax for every library I want to use.

How can I learn to use these components effectively? Am I supposed to learn each small DSL as a black box?

share|improve this question
4  
It does not add another layer of complexity, but another layer of abstractions which is quite the opposite. –  EricSchaefer Mar 15 '12 at 12:08
2  
looking at that code, I would expect timestamp to be a column in the table. –  Paul Nathan Mar 15 '12 at 17:04
    
Of course it is a column in the table. This is precisely my point. The example is indeed very readable, and one can tell at a glance what this code will do. But the syntax seems weird and it does not look like it matches what is really going on. –  Andrea Mar 16 '12 at 8:23
add comment

3 Answers

up vote 8 down vote accepted

Lisp languages (at least the good ones) usually come with a lot of tools to help you. If you have trouble understanding what exactly a macro does, macroexpand it(C-c C-m in slime), or jump to its definition (M-. in slime). I would suggest to read up on clojures introspection capabilities (I'm not as familiar with them as I am with Common Lisps, which is why my answer is mostly dialect agnostic, but I know it has macroexpand) and learn more about slime (if you're using it, which you should). And of course read the docs, if you're too lazy to read docs, practice speed reading.

Also you seem to be new to lisp, after close to 3.5 years of playing around with various lisps, other peoples macros rarely confuse me. Most macros use common patterns, definitions, new scopes of some kind, control flow etc. The macro you show as an example looks pretty obvious to me (note I haven't touched clojure in over a year and I've never used this particular library). I remember feeling a similar kind of frustration two years ago.

So in the end my advice is to just go ahead and learn more lisp. Find tools that can help, people to advise you and just get your hands dirty with a lot of code (writing but also a lot of reading). Hopefully you'll get there eventually. I assure you Clojure and Common Lisp are languages that reward you the better you know them. Steep learning curves are steep because you're climbing higher.

share|improve this answer
3  
+1 Steep learning curves are steep because you're climbing higher. –  Joonas Pulakka Mar 16 '12 at 7:33
add comment

Am I supposed to learn each small DSL as a black box?

I guess that's the idea of DSLs: encapsulating details so that things are simpler from the domain point of view. But, like you've experienced, whether something is easy or not, depends on how familiar it is, not on how simple it is. So, to make things manageable, yes, you have to learn each small DSL used. They should be black boxes, but in practice they're often grey and leaky, so in the end you may have two things to handle: the abstractions themselves, and the underlying details leaking in and out...

This boils down to balance. Macros should be used very sparingly. To justify a macro, it should provide a major benefit compared to using a bit more complex, but familiar, syntax.

Unfortunately, what I've seen, people do use macros way too much, trying to be a bit too smart. Maybe the problem is in the nature of open source: people are making software primarily for themselves, and for them the macro provides value, but for others it's merely one more headache.

Personally I prefer coding close to what I understand. Working on top of custom macros whose internals I don't quite understand is horrible. Particularly, when something goes awry, I'm screwed.

Language core constructs can well be macros (for example, Clojure's when is a macro). They're familiar anyway, and learning how they actually work is a great learning experience.

share|improve this answer
    
Yes, you are essentially confirming what my experience has been. I know how to implement most core macros. But I do not want to be forced to learn a new syntax for every new library. Am I at a loss? –  Andrea Mar 15 '12 at 11:58
    
Well, if a new library uses a new syntax and you don't want to learn it... If there's an alternative, I'm interested as well! –  Joonas Pulakka Mar 15 '12 at 12:03
6  
If you use any other library you have to learn the API too. Libraries which call them self DSLs only have a fancier API. –  EricSchaefer Mar 15 '12 at 12:03
1  
Yes, I have to learn the API anyway. But at least the API can be documented more easily: here is this function, it takes this stuff and returns this other stuff. Macros, on the other hand, are not so easily categorized. –  Andrea Mar 15 '12 at 13:18
1  
Well, of course some documentation is needed. But although now appears to be executed right away, it is pretty clear it will be executed at insertion time. This means probably default is a macro that wraps now or something like that. In any case, it looks like a layer of magic. This means that learning to use this library will require me to either rely on the magic or learn the internals of the library, whereas traditional libraries do not require this. –  Andrea Mar 15 '12 at 17:52
show 4 more comments

Sounds like you're in the process of rediscovering that with great power comes great responsibility. Lisp is a very sharp tool, you must use it wisely.

I've found the following tricks useful in Lisp / Clojure:

  • (doc some-function) is your friend. Well-written libraries and DSLs usually have decent docstrings that you can use to work out what a particular function or macro is doing.
  • Avoid mixing DSLs - in any single part of the code base, you should try to only use one DSL. That way, you know that a certain block of code is following the logic and conventions of that DSL.
  • Don't create DSLs for the sake of it - defining a DSL adds a layer of abstraction / conceptual overhead. You can do a lot in Clojure with simple functions. Don't create a DSL unless you are really sure you need it and know why it is better than coding in plain idiomatic Clojure.
  • Test your assumptions by writing a test - if you are a bit confused about how a particular part of a DSL is working, then write a test case that uses it. This will verify or force your to re-evaluate your assumption, plus you get an extra test case as a bonus :-)
  • If you bring in a library DSL, make sure you understand the core concepts - DSLs are languages in their own right, and you need to recognise that learning a new language is not a trivial task. I personally find the most important thing is to appreciate the core concepts of the DSL and what it is trying to achieve (usually by study / writing examples). You can always look up specific details later, but if you misunderstand the whole point of the DSL then you may find yourself in the painful position of trying to bend it into doing something it wasn't designed for.
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.