Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I use regular expressions on a daily basis, as my daily work is 90% in Perl (legacy codebase, but that's a different issue). Despite this, I still find lookahead and lookbehind to be terribly confusing and often unreadable. Right now, if I were to get a code review with a lookahead or lookbehind, I would immediately send it back to see if the problem can be solved by using multiple regular expressions or a different approach. The following are the main reasons I tend not to like them:

  • They can be terribly unreadable. Lookahead assertions, for example, start from the beginning of the string no matter where they are placed. That, among other things, can cause some very "interesting" and non-obvious behaviors.
  • It used to be the case that many languages didn't support lookahead/lookbehind (or supported them as "experimental features"). This isn't the case quite as much, but there's still always the question as to how well it's supported.
  • Quite frankly, they feel like a dirty hack. Regexps often already are, but they can also be quite elegant, and have gained widespread acceptance.
  • I've gotten by without any need for them at all... sometimes I think that they're extraneous.

Now, I'll freely admit that especially the last two reasons aren't really good ones, but I felt that I should enumerate what goes through my mind when I see one. I'm more than willing to change my mind about them, but I feel that they violate some of my core tenets of programming, including:

  • Code should be as readable as possible without sacrificing functionality -- this may include doing something in a less efficient, but clearer way as long as the difference is negligible or unimportant to the application as a whole.
  • Code should be maintainable -- if another programmer comes along to fix my code, non-obvious behavior can hide bugs or make functional code appear buggy (see readability)
  • "The right tool for the right job" -- I'm sure you can come up with contrived examples that could use lookahead, but I've never come across something that really needs them in my real-world development work. Is there anything that they're really the best tool for, as opposed to, say, multiple regexps (or, alternatively, are they the best tool for most cases they're used for today)?

My question is this: Is it good practice to use lookahead/lookbehind in regular expressions, or are they simply a hack that have found their way into modern production code?

I'd be perfectly happy to be convinced that I'm wrong about this, and simple examples are useful for examples or illustration, but by themselves, won't be enough to convince me.

share|improve this question
add comment

2 Answers

I still find lookahead and lookbehind to be terribly confusing and often unreadable.

You're aware that regular expressions can be exploded and commented, right?

$foo =~ m/^
  (?=.*a)           # must contain an a somewhere
  (?=.*c)           # must contain a c somewhere
  (?=.*1)           # must contain a 1 somewhere
  (?=.*2)           # must contain a 2 somewhere
  \S+               # all non-space characters
$/x

Is it good practice to use lookahead/lookbehind in regular expressions, or are they simply a hack that have found their way into modern production code?

They are quite indispensable, to avoid catastrophic backtracking and regex-related security issues. Ideally use plain atomic groups as well.

Compare how the above expression will backtrack, as compared with the naive equivalent:

$foo =~ m/^
  \S*a\S*c\S*1\S*2\S*      # a, then c, then 1, then 2
 |
  \S*a\S*c\S*2\S*1\S*      # a, c, 2, 1
 |
  \S*a\S*1\S*c\S*2\S*      # a, 1, c, 2
 |
  \S*a\S*1\S*2\S*c\S*      # a, 1, 2, c
 |
  # ... etc
$/x

Especially with a long input and a random sequence of a, c and 2 (no 1).

share|improve this answer
    
I see what you're getting at, but I'm not at all convinced. A couple of issues: (1) The links you gave to possible regex issues discuss badly-writen regular expressions, and don't even talk about lookahead as a possible solution. I'm aware of some of these concerns, but I don't see that lookahead is the best solution for them. (2) Your example regexp can trivially be rewritten with a few regexps and boolean logic. Maybe you can come up with an example that can't, but, honestly, I've never seen one in real code. I'm sure they exist; maybe lookahead should be reserved for those few cases? –  Greg Jackson Jun 24 '11 at 10:37
    
Once again, throwing examples at me isn't going to convince me without addressing my other concerns. Yes, regexps can be commented, but they usually aren't, and reasonably-sized comments don't always cover non-obvious behavior. If you have to write a small article in your code to explain how your code behaves and why, it's time to simplify. –  Greg Jackson Jun 24 '11 at 10:46
    
The article discusses atomic groups. Lookahead and look behind both are atomic. I only partially agree on regex comments. If a non-trivial comment is not enough then line split is insufficient. What expanded regex allows is to decompose a tricky branch into bite-sized expressions. Splitting and indenting should make them self-explanatory. –  Denis Jun 24 '11 at 13:34
    
It discusses Atomic Groups proper, and not any variations like lookahead and lookbehind. It's having the atomic operation that solves the issue. I could be much more easily convinced to accept plain Atomic Groups, because they don't break the flow of the regex, like lookahead and lookbehind often do, and aren't as prone to odd behavior. Once again, lookahead/lookbehind don't seem like the proper solution to these issues, but rather plain atomic groups. –  Greg Jackson Jun 24 '11 at 15:26
    
But then, lookahead/lookbehind are thrown away and won't be backtracked once used - i.e. they're atomic groups too. They're quite handy too in my experience. Apparently not in yours. Then again, if you've a simple solution to match exactly (and only) bar in foobarbaz but not in bar or bazbarfoo without using them, I'm all ears open. –  Denis Jun 27 '11 at 6:17
show 4 more comments

To take a very general view, you'll just have to weigh the trade-offs against each other. On one hand you have an advanced feature which is supported only by a handful implementations, which will be hard to read unless you're very proficient with it. On the other you have a (very likely longer) piece of code using simpler constructs. Which one is the best depends on the accessibility (directly related to the original programmer's and your own proficiency) and more abstract concerns such as accuracy and speed. As a general rule, my own opinion (after using regexes for about a decade):

  • Unless the code is throw-away (and we all know how that ends), use regexes sparingly. They are a bit like putting together an 8-dimensional model of the problem, and untangling them is accordingly difficult.
  • Don't equate a short regex with short processing time. A five-line substr/if-else construction could very well be faster, so try it before committing to an approach.
  • It often gets incredibly difficult to handle corner cases (witness all the people asking about regexes to parse HTML). Divide and conquer rather than eating the whole string regex-style.
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.