Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

The Setting

I often have trouble determining when and how to use exceptions. Let's consider a simple example: suppose I am scraping a webpage, say "http://www.abevigoda.com/", to determine if Abe Vigoda is still alive. To do this, all we need to do is download the page and look for times that the phrase "Abe Vigoda" appears. We return the first appearance, since that includes Abe's status. Conceptually, it will look like this:

def get_abe_status(url):
    # download the page
    page = download_page(url)

    # get all mentions of Abe Vigoda
    hits = page.find_all_mentions("Abe Vigoda")

    # parse the first hit for his status
    status = parse_abe_status(hits[0])

    # he's either alive or dead
    return status == "alive"

Where parse_abe_status(s) takes a string of the form "Abe Vigoda is something" and returns the "something" part.

Before you argue that there are much better and more robust ways of scraping this page for Abe's status, remember that this is just a simple and contrived example used to highlight a common situation I'm in.

Now, where can this code encounter problems? Among other errors, some "expected" ones are:

  • download_page might not be able to download the page, and throws an IOError.
  • The URL might not point to the right page, or the page is downloaded incorrectly, and so there are no hits. hits is the empty list, then.
  • The web page has been altered, possibly making our assumptions about the page wrong. Maybe we expect 4 mentions of Abe Vigoda, but now we find 5.
  • For some reasons, hits[0] might not be a string of the form "Abe Vigoda is something", and so it cannot be correctly parsed.

The first case isn't really a problem for me: an IOError is thrown and can be handled by the caller of my function. So let's consider the other cases and how I might handle them. But first, let's assume that we implement parse_abe_status in the stupidest way possible:

def parse_abe_status(s):
    return s[13:]

Namely, it doesn't do any error checking. Now, on to the options:

Option 1: Return None

I can tell the caller that something went wrong by returning None:

def get_abe_status(url):
    # download the page
    page = download_page(url)

    # get all mentions of Abe Vigoda
    hits = page.find_all_mentions("Abe Vigoda")

    if not hits:
        return None

    # parse the first hit for his status
    status = parse_abe_status(hits[0])

    # he's either alive or dead
    return status == "alive"

If the caller receives None from my function, he should assume that there were no mentions of Abe Vigoda, and so something went wrong. But this is pretty vague, right? And it doesn't help the case where hits[0] isn't what we thought it was.

On the other hand, we can put in some exceptions:

Option 2: Using Exceptions

If hits is empty, an IndexError will be thrown when we attempt hits[0]. But the caller shouldn't be expected to handle an IndexError thrown by my function, since he has no idea where that IndexError came from; it could have been thrown by find_all_mentions, for all he knows. So we'll create a custom exception class to handle this:

class NotFoundError(Exception):
    """Throw this when something can't be found on a page."""

def get_abe_status(url):
    # download the page
    page = download_page(url)

    # get all mentions of Abe Vigoda
    hits = page.find_all_mentions("Abe Vigoda")

    try:
        hits[0]
    except IndexError:
        raise NotFoundError("No mentions found.")

    # parse the first hit for his status
    status = parse_abe_status(hits[0])

    # he's either alive or dead
    return status == "alive"

Now what if the page has changed and there are an unexpected number of hits? This isn't catastrophic, as the code may still work, but a caller might want to be extra careful, or he might want to log a warning. So I'll throw a warning:

class NotFoundError(Exception):
    """Throw this when something can't be found on a page."""

def get_abe_status(url):
    # download the page
    page = download_page(url)

    # get all mentions of Abe Vigoda
    hits = page.find_all_mentions("Abe Vigoda")

    try:
        hits[0]
    except IndexError:
        raise NotFoundError("No mentions found.")

    # say we expect four hits...
    if len(hits) != 4:
        raise Warning("An unexpected number of hits.")
        logger.warning("An unexpected number of hits.")

    # parse the first hit for his status
    status = parse_abe_status(hits[0])

    # he's either alive or dead
    return status == "alive"

Lastly, we might find that status isn't either alive or dead. Maybe, for some odd reason, today it turned out to be comatose. Then I don't want to return False, as that implies that Abe is dead. What should I do here? Throw an exception, probably. But what kind? Should I create a custom exception class?

class NotFoundError(Exception):
    """Throw this when something can't be found on a page."""

def get_abe_status(url):
    # download the page
    page = download_page(url)

    # get all mentions of Abe Vigoda
    hits = page.find_all_mentions("Abe Vigoda")

    try:
        hits[0]
    except IndexError:
        raise NotFoundError("No mentions found.")

    # say we expect four hits...
    if len(hits) != 4:
        raise Warning("An unexpected number of hits.")
        logger.warning("An unexpected number of hits.")

    # parse the first hit for his status
    status = parse_abe_status(hits[0])

    if status not in ['alive', 'dead']:
        raise SomeTypeOfError("Status is an unexpected value.")

    # he's either alive or dead
    return status == "alive"

Option 3: Somewhere in between

I think that the second method, with exceptions, is preferable, but I'm not sure if I'm using exceptions correctly within it. I'm curious to see how more experienced programmers would handle this.

share|improve this question
add comment

3 Answers

up vote 5 down vote accepted

The recommendation in Python is to use exceptions to indicate failure. This is true even if you expect failure on a regular basis.

Look at it from the perspective of the caller of your code:

my_status = get_abe_status(my_url)

What if we return None? If the caller doesn't specifically handle the case that get_abe_status failed, it will simply try to continue on with my_stats being None. That may produce a difficult to diagnose bug later on. Even if you do check for None, this code has no clue why get_abe_status() failed.

But what if we raise an exception? If the caller doesn't specifically handle the case, the exception will propagate upward eventually hitting the default exception handler. That may not be the what you want, but its better then introducing a subtle bug elsewhere in the program. Additionally, the exception gives information about what went wrong which is lost in the first version.

From the caller's perspective, its simply more convenient to get an exception than a return value. And that's the python style, to use exceptions to indicate failure conditions not return values.

Some will take a different perspective and argue that you should only use exceptions for cases you never really expect to happen. They argue that normally running running could should not raise any exceptions. One reason that is given for this is that exceptions are grossly inefficient, but that's not actually true for Python.

A couple of points on your code:

try:
    hits[0]
except IndexError:
    raise NotFoundError("No mentions found.")

That's a really confusing way to check for an empty list. Don't induce an exception just to check something. Use an if.

# say we expect four hits...
if len(hits) != 4:
    raise Warning("An unexpected number of hits.")
    logger.warning("An unexpected number of hits.")

You do realize that the logger.warning line will never run right?

share|improve this answer
    
Thanks (belatedly) for your response. It, along with looking at published code, has improved my feeling for when and how to throw an exception. –  jme Jan 17 at 15:13
add comment

You should use exceptions when something exceptional occurs. That is, something that should not occur given proper use of the application. If it is allowable and expected for the consumer of your method to search for something that will not be found, then "not found" is not an exceptional case. In this case, you should return null or "None" or {}, or something indicating an empty return set.

If, on the other hand, your really do expect the consumers of your method to always (unless they screwed up somehow) find what is being searched for, then not finding it would be an exception and you should go with that.

The key is that exception handling can be expensive--exceptions are supposed to gather information about the state of your application when they occur, such as a stack trace, in order to help folks decipher why they occurred. I don't think that's what you are trying to do.

share|improve this answer
1  
If you decide that not finding a value is allowable, be careful about what you use to indicate that was what happened. If your method is supposed to return a String and you choose "None" as your indicator, this means that you have to be careful that "None" would never be a valid value. Also note that there is a difference between looking at the data and not finding a value and not being able to retrieve the data, thus we can't find the data. Having the same result for these two cases means you have no visibility once you are getting no value when you expect there to be one. –  unholysampler Nov 23 '13 at 17:49
    
Inline code blocks are marked with backticks (`), perhaps that's what you meant to do with the "None"? –  Izkata Dec 8 '13 at 4:43
add comment

If I was writing a function

 def abe_is_alive():

I would write it to return True or False in the cases where I am absolutely certain of one or the other, and raise an error in any other case (e.g. raise ValueError("Status neither 'dead' nor 'alive'")). This is because the function calling mine is expecting a boolean, and if I can't provide that with certainly the regular program flow shouldn't continue.

Something like your example of getting a different number of "hits" than expected, I would probably ignore; as long as one of the hits still matches my pattern "Abe Vigoda is {dead|alive}", that's fine. This allows the page to be rearranged but still gets the appropriate information.

Rather than

try:
    hits[0] 
except IndexError:
    raise NotFoundError

I would check explicitly:

if len(hits) == 0:
    raise NotFoundError

as this tends to be "cheaper" then setting up the try.

I agree with you on IOError; I also wouldn't try to error handle connecting to the website - if we can't, for some reason, this isn't the appropriate place to handle it (as it doesn't help us answer our question) and it should pass out to the calling function.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.