Python: testing beyond exceptions

Recently working on some code I was doing some basic tests that checked for an exception being raised.

The tests looked like this:

def test_user_identification_is_valid_option(self):
   with self.assertRaises(ValueError):
       example_seller(products=[
           'id': 'hai',
           'description': 'a description',
           'recurrence': None,
           'user_identification': True,
       }])


def test_user_identification_is_required(self):
     with self.assertRaises(ValueError):
         example_seller(products=[{
             'id': 'hai',
             'description': 'a description',
             'recurrence': None,
         }])

I realised that it was really easy to be tricked by other parts of the code that might raise the same exception. For example, in this case the required code is raising ValueError separately.

In JavaScript using assertions in chai, assert.throws takes a function that should throw an exception, the 2nd arg is the exception instance you're expecting and the third arg can be a regex to match the string passed to the exception.

I wanted to do the same thing in Python.

assertRaisesRegexp

In Python (2.7+) assertRaisesRegexp does exactly what we want so we can fix our tests to be more explicit like so:

def test_user_identification_is_valid_option(self):
    with self.assertRaisesRegexp(ValueError, 'user_identification must'):
       example_seller(products=[
           'id': 'hai',
           'description': 'a description',
           'recurrence': None,
           'user_identification': True,
       }])


def test_user_identification_is_required(self):
     with self.assertRaisesRegexp(ValueError, 'Missing user_ident'):
         example_seller(products=[{
             'id': 'hai',
             'description': 'a description',
             'recurrence': None,
         }])

The second argument can either be a a regular expression object or a string containing a regular expression suitable for use by re.search().

With this in place we can be sure our tests are asserting on the actual part of the code that raises the exception not just asserting that any old ValueError is good enough.