Pyjo.Test - Simple writing of test scripts

from Pyjo.Test import *  # noqa

if not have_some_feature:
    plan_skip_all(why)
else:
    plan_tests(number_of_tests_run)

ok(got == expected, test_name)

is_ok(got, expected, test_name)
isnt_ok(got, expected, test_name)

diag("here's what went wrong")

like_ok(got, r'expected', r_flags, test_name)
unlike_ok(got, r'expected', r_flags, test_name)

cmp_ok(got, '==', expected, test_name)

is_deeply_ok(got_complex_structure, expected_complex_structure, test_name)

if not have_some_feature:
    skip(why, how_many)
else:
    ok(foo(), test_name)
    is_ok(foo(42), 23, test_name)

isa_ok(obj, cls)

pass_ok(test_name)
fail_ok(test_name)

done_testing()

This is a module for writing a test scripts which return theirs results as a Test Anything Protocol (TAP) output.

Each test script is a separate Python program which usually is placed in t subdirectory.

prove

The test scripts can be started with prove command:

$ PYTHONPATH=. prove --ext=py --exec=python t

You can also put the arguments in the configuration file .proverc:

--ext=py
--exec=python

prove requires that __init__.py files have to be also runnable scripts. For example:

if __name__ == '__main__':
    from Pyjo.Test import *  # noqa
    pass_ok('__init__')
    done_testing()

nosetests

The test scripts can be started with nosetests command:

$ PYTHONPATH=`pwd` nosetests --where=t --match=. --attr=test_nose

You can also put the arguments in the configuration file setup.cfg:

[nosetests]
where=t
match=.
attr=test_nose

nosetests command requires additional boilerplate code in test script:

import Pyjo.Test

class NoseTest(Pyjo.Test.NoseTest):
    script = __file__
    srcdir = '../..'

if __name__ == '__main__':
    from Pyjo.Test import *  # noqa
    ok('main test script')
    ...

unittest

The test scripts can be started with python -m unittest command:

$ PYTHONPATH=. python -m unittest discover -s t -p '*.py'

unittest module does not discover test script in subdirectories so it have to be used for each subdirectory separately:

$ PYTHONPATH=. python -m unittest discover -s t/subdir1 -p '*.py'
$ PYTHONPATH=. python -m unittest discover -s t/subdir2 -p '*.py'

unittest module requires additional boilerplate code in test script:

import Pyjo.Test

class UnitTest(Pyjo.Test.UnitTest):
    script = __file__

if __name__ == '__main__':
    from Pyjo.Test import *  # noqa
    ok('main test script')
    ...

setuptools

The test scripts can be started with python setup.py test command.

$ python setup.py test

Additional helper script test.py is required in the source directory of the package:

#!/usr/bin/env python

import unittest

dirs = ['t']

class TestSuite(unittest.TestSuite):
    def __init__(self, *args, **kwargs):
        super(TestSuite, self).__init__(*args, **kwargs)
        test_loader = unittest.defaultTestLoader
        for d in dirs:
            test_suite = test_loader.discover(d, pattern='*.py', top_level_dir='.')
            for t in test_suite:
                self.addTest(t)

    def __iter__(self):
        return iter(self._tests)

if __name__ == '__main__':
    unittest.main(defaultTest='TestSuite')

The setup.py script should set additional options:

setup(
    packages=find_packages(exclude=['t', 't.*']),
    test_suite='test.TestSuite',
    ...
)

Classes and functions

class Pyjo.Test.Error

Exception raised if test script could not be started.

class Pyjo.Test.NoseTest

Pyjo.Test wrapper for nosetests command.

class Pyjo.Test.UnitTest(methodName='runTest')

Pyjo.Test wrapper for python -m unittest command.

Pyjo.Test.cmp_ok(got, operator, expected, test_name=None)
# ok(got == expected)
cmp_ok(got, '==', expected, 'this == that')

Compare two arguments using binary Python operator (==, >=, >, <=, < or !=). The test passes if the comparison is true and fails otherwise.

Pyjo.Test.done_testing(how_many=None)
plan(1)
ok(True)
done_testing(1)

Issue the plan when it’s done running tests. It fails when plan doesn’t match this one in plan(). The plan is optional and any number of tests is accepted if plan is None.

Pyjo.Test.diag(*args)
if not in_ok(users, 'foo', "There's a foo user"):
    diag("Since there's no foo, check that /etc/bar is set up right")

Print a diagnostic message which is guaranteed not to interfere with test output.

Returns false, so as to preserve failure.

Pyjo.Test.fail_ok(test_name=None)
fail_ok("This should not happen")

The synonym for ok(False).

Pyjo.Test.in_ok(got, elem, test_name=None)
in_ok(['foo', 'bar'], 'foo', "foo is in list")

Check if element exists in given object. The same as ok(elem in got), but with more meaningful diagnostic message.

Pyjo.Test.is_ok(got, expected, test_name=None)
is_ok(ultimate_answer(), 42, "Meaning of Life")

Compare two arguments with ‘==’ operator.

Pyjo.Test.isa_ok(got, cls, test_name=None)
obj = SomeClass()
isa_ok(obj, SomeClass)
isa_ok(123, (int, float,))

Check if object is an instance of class or one of the classes.

Pyjo.Test.is_deeply_ok(got, expected, test_name=None)
is_deeply_ok([[1,2], [3, [4,5]]], [[1,2], [3]], "deep structure")

Do a deep comparison walking each data structure to see if they are equivalent. If the two structures are different, it will display the place where they start differing.

Pyjo.Test.isnt_ok(got, expected, test_name=None)
isnt(foo, '', "Got some foo")

Compare two arguments with ‘!=’ operator.

Pyjo.Test.like_ok(got, expected, test_name=None)
like_ok(got, 'expected', test_name)
like_ok(got, ('expected', 'i'), test_name)
like_ok(got, re.compile('expected'), test_name)

Check if value matches against the regex. The regex can be already compiled as a :mod:re object.

The regex can compiled with additional flags: d, i, m, s, u and x as a second element of list or tuple.

Pyjo.Test.none_ok(got, test_name=None)
none_ok(req.headers.transfer_encoding, "no Transfer-Encoding value")

Check if value is None.

Pyjo.Test.not_in_ok(got, elem, test_name=None)
import os
not_in_ok(os.environ, 'DISPLAY', "DISPLAY is not set")

The same as in_ok(), only it checks if element does not exists in given object.

Pyjo.Test.ok(check, test_name=None)
ok(exp(9) == 81, "simple exponential")
ok(isinstance('db_Main', Film), "set_db()")
ok(p.tests == 4, "saw tests")
ok(None not in items, "all items defined")

This simply evaluates any expression and uses that to determine if the test succeeded or failed. A true expression passes, a false one fails.

Pyjo.Test.pass_ok(test_name=None)
pass_ok("Step 1")

The synonym for ok(True).

Pyjo.Test.plan_tests(how_many)
hosts = ['pyjoyment.net', 'pypi.python.org']
plan_tests(len(hosts))
for host in hosts:
    ok(check_if_available(host))

Declare how many tests script is going to run to protect against premature failure.

The plan can be also issued when it’s done running tests. Then done_testing() function should be used instead.

Pyjo.Test.plan_skip_all(why)
import sys
if sys.version_info < (3, 0):
    plan_skip_all("Python 3.x is required")

Completely skip an entire testing script.

Pyjo.Test.skip(why=None, how_many=1)
plan_tests(1)
if sys.version_info < (3, 0):
    isa_ok(u'string', unicode, "string is unicode")
else:
    skip("Requires Python 2.x", 1)

Declare how many tests might be skipped so tests run will match up with the plan.

Pyjo.Test.throws_ok(cb, expected, test_name=None)
elements = [1, 2, 3]
throws_ok(lambda: elements[3], IndexError, 'no more elements')

Check if callback (function or lambda) throws a proper exception.

Pyjo.Test.unlike_ok(got, expected, test_name=None)
unlike_ok(got, 'expected', test_name)
unlike_ok(got, ('expected', 'i'), test_name)
unlike_ok(got, re.compile('expected'), test_name)

The same as like_ok(), only it checks if value does not match the given pattern.