unittest.TestResult

class unittest.TestResult

This class is used to compile information about which tests have succeeded and which have failed.

A TestResult object stores the results of a set of tests. The TestCase and TestSuite classes ensure that results are properly recorded; test authors do not need to worry about recording the outcome of tests.

Testing frameworks built on top of unittest may want access to the TestResult object generated by running a set of tests for reporting purposes; a TestResult instance is returned by the TestRunner.run() method for this purpose.

TestResult instances have the following attributes that will be of interest when inspecting the results of running a set of tests:

errors

A list containing 2-tuples of TestCase instances and strings holding formatted tracebacks. Each tuple represents a test which raised an unexpected exception.

failures

A list containing 2-tuples of TestCase instances and strings holding formatted tracebacks. Each tuple represents a test where a failure was explicitly signalled using the TestCase.assert*() methods.

skipped

A list containing 2-tuples of TestCase instances and strings holding the reason for skipping the test.

New in version 3.1.

expectedFailures

A list containing 2-tuples of TestCase instances and strings holding formatted tracebacks. Each tuple represents an expected failure of the test case.

unexpectedSuccesses

A list containing TestCase instances that were marked as expected failures, but succeeded.

shouldStop

Set to True when the execution of tests should stop by stop().

testsRun

The total number of tests run so far.

buffer

If set to true, sys.stdout and sys.stderr will be buffered in between startTest() and stopTest() being called. Collected output will only be echoed onto the real sys.stdout and sys.stderr if the test fails or errors. Any output is also attached to the failure / error message.

New in version 3.2.

failfast

If set to true stop() will be called on the first failure or error, halting the test run.

New in version 3.2.

tb_locals

If set to true then local variables will be shown in tracebacks.

New in version 3.5.

wasSuccessful()

Return True if all tests run so far have passed, otherwise returns False.

Changed in version 3.4: Returns False if there were any unexpectedSuccesses from tests marked with the expectedFailure() decorator.

stop()

This method can be called to signal that the set of tests being run should be aborted by setting the shouldStop attribute to True. TestRunner objects should respect this flag and return without running any additional tests.

For example, this feature is used by the TextTestRunner class to stop the test framework when the user signals an interrupt from the keyboard. Interactive tools which provide TestRunner implementations can use this in a similar manner.

The following methods of the TestResult class are used to maintain the internal data structures, and may be extended in subclasses to support additional reporting requirements. This is particularly useful in building tools which support interactive reporting while tests are being run.

startTest(test)

Called when the test case test is about to be run.

stopTest(test)

Called after the test case test has been executed, regardless of the outcome.

startTestRun()

Called once before any tests are executed.

New in version 3.1.

stopTestRun()

Called once after all tests are executed.

New in version 3.1.

addError(test, err)

Called when the test case test raises an unexpected exception. err is a tuple of the form returned by sys.exc_info(): (type, value, traceback).

The default implementation appends a tuple (test, formatted_err) to the instance’s errors attribute, where formatted_err is a formatted traceback derived from err.

addFailure(test, err)

Called when the test case test signals a failure. err is a tuple of the form returned by sys.exc_info(): (type, value, traceback).

The default implementation appends a tuple (test, formatted_err) to the instance’s failures attribute, where formatted_err is a formatted traceback derived from err.

addSuccess(test)

Called when the test case test succeeds.

The default implementation does nothing.

addSkip(test, reason)

Called when the test case test is skipped. reason is the reason the test gave for skipping.

The default implementation appends a tuple (test, reason) to the instance’s skipped attribute.

addExpectedFailure(test, err)

Called when the test case test fails, but was marked with the expectedFailure() decorator.

The default implementation appends a tuple (test, formatted_err) to the instance’s expectedFailures attribute, where formatted_err is a formatted traceback derived from err.

addUnexpectedSuccess(test)

Called when the test case test was marked with the expectedFailure() decorator, but succeeded.

The default implementation appends the test to the instance’s unexpectedSuccesses attribute.

addSubTest(test, subtest, outcome)

Called when a subtest finishes. test is the test case corresponding to the test method. subtest is a custom TestCase instance describing the subtest.

If outcome is None, the subtest succeeded. Otherwise, it failed with an exception where outcome is a tuple of the form returned by sys.exc_info(): (type, value, traceback).

The default implementation does nothing when the outcome is a success, and records subtest failures as normal failures.

New in version 3.4.