| Allegro CL version 8.2 Unrevised from 8.1 to 8.2. 8.1 version |
This document contains the following sections:
1.0 The tester module APIANSI Common Lisp contains no functionality designed specifically for testing applications. Because testing is an essential part of application development, Franz Inc. is making public the test harness used internally for testing Allegro CL itself. (A test harness is a collection of macros and variables associated with testing, along with templates for test forms.)
To use the test harness, you must load the tester.fasl module. Do this by evaluating
(require :tester)
All of the following symbols are exported from the
util.test
package.
The test harness API includes the following variables, each described fully on its own page and briefly here.
*break-on-test-failures*
:
If true, break is called when
a test fails.
*error-protect-tests*
: If true, errors
(other than in a test-error form) will be considered a failure and
testing continues.
*test-errors*
:
The value is the number of test errors which have occurred.
*test-successes*
:
The value is the number of test successes which have occurred.
*test-unexpected-failures*
:
The value is the number of unexpected test failures which have occurred.
These macros wrap around a form to be tested and supply the expected value (for the test macro) or the expected behavior, which is encoded in the macro name (e.g. test-error). For example:
(test 1 (+ 0 1)) ;; (testing that the result of (+ 0 1) is ;; the fixnum 1) (test-error (+ 1 "2")) ;; (testing that an error is ;; signaled when a string is ;; passed as an argument to +)
Many more examples are given below.
with-tests wraps dynamically around a collection of test or test-* forms.
Note that many of the macros have fail-info and known-failure keyword arguments.
Each macro is described briefly here and fully on its documentation page.
Arguments: expected-value test-form &key (test (function eql)) multiple-values fail-info known-failure
Perform a single test and compare expected-value with the value actually returned by test-form.
Arguments: form &key announce catch-breaks fail-info known-failure (condition-type (quote simple-error)) include-subtypes format-control format-arguments
Perform a single test to see whether form signals an error.
Arguments: form &key announce catch-breaks fail-info known-failure
Perform a single test to see that form does not signal an error.
Arguments: form &key fail-info known-failure
Perform a single test to see that form signals a warning.
Arguments: form &key fail-info known-failure
Perform a single test to see that form does not signal a warning.
Arguments: (&key (name "unnamed")) &body body
Evaluates body, which should be a list of test forms, and reports on the results.
The following are simple examples using the test harness. The test forms themselves are trivial, and the purpose is to indicate the behavior of the test harness macros.
user(1): (require :tester) ; Fasl loading .../tester.fasl t user(2): (use-package :util.test) t user(3): (test 1 1) t user(4): (test 1 2) * * * UNEXPECTED TEST FAILURE * * * Test failed: 2 wanted: 1 got: 2 nil user(5): (defun foo (x) x) foo user(6): (test 1 (foo 1)) t user(7): (test 1 (foo 2)) * * * UNEXPECTED TEST FAILURE * * * Test failed: (foo 2) wanted: 1 got: 2 nil user(8): (setq *break-on-test-failures* t) t user(9): (test 1 (foo 2)) * * * UNEXPECTED TEST FAILURE * * * Test failed: (foo 2) wanted: 1 got: 2 Break: *break-on-test-failures* is non-nil. Restart actions (select using :continue): 0: return from break. 1: Return to Top Level (an "abort" restart) 2: Abort #<process Initial Lisp Listener> [1c] user(10): :pop user(11): (setq *break-on-test-failures* nil) nil user(12): (test 1 (error "foo")) Error: foo Restart actions (select using :continue): 0: Return to Top Level (an "abort" restart) 1: Abort #<process Initial Lisp Listener> [1] user(13): :pop user(14): (setq *error-protect-tests* t) t user(15): (test 1 (error "foo")) Condition type: simple-error Message: foo * * * UNEXPECTED TEST FAILURE * * * Test failed: (error "foo") Reason: an error (of type `simple-error') was detected. nil user(16): (setq *error-protect-tests* nil) nil user(17): *test-errors* 4 user(18): *test-successes* 2 user(19): (test 1 2 :known-failure t) Test failed: known failure: 2 wanted: 1 got: 2 nil user(20): (test 1 (foo 1) :known-failure t) Expected test failure for (foo 1) did not occur. nil user(21): (test 1 (foo 1) :known-failure t :fail-info "This is bug666.") Expected test failure for (foo 1) did not occur. Additional info: This is bug666. nil user(22): (test-error (error "foo")) t user(23): (test-no-error (error "foo")) * * * UNEXPECTED TEST FAILURE * * * Test failed: (error "foo") Reason: detected an unexpected error of type `simple-error'. nil user(24): (test-error (car '(10))) * * * UNEXPECTED TEST FAILURE * * * Test failed: (car '(10)) Reason: expected but did not detect an error of type `condition'. nil user(25): (test-no-error (car '(10))) t user(26): (test-warning (warn "foo")) t user(27): (test-no-warning (warn "foo")) * * * UNEXPECTED TEST FAILURE * * * Test failed: (warn "foo") wanted: no warning got: a warning nil user(28): (test-warning (car '(10))) * * * UNEXPECTED TEST FAILURE * * * Test failed: (car '(10)) wanted: a warning got: no warning nil user(29): (test-no-warning (car '(10))) t user(30): (test-error (error "foo: ~a" 10)) t user(31): (test-error (error "foo: ~a" 10) :format-control "foo: ~a") t user(32): (test-error (error "foo: ~a" 10) :format-control "foo: ~a" :format-arguments '(10)) t user(33): (test-error (error "foo: ~a" 10) :format-control "foo: ~a") * * * UNEXPECTED TEST FAILURE * * * Test failed: (error "foo: ~a" 10) Reason: the format-control was incorrect. wanted: "~1@<foo: ~a~:@>" got: "~1@<foo: ~a~:@>" nil user(34): (test-error (error "foo: ~a" 10) :format-control "foo: ~a" :format-arguments '(11)) * * * UNEXPECTED TEST FAILURE * * * Test failed: (error "foo: ~a" 10) Reason: the format-arguments were incorrect. wanted: (10) got: (11) nil user(35): (test-error (error "foo: ~a" 10) :condition-type 'condition :include-subtypes t) t user(36): (test-error (error "foo: ~a" 10) :condition-type 'simple-break :include-subtypes t) * * * UNEXPECTED TEST FAILURE * * * Test failed: (error "foo: ~a" 10) Reason: detected an incorrect condition type. wanted: simple-break got: #<standard-class simple-error> nil user(37): (test-error (break "foo: ~a" 10) :condition-type 'simple-break :include-subtypes t) Break: foo: 10 [condition type: simple-break] Restart actions (select using :continue): 0: return from break. 1: Return to Top Level (an "abort" restart) 2: Abort #<process Initial Lisp Listener> [1c] user(38): :pop user(39): (test-error (break "foo: ~a" 10) :catch-breaks t :condition-type 'simple-break :include-subtypes t) t
Copyright (c) 1998-2016, Franz Inc. Oakland, CA., USA. All rights reserved.
This page was not revised from the 8.1 page.
Created 2010.1.21.
| Allegro CL version 8.2 Unrevised from 8.1 to 8.2. 8.1 version |