NAME - tester workflow

DESCRIPTION

The tester manual overview refers to all other sections:

• tester Tester overview

NOTES

TEST WORK FLOW

Test work flow: tests execute in the following order:

define tests

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Do until all tests are completed or "end" Run multiple tests in order of listing in file(s) names name array for $<name> substitution envall environment commands to every run command tests create a tests description file - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

start all tests

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - contains $FILENAME, $TESTID, $TESTSCMD, $TESTSDESC values beforeall run before all tests - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

define a default group

- - - - - - - - - - - - - - - - - - - - - - - - - - - - Do until all test groups are completed - - - - - - - - - - - - - - - - - - - - - - - - - - - - optionally define a set of defaults for a Test Group default reset any default if specified (optional) beforeall set up commands for a test group class class of test (additive) env environment commands for a test before set up commands for a test prune commands to prune test results expect expected output by result tag save saved output by result tag exit expected exit code value diff diff command for comparing results after tear down commands for a test afterall tear down commands for a test group - - - - - - - - - - - - - - - - - - - - - - - - - - - -

start a default group

- - - - - - - - - - - - - - - - - - - - - - - - - - - - contains $DEFDESC value; substitute names at test time beforeall run set up commands for this Test Group Do until all tests are completed for this Test Group - - - - - - - - - - - - - - - - - - - - - - - - - - -

define a test

- - - - - - - - - - - - - - - - - - - - - - - - - - - test create a test description file class optional class of test (add to default) optionally overwrite a default for a specific Test env environment commands for a test before set up commands for a test expect expected output by result tag save saved output by result tag exit expected exit code value diff diff command for comparing results after tear down commands for a test apply names substitution for each test - - - - - - - - - - - - - - - - - - - - - - - - -

start a test

- - - - - - - - - - - - - - - - - - - - - - - - - Do this test contains $TESTCLASS, $TESTDESC, $TESTNUM, $TESTNAME values before run before this test (set up) envall execute global environment commands env execute environment commands run execute this test prune execute commands to prune results expect capture expected output by result tag save capture saved output by result tag exit capture expected exit value diff check expected result by result tag or generate result files after run after this test - - - - - - - - - - - - - - - - - - - - - - - - - - -

end a test

- - - - - - - - - - - - - - - - - - - - - - - - - - - Results tabulated on this test; append or create - - - - - - - - - - - - - - - - - - - - - - - - - - -

end a default group

- - - - - - - - - - - - - - - - - - - - - - - - - - - - afterall run once after default group of tests - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

end all tests

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - afterall run after all tests - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

.

runtime

Some items only resolve themselves at run time. The $TESTDIR value is resolved by any shell at run time. It is filled in with the current working directory where the tester was invoked.

Similarly, $FILESPEC will fill in $TESTDIR for the current $FILENAME .

Also, $TESTSPEC will fill in $TESTDIR for the current $TESTNAME .

The first test run will create a database of test descriptions and each test is incremented sequentially. All new tests will be added to the database sequentially. This allows tests to be run and then changed with little side affects.

Note that if a test description changes, it is a new test to the database. So some tests may over time be orphaned.