DESCRIPTION

The tester manual overview refers to all other sections:

• tester Tester overview

NOTES

Options used by the tester

The tester need only enter enough letters of any option so that it is unique. Most options are unique with the first letter of the alphabet. For example, -a or -al or -all.

The following options are described in alphabetical order. It should be mentioned that the tester is designed to run multiple tests from multiple test specifications from a single command line.

Any one test specification is limited to 999 individual tests. Since tests generate tons of files, the resource limits are often exhausted long before this test limit.

The true power of testing is to be able to express a test of specific programs without having to write hundreds of small script programs which test, exercise and benchmark a program. Many of the options are specifically written to allow the selection, interaction and benchmarking of tests to occur independently of the test.

The second great feat of a test framework is to generate program output without having to manually create it. This coupled with the automation of checked output takes many hours of tedium away from the tester / programmer.

The test language can be used to create the utility commands which run and reproduce test results. The options reflect the day-to-day usage of such tests.

-all

The -all option will preserve blank lines that occur in a test specifications. The default is to ignore all blank lines.

-bench

The -bench option requests that the tester report on how long it takes for the requested tests to execute. This can be used to create a benchmark . This combined with a clever class name like ``bench'' can automate the collection of critical bench statistics.

-class list

The -class option allows the tester to request only specific tests to be run. The hints sections shows how this allows the test development to be managed by user defined names.

Class names can be used to create utility tests as shown by the t.form test in the tester release files.

Class names may be quoted to specify more than one class or place a plus symbol between each class name. For example, to run both io and error tests specify the following:

tester -class io+error -execute t.mytest

The alternate form is with quotes:

tester -class ``io error'' -execute t.mytest

See the option -unclass to exclude a class of tests.

-delete

The -delete option will delete the summary of results for each individual test run. Each test maintains results in `` $TESTSPEC .result''.

As tests are added to the test specification the name of the result file changes according to the physical ordering of the tests. They are numbered sequentially from 001 to 999.

It does not delete the main summary of results found in `` $FILESPEC .result''.

-execute

The -execute option requests the tester to execute the commands that it builds. By default the tester writes bourne shell commands as standard output. The -execute option overrides this default to execute tests immediately.

This facilitates a way to create parallel tests since tests are executed as soon as they are encountered.

tester -execute t.mytest

The -x option is does the same but is shorter to enter.

It can also execute as a csh test or as a process. See the -process and -shell options for more information.

tester -execute b<-shell> csh t.mytest

-flow

Normally this options is not used. When selected it askes the tester to display information about the current test generation based on the TEST WORK FLOW .

All flow information is written to standard error.

-generate

The -generate option will generate result files whether they exist or not. Results files are generally named as `` $TESTSPEC .R_'' where the tags are typically SE, SO, SESO and so forth. The tags represent standard error, standard output, both or any specified user tags.

The test specification uses a test language to refer to these files. stderr for standard error output. stdout for standard output. both for a stream of both standard error output and standard output.

In Unix the standard error output is used for error messages back to a terminal normally. Standard output is often redirected to a file by an application. When mixed they often mix at differently from run to run of the same test. So often the only way to get a consistent test is to expect both separately.

-help

Return a complete help screen.

-html

Generate html documentation using pod2html2 and the tester pod files. These files must exist in the same subdirectory.

``pod2html'' was written to allow linked files across perl pod files. ``pod2html2'' ;builds links between all pod files if requested by the ``-all'' option. Check out ``pod2html2 -help'' to see all of its features.

-interactive

This -interactive option requests an interactive prompt for each test before executing it. Any input other than ``y'' or ``Y'' will skip that test.

This options still applies the rules for -class and -unclass .

-keep

The -keep option will keep files around that are normally deleted by the tester execution of tests. By default just the failed output is kept for test analysis. Other files include all other files that are expected or saved with the expect and save keywords.

It may be necessary to delete files after using this option.

-lines

Normally this option is not used. It shows each line generated for a test specification. By default this is already being written to standard out. In certain cases, it is useful to write this to standard error as it builds and / or executes a test.

All lines are written to standard error.

-man

Create a man page for tester to standard out.

-nocleanup

The -nocleanup option will keep files around by not executing any after or afterall logic that is found in the execution of a test. These phases of a test are considered clean up.

-other [@]file

The -other option specifies a name file which will override any local names specified in the actual test specification.

Use of this option provides test independence for resources.

The -other <@file> option specifies a name file which will add to any local names specified in the actual test specification. The local names will override any global names from that file.

Use of this option provides outside names for global resources.

-process cmd

The -process option can specify an alternate process to read run command input. By default the tester will read from any files specified on the command line. If no files are specified then it will read from standard input.

-queuejobs

The -queuejobs option will queue commands and execute them as a single execute stream to the shell or process. This can ensure that environment variables stay in place through the entire testing session.

Some testing requires a strict environment that is easier done outside of all tests. Use of the envall command with the -queuejobs options enables this environment to be global.

To create parallel tests the tester will often use this serial test stream with other parallel tests, -execute .

-restart

The -restart option will force a complete restart of the test process for each test. By default tests are run as one complete stream of commands.

-shell shell

The option -shell specifies which shell to use for all tests. By default the bourne shell will run all tests. Since commands vary drastically from shell to shell, it only makes sense to use either a bourne shell or a cshell commands. Fortunately, cshell commands are compatible with korn and tcsh shells.

Tests can be written to survive host reboots given the source of a test which will put the right logic into an /etc/rc file in unix.

-tag id

The -tag option uniquely identifies a particular run of tests which is useful when on set of tests are used for multiple passes when testing various environments, resource loads and so forth.

The name that results for the test is: `` $FILENAME $TESTID $TESTID '' which within the test is logically named `` $TESTNAME ''.

-unclass list

The -unclass option allows the tester to reject only specific tests to be run. The hints sections shows how this allows the test development to be managed by user defined names.

Class names can be used to create special tests like known failures. The test group can then decide to check on bug fixes (known failures) after the fix has been applied.

Class names may be quoted to specify more than one class or place a plus symbol between each class name. For example, to skip both known failures and utility tests specify the following:

tester -unclass kf+utility -execute t.mytest

The alternate form is with quotes:

tester -unclass ``kf utility'' -execute t.mytest

See the option -class to include a class of tests.

-verbose

The -verbose option will describe the process of each particular test in a exhaustive fashion. All options and phases of each test will be shown.

Some tests require certain options to be set for a successful test. This options allows this to be monitored along with the testing.

Use this option to help debug tests.

-without

The -without option produces a simple sequential ordering for all generated tests. By default the tester will use a database of test descriptions so that future tests can be added in any order. See Test Ordering for more information.

-x

The -x option is the same as the -execute option. By default the tester will only create the shell script required to run a set of tests. The -x options requests that those tests be run by the tester.

-zipnames

The -zipnames option will substitutes names in a special fashion. By default names are only substituted on word boundaries. So $TESTNAME .pie will result in the substitution of $TESTNAME properly.

If one has names within names like $TESTNAMEpie then the tester by default will not substitute $TESTNAME for $TESTNAMEpie. The -zipnames options requests that any name be substituted anywhere. $TESTNAMEpie will substitute $TESTNAME .

.