xref: /illumos-gate/usr/src/test/README (revision 7b1753e6)
1*7b1753e6SJohn Wren Kennedy#
2*7b1753e6SJohn Wren Kennedy# This file and its contents are supplied under the terms of the
3*7b1753e6SJohn Wren Kennedy# Common Development and Distribution License ("CDDL"), version 1.0.
4*7b1753e6SJohn Wren Kennedy# You may only use this file in accordance with the terms of version
5*7b1753e6SJohn Wren Kennedy# 1.0 of the CDDL.
6*7b1753e6SJohn Wren Kennedy#
7*7b1753e6SJohn Wren Kennedy# A full copy of the text of the CDDL should have accompanied this
8*7b1753e6SJohn Wren Kennedy# source.  A copy of the CDDL is also available via the Internet at
9*7b1753e6SJohn Wren Kennedy# http://www.illumos.org/license/CDDL.
10*7b1753e6SJohn Wren Kennedy#
11*7b1753e6SJohn Wren Kennedy
12*7b1753e6SJohn Wren Kennedy#
13*7b1753e6SJohn Wren Kennedy# Copyright (c) 2013 by Delphix. All rights reserved.
14*7b1753e6SJohn Wren Kennedy#
15*7b1753e6SJohn Wren Kennedy
16*7b1753e6SJohn Wren Kennedyillumos Testing README
17*7b1753e6SJohn Wren Kennedy
18*7b1753e6SJohn Wren Kennedy1. A Brief History of usr/src/test
19*7b1753e6SJohn Wren Kennedy2. How to Run These Tests
20*7b1753e6SJohn Wren Kennedy3. How to Develop New Tests
21*7b1753e6SJohn Wren Kennedy4. Porting Tests from Other Frameworks
22*7b1753e6SJohn Wren Kennedy
23*7b1753e6SJohn Wren Kennedy--------------------------------------------------------------------------------
24*7b1753e6SJohn Wren Kennedy
25*7b1753e6SJohn Wren Kennedy1. A Brief History of usr/src/test
26*7b1753e6SJohn Wren Kennedy
27*7b1753e6SJohn Wren KennedyThe tests here come in two varieties - tests written from scratch, and tests
28*7b1753e6SJohn Wren Kennedythat have been ported from the Solaris Test Collection. Not all of the STC
29*7b1753e6SJohn Wren Kennedytests have been ported, and a forked repository of those that have been made
30*7b1753e6SJohn Wren Kennedypublicly available may currently be found here:
31*7b1753e6SJohn Wren Kennedy
32*7b1753e6SJohn Wren Kennedy	https://bitbucket.org/illumos/illumos-stc/
33*7b1753e6SJohn Wren Kennedy
34*7b1753e6SJohn Wren KennedyRegardless of origin, all of these tests are executed using the run(1) script
35*7b1753e6SJohn Wren Kennedydescribed in the next section.
36*7b1753e6SJohn Wren Kennedy
37*7b1753e6SJohn Wren Kennedy2. How to Run These Tests
38*7b1753e6SJohn Wren Kennedy
39*7b1753e6SJohn Wren KennedyCurrently, all the test suites under usr/src/test provide a wrapper script
40*7b1753e6SJohn Wren Kennedyaround run(1). These wrappers allow environment variables to be set up which
41*7b1753e6SJohn Wren Kennedymay (for example) allow the script to specify which disks may be used by a test
42*7b1753e6SJohn Wren Kennedysuite, and which must be preserved. Additionally, the wrappers allow options to
43*7b1753e6SJohn Wren Kennedyrun(1) to be passed through so that a user may specify a custom configuration
44*7b1753e6SJohn Wren Kennedyfile for a test suite. For specifics on the options available in the framework
45*7b1753e6SJohn Wren Kennedyitself, please see the run(1) manpage.
46*7b1753e6SJohn Wren Kennedy
47*7b1753e6SJohn Wren Kennedy3. How to Develop New Tests
48*7b1753e6SJohn Wren Kennedy
49*7b1753e6SJohn Wren KennedyNew tests should mimic the directory layout of existing tests to the degree
50*7b1753e6SJohn Wren Kennedypossible. This includes the following directories:
51*7b1753e6SJohn Wren Kennedy
52*7b1753e6SJohn Wren Kennedycmd - Any support binaries or scripts used by the tests in this package.
53*7b1753e6SJohn Wren Kennedydoc - READMEs or other support documentation to be delivered with the package.
54*7b1753e6SJohn Wren Kennedyrunfiles -  Configuration files that dictate how the tests are run.
55*7b1753e6SJohn Wren Kennedytests - The tests themselves (see below).
56*7b1753e6SJohn Wren Kennedy
57*7b1753e6SJohn Wren KennedyThe tests you create will be run, and given a PASS or FAIL status in accordance
58*7b1753e6SJohn Wren Kennedywith the exit value returned by the test. A test may also be marked SKIPPED in
59*7b1753e6SJohn Wren Kennedythe event that a prerequisite test is marked FAIL, or marked KILLED in the
60*7b1753e6SJohn Wren Kennedyevent the test times out. Note that there is no way to force a test to be
61*7b1753e6SJohn Wren Kennedymarked SKIPPED; this is intentional. If a test must be skipped due to
62*7b1753e6SJohn Wren Kennedyinsufficient resources for example, then a wrapper script should be provided
63*7b1753e6SJohn Wren Kennedythat chooses or creates an appropriate configuration file. The goal of every
64*7b1753e6SJohn Wren Kennedyrun is that every test is marked PASS.
65*7b1753e6SJohn Wren Kennedy
66*7b1753e6SJohn Wren Kennedy4. Porting Tests from Other Frameworks
67*7b1753e6SJohn Wren Kennedy
68*7b1753e6SJohn Wren KennedySTF (Solaris Test Framework)
69*7b1753e6SJohn Wren Kennedy
70*7b1753e6SJohn Wren KennedyPorting tests from this framework is relatively straightforward. For the most
71*7b1753e6SJohn Wren Kennedypart the tests can be arranged in the new directory structure and added to the
72*7b1753e6SJohn Wren Kennedyconfiguration file. The template for a configuration file can easily be created
73*7b1753e6SJohn Wren Kennedyusing the -w option to run(1). There are a few other changes that may be
74*7b1753e6SJohn Wren Kennedyrequired:
75*7b1753e6SJohn Wren Kennedy
76*7b1753e6SJohn Wren KennedySome STF tests consume the values of variables from the user's environment.
77*7b1753e6SJohn Wren KennedyThese variables must be set before beginning the test run either manually, or
78*7b1753e6SJohn Wren Kennedyvia a wrapper script.
79*7b1753e6SJohn Wren Kennedy
80*7b1753e6SJohn Wren KennedyGroups of tests in STF can automatically source .cfg files specified in the
81*7b1753e6SJohn Wren Kennedy'stf_description' file that exists in an STF directory. Ported tests that
82*7b1753e6SJohn Wren Kennedyrequire any of these variables must manually source the .cfg file instead.
83*7b1753e6SJohn Wren Kennedy
84*7b1753e6SJohn Wren KennedyThe configuration file of a newly ported test must specify the user the test
85*7b1753e6SJohn Wren Kennedyshould run as, along with the timeout value. In STF, these are also specified
86*7b1753e6SJohn Wren Kennedyin the 'stf_description' file.
87