RPCS3/llvm-mirror
1
0
Fork 0
mirror of https://github.com/RPCS3/llvm-mirror.git synced 2024-11-23 03:02:36 +01:00
Code Issues Projects Releases Wiki Activity

docs/lit: Add some notes on the lit test run output format.

Browse Source 
llvm-svn: 153450
This commit is contained in:
Daniel Dunbar 2012-03-26 18:01:14 +00:00
parent 605c594f4f
commit 45355bcdaa
1 changed files with 47 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

47
docs/CommandGuide/lit.pod
View File

@ -341,6 +341,53 @@ define subdirectories of optional tests, or to change other configuration
parameters -- for example, to change the test format, or the suffixes which
identify test files.
=head2 TEST RUN OUTPUT FORMAT
The b<lit> output for a test run conforms to the following schema, in both short
and verbose modes (although in short mode no PASS lines will be shown). This
schema has been chosen to be relatively easy to reliably parse by a machine (for
example in buildbot log scraping), and for other tools to generate.
Each test result is expected to appear on a line that matches:
<result code>: <test name> (<progress info>)
where <result-code> is a standard test result such as PASS, FAIL, XFAIL, XPASS,
UNRESOLVED, or UNSUPPORTED. The performance result codes of IMPROVED and
REGRESSED are also allowed.
The <test name> field can consist of an arbitrary string containing no newline.
The <progress info> field can be used to report progress information such as
(1/300) or can be empty, but even when empty the parentheses are required.
Each test result may include additional (multiline) log information in the
following format.
<log delineator> TEST '(<test name>)' <trailing delineator>
... log message ...
<log delineator>
where <test name> should be the name of a preceeding reported test, <log
delineator> is a string of '*' characters I<at least> four characters long (the
recommended length is 20), and <trailing delineator> is an arbitrary (unparsed)
string.
The following is an example of a test run output which consists of four tests A,
B, C, and D, and a log message for the failing test C.
=head3 Example Test Run Output Listing
PASS: A (1 of 4)
PASS: B (2 of 4)
FAIL: C (3 of 4)
******************** TEST 'C' FAILED ********************
Test 'C' failed as a result of exit code 1.
********************
PASS: D (4 of 4)
=back
=head2 LIT EXAMPLE TESTS
The B<lit> distribution contains several example implementations of test suites