llvm-mirror

mirror of https://github.com/RPCS3/llvm-mirror.git synced 2024-10-21 12:02:58 +02:00

History

Julian Lettner 95339f1d20 [lit] Better/earlier errors for empty runs Fail early, when we discover no tests at all, or filter out all of them. There is also `--allow-empty-runs` to disable test to allow workflows like `LIT_FILTER=abc ninja check-all`. Apparently `check-all` invokes lit multiple times if certain projects are enabled, which would produce unwanted "empty runs". Specify via `LIT_OPTS=--allow-empty-runs`. There are 3 causes for empty runs: 1) No tests discovered. This is always an error. Fix test suite config or command line. 2) All tests filtered out. This is an error by default, but can be suppressed via `--alow-empty-runs`. Should prevent accidentally passing empty runs, but allow the workflow above. 3) The number of shards is greater than the number of tests. Currently, this is never an error. Personally, I think we should consider making this an error by default; if this happens, you are doing something wrong. I added a warning but did not change the behavior, since this warrants more discussion. Reviewed By: atrick, jdenny Differential Revision: https://reviews.llvm.org/D70105		2019-11-12 09:11:36 -08:00
..
examples	Fix lit/example/many-tests pickling issue	2018-09-14 19:44:09 +00:00
lit	[lit] Better/earlier errors for empty runs	2019-11-12 09:11:36 -08:00
tests	[lit] Better/earlier errors for empty runs	2019-11-12 09:11:36 -08:00
utils
CMakeLists.txt
lit.py	[lit] Make main.py a pure Python module	2019-10-28 09:56:37 -07:00
MANIFEST.in	lit: Make sure the builtin_commands directory is packaged by setup.py	2018-09-26 14:56:11 +00:00
README.txt
setup.py	lit: Use a License classifier that pypi will accept	2019-08-08 17:23:33 +00:00

README.txt

===============================
 lit - A Software Testing Tool
===============================

lit is a portable tool for executing LLVM and Clang style test suites,
summarizing their results, and providing indication of failures. lit is designed
to be a lightweight testing tool with as simple a user interface as possible.

=====================
 Contributing to lit
=====================

Please browse the Test Suite > lit category in LLVM's Bugzilla for ideas on
what to work on.

Before submitting patches, run the test suite to ensure nothing has regressed:

    # From within your LLVM source directory.
    utils/lit/lit.py \
        --path /path/to/your/llvm/build/bin \
        utils/lit/tests

Note that lit's tests depend on 'not' and 'FileCheck', LLVM utilities.
You will need to have built LLVM tools in order to run lit's test suite
successfully.

You'll also want to confirm that lit continues to work when testing LLVM.
Follow the instructions in http://llvm.org/docs/TestingGuide.html to run the
regression test suite:

    make check-llvm

And be sure to run the llvm-lit wrapper script as well:

    /path/to/your/llvm/build/bin/llvm-lit utils/lit/tests

Finally, make sure lit works when installed via setuptools:

    python utils/lit/setup.py install
    lit --path /path/to/your/llvm/build/bin utils/lit/tests