llvm-mirror

mirror of https://github.com/RPCS3/llvm-mirror.git synced 2024-11-23 11:13:28 +01:00

History

Dan Liew a50a440b7c [lit] Try to remove the flakeyness of `shtest-timeout.py` and `googletest-timeout.py`. The tests previously relied on the `short.py` and `FirstTest.subTestA` script being executed on a machine within a short time window (1 or 2 seconds). While this "seems to work" it can fail on resource constrained machines. We could bump the timeout a little bit (bumping it too much would mean the test would take a long time to execute) but it wouldn't really solve the problem of the test being prone to failures. This patch tries to remove this flakeyness by separating testing into two separate parts: 1. Testing if a test can hit a timeout. 2. Testing if a test can run to completion in the presence of a timeout. This way we can give (1.) a really short timeout (to make the test run as fast as possible) and (2.) a really long timeout. This means for (2.) we are no longer trying to rely on the "short" test executing within some short time window. Instead the window is now 3600 seconds which should be long enough even for a heavily resource constrained machine to execute the "short" test. Thanks to Julian Lettner for suggesting this approach. This superseeds my original approach in https://reviews.llvm.org/D88807. This patch also changes the command line override test to run the quick test rather than the slow one to make the test run faster. Differential Revision: https://reviews.llvm.org/D89020		2020-10-08 10:46:18 -07:00
..
examples
lit	Revert [lit] Support running tests on Windows without GnuWin32	2020-10-06 15:38:18 -04:00
tests	[lit] Try to remove the flakeyness of `shtest-timeout.py` and `googletest-timeout.py`.	2020-10-08 10:46:18 -07:00
utils
CMakeLists.txt
lit.py
MANIFEST.in
README.txt
setup.py

README.txt

===============================
 lit - A Software Testing Tool
===============================

lit is a portable tool for executing LLVM and Clang style test suites,
summarizing their results, and providing indication of failures. lit is designed
to be a lightweight testing tool with as simple a user interface as possible.

=====================
 Contributing to lit
=====================

Please browse the Test Suite > lit category in LLVM's Bugzilla for ideas on
what to work on.

Before submitting patches, run the test suite to ensure nothing has regressed:

    # From within your LLVM source directory.
    utils/lit/lit.py \
        --path /path/to/your/llvm/build/bin \
        utils/lit/tests

Note that lit's tests depend on 'not' and 'FileCheck', LLVM utilities.
You will need to have built LLVM tools in order to run lit's test suite
successfully.

You'll also want to confirm that lit continues to work when testing LLVM.
Follow the instructions in http://llvm.org/docs/TestingGuide.html to run the
regression test suite:

    make check-llvm

And be sure to run the llvm-lit wrapper script as well:

    /path/to/your/llvm/build/bin/llvm-lit utils/lit/tests

Finally, make sure lit works when installed via setuptools:

    python utils/lit/setup.py install
    lit --path /path/to/your/llvm/build/bin utils/lit/tests