llvm-mirror

mirror of https://github.com/RPCS3/llvm-mirror.git synced 2024-11-24 19:52:54 +01:00

History

Brian Homerding c40a5a9f25 [lit] - Allow 1 test to report multiple micro-test results to provide support for microbenchmarks. Summary: These changes are to allow to a Result object to have nested Result objects in order to support microbenchmarks. Currently lit is restricted to reporting one result object for one test, this change provides support tests that want to report individual timings for individual kernels. This revision is the result of the discussions in https://reviews.llvm.org/D32272#794759, https://reviews.llvm.org/D37421#f8003b27 and https://reviews.llvm.org/D38496. It is a separation of the changes purposed in https://reviews.llvm.org/D40077. This change will enable adding LCALS (Livermore Compiler Analysis Loop Suite) collection of loop kernels to the llvm test suite using the google benchmark library (https://reviews.llvm.org/D43319) with tracking of individual kernel timings. Previously microbenchmarks had been handled by using macros to section groups of microbenchmarks together and build many executables while still getting a grouped timing (MultiSource/TSVC). Recently the google benchmark library was added to the test suite and utilized with a litsupport plugin. However the limitation of 1 test 1 result limited its use to passing a runtime option to run only 1 microbenchmark with several hand written tests (MicroBenchmarks/XRay). This runs the same executable many times with different hand-written tests. I will update the litsupport plugin to utilize the new functionality (https://reviews.llvm.org/D43316). These changes allow lit to report micro test results if desired in order to get many precise timing results from 1 run of 1 test executable. Reviewers: MatzeB, hfinkel, rengolin, delcypher Differential Revision: https://reviews.llvm.org/D43314 llvm-svn: 327422		2018-03-13 16:37:59 +00:00
..
examples
lit	[lit] - Allow 1 test to report multiple micro-test results to provide support for microbenchmarks.	2018-03-13 16:37:59 +00:00
tests	[lit] - Allow 1 test to report multiple micro-test results to provide support for microbenchmarks.	2018-03-13 16:37:59 +00:00
utils
CMakeLists.txt	[lit] Add a test for the builtin config map.	2017-09-21 16:18:28 +00:00
lit.py	[lit] Undo the patch to stop writing pyc files.	2017-09-20 20:31:24 +00:00
MANIFEST.in
README.txt	[lit] Remove TODO	2016-11-03 23:41:49 +00:00
setup.py

README.txt

===============================
 lit - A Software Testing Tool
===============================

lit is a portable tool for executing LLVM and Clang style test suites,
summarizing their results, and providing indication of failures. lit is designed
to be a lightweight testing tool with as simple a user interface as possible.

=====================
 Contributing to lit
=====================

Please browse the Test Suite > lit category in LLVM's Bugzilla for ideas on
what to work on.

Before submitting patches, run the test suite to ensure nothing has regressed:

    # From within your LLVM source directory.
    utils/lit/lit.py \
        --path /path/to/your/llvm/build/bin \
        utils/lit/tests

Note that lit's tests depend on 'not' and 'FileCheck', LLVM utilities.
You will need to have built LLVM tools in order to run lit's test suite
successfully.

You'll also want to confirm that lit continues to work when testing LLVM.
Follow the instructions in http://llvm.org/docs/TestingGuide.html to run the
regression test suite:

    make check-llvm

And be sure to run the llvm-lit wrapper script as well:

    /path/to/your/llvm/build/bin/llvm-lit utils/lit/tests

Finally, make sure lit works when installed via setuptools:

    python utils/lit/setup.py install
    lit --path /path/to/your/llvm/build/bin utils/lit/tests