diff --git a/docs/XRayExample.rst b/docs/XRayExample.rst new file mode 100644 index 00000000000..7f52552c793 --- /dev/null +++ b/docs/XRayExample.rst @@ -0,0 +1,274 @@ +=================== +Debugging with XRay +=================== + +This document shows an example of how you would go about analyzing applications +built with XRay instrumentation. Here we will attempt to debug ``llc`` +compiling some sample LLVM IR generated by Clang. + +.. contents:: + :local: + +Building with XRay +------------------ + +To debug an application with XRay instrumentation, we need to build it with a +Clang that supports the ``-fxray-instrument`` option. See `XRay +&) const + 124 32 [ 0.000003, 0.000007, 0.000016, 0.000041, 0.000041] 0.000310 Verifier.cpp:1967:0: (anonymous namespace)::Verifier::visitFunction(llvm::Function const&) + 123 1 [ 0.000302, 0.000302, 0.000302, 0.000302, 0.000302] 0.000302 LLVMContextImpl.cpp:54:0: llvm::LLVMContextImpl::~LLVMContextImpl() + 139 46 [ 0.000000, 0.000002, 0.000006, 0.000008, 0.000019] 0.000138 TargetLowering.cpp:506:0: llvm::TargetLowering::SimplifyDemandedBits(llvm::SDValue, llvm::APInt const&, llvm::APInt&, llvm::APInt&, llvm::TargetLowering::TargetLoweringOpt&, unsigned int, bool) const + +This shows us that for our input file, ``llc`` spent the most cumulative time +in the lexer (a total of 1 millisecond). If we wanted for example to work with +this data in a spreadsheet, we can output the results as CSV using the +``-format=csv`` option to the command for further analysis. + +If we want to get a textual representation of the raw trace we can use the +``llvm-xray convert`` tool to get YAML output. The first few lines of that +ouput for an example trace would look like the following: + +:: + + $ llvm-xray convert -f yaml -symbolize -instr_map=./bin/llc xray-log.llc.m35qPB + --- + header: + version: 1 + type: 0 + constant-tsc: true + nonstop-tsc: true + cycle-frequency: 2601000000 + records: + - { type: 0, func-id: 110, function: __cxx_global_var_init.8, cpu: 37, thread: 69819, kind: function-enter, tsc: 5434426023268520 } + - { type: 0, func-id: 110, function: __cxx_global_var_init.8, cpu: 37, thread: 69819, kind: function-exit, tsc: 5434426023523052 } + - { type: 0, func-id: 164, function: __cxx_global_var_init, cpu: 37, thread: 69819, kind: function-enter, tsc: 5434426029925386 } + - { type: 0, func-id: 164, function: __cxx_global_var_init, cpu: 37, thread: 69819, kind: function-exit, tsc: 5434426030031128 } + - { type: 0, func-id: 142, function: '(anonymous namespace)::CommandLineParser::ParseCommandLineOptions(int, char const* const*, llvm::StringRef, llvm::raw_ostream*)', cpu: 37, thread: 69819, kind: function-enter, tsc: 5434426046951388 } + - { type: 0, func-id: 142, function: '(anonymous namespace)::CommandLineParser::ParseCommandLineOptions(int, char const* const*, llvm::StringRef, llvm::raw_ostream*)', cpu: 37, thread: 69819, kind: function-exit, tsc: 5434426047282020 } + - { type: 0, func-id: 187, function: 'llvm::LLLexer::LexIdentifier()', cpu: 37, thread: 69819, kind: function-enter, tsc: 5434426047857332 } + - { type: 0, func-id: 187, function: 'llvm::LLLexer::LexIdentifier()', cpu: 37, thread: 69819, kind: function-exit, tsc: 5434426047984152 } + - { type: 0, func-id: 187, function: 'llvm::LLLexer::LexIdentifier()', cpu: 37, thread: 69819, kind: function-enter, tsc: 5434426048036584 } + - { type: 0, func-id: 187, function: 'llvm::LLLexer::LexIdentifier()', cpu: 37, thread: 69819, kind: function-exit, tsc: 5434426048042292 } + - { type: 0, func-id: 187, function: 'llvm::LLLexer::LexIdentifier()', cpu: 37, thread: 69819, kind: function-enter, tsc: 5434426048055056 } + - { type: 0, func-id: 187, function: 'llvm::LLLexer::LexIdentifier()', cpu: 37, thread: 69819, kind: function-exit, tsc: 5434426048067316 } + +Controlling Fidelity +-------------------- + +So far in our examples, we haven't been getting full coverage of the functions +we have in the binary. To get that, we need to modify the compiler flags so +that we can instrument more (if not all) the functions we have in the binary. +We have two options for doing that, and we explore both of these below. + +Instruction Threshold +````````````````````` + +The first "blunt" way of doing this is by setting the minimum threshold for +function bodies to 1. We can do that with the +``-fxray-instruction-threshold=N`` flag when building our binary. We rebuild +``llc`` with this option and observe the results: + +:: + + $ rm CMakeCache.txt + $ cmake -GNinja ../llvm -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_C_FLAGS_RELEASE="-fxray-instrument -fxray-instruction-threshold=1" \ + -DCMAKE_CXX_FLAGS="-fxray-instrument -fxray-instruction-threshold=1" + $ ninja llc + $ XRAY_OPTIONS="patch_premain=true" ./bin/llc input.ll + ==69819==XRay: Log file in 'xray-log.llc.5rqxkU' + + $ llvm-xray account xray-log.llc.5rqxkU -top=10 -sort=sum -sortorder=dsc -instr_map ./bin/llc + Functions with latencies: 36652 + funcid count [ min, med, 90p, 99p, max] sum function + 75 1 [ 0.672368, 0.672368, 0.672368, 0.672368, 0.672368] 0.672368 llc.cpp:271:0: main + 78 1 [ 0.626455, 0.626455, 0.626455, 0.626455, 0.626455] 0.626455 llc.cpp:381:0: compileModule(char**, llvm::LLVMContext&) + 139617 1 [ 0.472618, 0.472618, 0.472618, 0.472618, 0.472618] 0.472618 LegacyPassManager.cpp:1723:0: llvm::legacy::PassManager::run(llvm::Module&) + 139610 1 [ 0.472618, 0.472618, 0.472618, 0.472618, 0.472618] 0.472618 LegacyPassManager.cpp:1681:0: llvm::legacy::PassManagerImpl::run(llvm::Module&) + 139612 1 [ 0.470948, 0.470948, 0.470948, 0.470948, 0.470948] 0.470948 LegacyPassManager.cpp:1564:0: (anonymous namespace)::MPPassManager::runOnModule(llvm::Module&) + 139607 2 [ 0.147345, 0.315994, 0.315994, 0.315994, 0.315994] 0.463340 LegacyPassManager.cpp:1530:0: llvm::FPPassManager::runOnModule(llvm::Module&) + 139605 21 [ 0.000002, 0.000002, 0.102593, 0.213336, 0.213336] 0.463331 LegacyPassManager.cpp:1491:0: llvm::FPPassManager::runOnFunction(llvm::Function&) + 139563 26096 [ 0.000002, 0.000002, 0.000037, 0.000063, 0.000215] 0.225708 LegacyPassManager.cpp:1083:0: llvm::PMDataManager::findAnalysisPass(void const*, bool) + 108055 188 [ 0.000002, 0.000120, 0.001375, 0.004523, 0.062624] 0.159279 MachineFunctionPass.cpp:38:0: llvm::MachineFunctionPass::runOnFunction(llvm::Function&) + 62635 22 [ 0.000041, 0.000046, 0.000050, 0.126744, 0.126744] 0.127715 X86TargetMachine.cpp:242:0: llvm::X86TargetMachine::getSubtargetImpl(llvm::Function const&) const + + +Instrumentation Attributes +`````````````````````````` + +The other way is to use configuration files for selecting which functions +should always be instrumented by the compiler. This gives us a way of ensuring +that certain functions are either always or never instrumented by not having to +add the attribute to the source. + +To use this feature, you can define one file for the functions to always +instrument, and another for functions to never instrument. The format of these +files are exactly the same as the SanitizerLists files that control similar +things for the sanitizer implementations. For example, we can have two +different files like below: + +:: + + # always-instrument.txt + # always instrument functions that match the following filters: + fun:main + + # never-instrument.txt + # never instrument functions that match the following filters: + fun:__cxx_* + +Given the above two files we can re-build by providing those two files as +arguments to clang as ``-fxray-always-instrument=always-instrument.txt`` or +``-fxray-never-instrument=never-instrument.txt``. + +Further Exploration +------------------- + +The ``llvm-xray`` tool has a few other subcommands that are in various stages +of being developed. One interesting subcommand that can highlight a few +interesting things is the ``graph`` subcommand. Given for example the following +toy program that we build with XRay instrumentation, we can see how the +generated graph may be a helpful indicator of where time is being spent for the +application. + +.. code-block:: c++ + + // sample.cc + #include + #include + + [[clang::xray_always_intrument]] void f() { + std::cerr << '.'; + } + + [[clang::xray_always_intrument]] void g() { + for (int i = 0; i < 1 << 10; ++i) { + std::cerr << '-'; + } + } + + int main(int argc, char* argv[]) { + std::thread t1([] { + for (int i = 0; i < 1 << 10; ++i) + f(); + }); + std::thread t2([] { + g(); + }); + t1.join(); + t2.join(); + std::cerr << '\n'; + } + +We then build the above with XRay instrumentation: + +:: + + $ clang++ -o sample -O3 sample.cc -std=c++11 -fxray-instrument -fxray-instruction-threshold=1 + $ XRAY_OPTIONS="patch_premain=true" ./sample + +We can then explore the graph rendering of the trace generated by this sample +application. We assume you have the graphviz toosl available in your system, +including both ``unflatten`` and ``dot``. If you prefer rendering or exploring +the graph using another tool, then that should be feasible as well. ``llvm-xray +graph`` will create DOT format graphs which should be usable in most graph +rendering applications. One example invocation of the ``llvm-xray graph`` +command should yield some interesting insights to the workings of C++ +applications: + +:: + + $ llvm-xray graph xray-log.sample.* -m sample -color-edges=sum -edge-label=sum \ + | unflatten -f -l10 | dot -Tsvg -o sample.svg + +Next Steps +---------- + +If you have some interesting analyses you'd like to implement as part of the +llvm-xray tool, please feel free to propose them on the llvm-dev@ mailing list. +The following are some ideas to inspire you in getting involved and potentially +making things better. + + - Implement a query/filtering library that allows for finding patterns in the + XRay traces. + - A conversion from the XRay trace onto something that can be visualised + better by other tools (like the Chrome trace viewer for example). + - Collecting function call stacks and how often they're encountered in the + XRay trace. + + diff --git a/docs/index.rst b/docs/index.rst index 341a9c16325..2385bb0f3cc 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -399,6 +399,9 @@ For API clients and LLVM developers. :doc:`XRay` High-level documentation of how to use XRay in LLVM. +:doc:`XRayExample` + An example of how to debug an application with XRay. + :doc:`The Microsoft PDB File Format ` A detailed description of the Microsoft PDB (Program Database) file format.