1
0
mirror of https://github.com/RPCS3/llvm-mirror.git synced 2024-10-19 19:12:56 +02:00
Commit Graph

43 Commits

Author SHA1 Message Date
Dean Michael Berris
2fe42806e8 [XRay] Improve FDR trace handling and error messaging
Summary:
This change covers a number of things spanning LLVM and compiler-rt,
which are related in a non-trivial way.

In LLVM, we have a library that handles the FDR mode even log loading,
which uses C++'s runtime polymorphism feature to better faithfully
represent the events that are written down by the FDR mode runtime. We
do this by interpreting a trace that's serliased in a common format
agreed upon by both the trace loading library and the FDR mode runtime.
This library is under active development, which consists of features
allowing us to reconstitute a higher-level event log.

This event log is used by the conversion and visualisation tools we have
for interpreting XRay traces.

One of the tools we have is a diagnostic tool in llvm-xray called
`fdr-dump` which we've been using to debug our expectations of what the
FDR runtime should be writing and what the logical FDR event log
structures are. We use this fairly extensively to reason about why some
non-trivial traces we're generating with FDR mode runtimes fail to
convert or fail to parse correctly.

One of these failures we've found in manual debugging of some of the
traces we've seen involve an inconsistency between the buffer extents (a
record indicating how many bytes to follow are part of a logical
thread's event log) and the record of the bytes written into the log --
sometimes it turns out the data could be garbage, due to buffers being
recycled, but sometimes we're seeing the buffer extent indicating a log
is "shorter" than the actual records associated with the buffer. This
case happens particularly with function entry records with a call
argument.

This change for now updates the FDR mode runtime to write the bytes for
the function call and arg record before updating the buffer extents
atomically, allowing multiple threads to see a consistent view of the
data in the buffer using the atomic counter associated with a buffer.
What we're trying to prevent here is partial updates where we see the
intermediary updates to the buffer extents (function record size then
call argument record size) becoming observable from another thread, for
instance, one doing the serialization/flushing.

To do both diagnose this issue properly, we need to be able to honour
the extents being set in the `BufferExtents` records marking the
beginning of the logical buffers when reading an FDR trace. Since LLVM
doesn't use C++'s RTTI mechanism, we instead follow the advice in the
documentation for LLVM Style RTTI
(https://llvm.org/docs/HowToSetUpLLVMStyleRTTI.html). We then rely on
this RTTI feature to ensure that our file-based record producer (our
streaming "deserializer") can honour the extents of individual buffers
as we interpret traces.

This also sets us up to be able to eventually do smart
skipping/continuation of FDR logs, seeking instead to find BufferExtents
records in cases where we find potentially recoverable errors. In the
meantime, we make this change to operate in a strict mode when reading
logical buffers with extent records.

Reviewers: mboerger

Subscribers: hiraditya, llvm-commits, jfb

Differential Revision: https://reviews.llvm.org/D54201

llvm-svn: 346473
2018-11-09 06:26:48 +00:00
Dean Michael Berris
d9e9a38395 [XRay] Use TSC delta encoding for custom/typed events
Summary:
This change updates the version number for FDR logs to 5, and update the
trace processing to support changes in the custom event records.

In the runtime, since we're already writing down the record preamble to
handle CPU migrations and TSC wraparound, we can use the same TSC delta
encoding in the custom event and typed event records that we use in
function event records. We do the same change to typed events (which
were unsupported before this change in the trace processing) which now
show up in the trace.

Future changes should increase our testing coverage to make custom and
typed events as first class entities in the FDR mode log processing
tools.

This change is also a good example of how we end up supporting new
record types in the FDR mode implementation. This shows the places where
new record types are added and supported.

Depends on D54139.

Reviewers: mboerger

Subscribers: hiraditya, arphaman, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D54140

llvm-svn: 346293
2018-11-07 04:37:42 +00:00
Dean Michael Berris
21c2bce874 [XRay] Update XRayRecord to support Custom/Typed Events
Summary:
This change cuts across LLVM and compiler-rt to add support for
rendering custom events in the XRayRecord type, to allow for including
user-provided annotations in the output YAML (as raw bytes).

This work enables us to add custom event and typed event records into
the `llvm::xray::Trace` type for user-provided events. This can then be
programmatically handled through the C++ API and can be included in some
of the tooling as well. For now we support printing the raw data we
encounter in the custom events in the converted output.

Future work will allow us to start interpreting these custom and typed
events through a yet-to-be-defined API for extending the trace analysis
library.

Reviewers: mboerger

Subscribers: hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D54139

llvm-svn: 346214
2018-11-06 08:51:37 +00:00
Dean Michael Berris
17defd68a8 [XRay] Add CPU ID in Custom Event FDR Records
Summary:
This change cuts across compiler-rt and llvm, to increment the FDR log
version number to 4, and include the CPU ID in the custom event records.

This is a step towards allowing us to change the `llvm::xray::Trace`
object to start representing both custom and typed events in the stream
of records. Follow-on changes will allow us to change the kinds of
records we're presenting in the stream of traces, to incorporate the
data in custom/typed events.

A follow-on change will handle the typed event case, where it may not
fit within the 15-byte buffer for metadata records.

This work is part of the larger effort to enable writing analysis and
processing tools using a common in-memory representation of the events
found in traces. The work will focus on porting existing tools in LLVM
to use the common representation and informing the design of a
library/framework for expressing trace event analysis as C++ programs.

Reviewers: mboerger, eizan

Subscribers: hiraditya, mgrang, llvm-commits

Differential Revision: https://reviews.llvm.org/D53920

llvm-svn: 345798
2018-11-01 00:18:52 +00:00
Dean Michael Berris
005b3bfebc [XRay][compiler-rt] FDRLogWriter Abstraction
Summary:
This change introduces an `FDRLogWriter` type which is responsible for
serialising metadata and function records to character buffers. This is
the first step in a refactoring of the implementation of the FDR runtime
to allow for more granular testing of the individual components of the
implementation.

The main contribution of this change is a means of hiding the details of
how specific records are written to a buffer, and for managing the
extents of these buffers. We make use of C++ features (templates and
some metaprogramming) to reduce repetition in the act of writing out
specific kinds of records to the buffer.

In this process, we make a number of changes across both LLVM and
compiler-rt to allow us to use the `Trace` abstraction defined in the
LLVM project in the testing of the runtime implementation. This gives us
a closer end-to-end test which version-locks the runtime implementation
with the loading implementation in LLVM.

We also allow using gmock in compiler-rt unit tests, by adding the
requisite definitions in the `AddCompilerRT.cmake` module. We also add
the terminfo library detection along with inclusion of the appropriate
compiler flags for header include lookup.

Finally, we've gone ahead and updated the FDR logging implementation to
use the FDRLogWriter for the lowest-level record-writing details.

Following patches will isolate the state machine transitions which
manage the set-up and tear-down of the buffers we're using in multiple
threads.

Reviewers: mboerger, eizan

Subscribers: mgorny, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D52220

llvm-svn: 342617
2018-09-20 05:22:37 +00:00
Evgeniy Stepanov
9b8acdf4a7 Revert "[XRay][compiler-rt] FDRLogWriter Abstraction" and 1 more.
Revert the following 2 commits to fix standalone compiler-rt build:
* r342523 [XRay] Detect terminfo library
* r342518 [XRay][compiler-rt] FDRLogWriter Abstraction

llvm-svn: 342596
2018-09-19 22:29:56 +00:00
Dean Michael Berris
4a3b31cfb1 [XRay][compiler-rt] FDRLogWriter Abstraction
Summary:
This change introduces an `FDRLogWriter` type which is responsible for
serialising metadata and function records to character buffers. This is
the first step in a refactoring of the implementation of the FDR runtime
to allow for more granular testing of the individual components of the
implementation.

The main contribution of this change is a means of hiding the details of
how specific records are written to a buffer, and for managing the
extents of these buffers. We make use of C++ features (templates and
some metaprogramming) to reduce repetition in the act of writing out
specific kinds of records to the buffer.

In this process, we make a number of changes across both LLVM and
compiler-rt to allow us to use the `Trace` abstraction defined in the
LLVM project in the testing of the runtime implementation. This gives us
a closer end-to-end test which version-locks the runtime implementation
with the loading implementation in LLVM.

We also allow using gmock in compiler-rt unit tests, by adding the
requisite definitions in the `AddCompilerRT.cmake` module.

Finally, we've gone ahead and updated the FDR logging implementation to
use the FDRLogWriter for the lowest-level record-writing details.

Following patches will isolate the state machine transitions which
manage the set-up and tear-down of the buffers we're using in multiple
threads.

Reviewers: mboerger, eizan

Subscribers: mgorny, jfb, llvm-commits

Differential Revision: https://reviews.llvm.org/D52220

llvm-svn: 342518
2018-09-18 23:59:32 +00:00
Dean Michael Berris
37533d7125 [XRay] Bug fixes for FDR custom event and arg-logging
Summary:
This change has a number of fixes for FDR mode in compiler-rt along with
changes to the tooling handling the traces in llvm.

In the runtime, we do the following:

- Advance the "last record" pointer appropriately when writing the
  custom event data in the log.

- Add XRAY_NEVER_INSTRUMENT in the rewinding routine.

- When collecting the argument of functions appropriately marked, we
  should not attempt to rewind them (and reset the counts of functions
  that can be re-wound).

In the tooling, we do the following:

- Remove the state logic in BlockIndexer and instead rely on the
  presence/absence of records to indicate blocks.

- Move the verifier into a loop associated with each block.

Reviewers: mboerger, eizan

Subscribers: llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D51965

llvm-svn: 342122
2018-09-13 09:25:42 +00:00
Dean Michael Berris
6a5880f261 [XRay] Ensure lambda outlives llvm::function_ref
Follow-up to D51912.

llvm-svn: 341912
2018-09-11 08:03:30 +00:00
Dean Michael Berris
76f5aab534 [XRay] Use FDR Records+Visitors for Trace Loading
Summary:
In this change, we overhaul the implementation for loading
`llvm::xray::Trace` objects from files by using the combination of
specific FDR Record types and visitors breaking up the logic to
reconstitute an execution trace from flight-data recorder mode traces.

This change allows us to handle out-of-temporal order blocks as written
in files, and more consistently recreate an execution trace spanning
multiple blocks and threads. To do this, we use the `WallclockRecord`
associated with each block to maintain temporal order of blocks, before
attempting to recreate an execution trace.

The new addition in this change is the `TraceExpander` type which can be
thought of as a decompression/decoding routine. This allows us to
maintain the state of an execution environment (thread+process) and
create `XRayRecord` instances that fit nicely into the `Trace`
container. We don't have a specific unit test for the TraceExpander
type, since the end-to-end tests for the `llvm-xray convert` tools
already cover precisely this codepath.

This change completes the refactoring started with D50441.

Depends on D51911.

Reviewers: mboerger, eizan

Subscribers: mgorny, hiraditya, mgrang, llvm-commits

Differential Revision: https://reviews.llvm.org/D51912

llvm-svn: 341906
2018-09-11 06:45:59 +00:00
Dean Michael Berris
501327fe86 [XRay] Add TSC to NewCPUId Records
Summary:
This more correctly reflects the data written by the FDR mode runtime.

This is a continuation of the work in D50441.

Reviewers: mboerger, eizan

Subscribers: hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D51911

llvm-svn: 341905
2018-09-11 06:36:51 +00:00
Dean Michael Berris
54bc26f33d [XRay] Add the llvm-xray fdr-dump implementation
Summary:
In this change, we implement a `BlockPrinter` which orders records in a
Block that's been indexed by the `BlockIndexer`. This is used in the
`llvm-xray fdr-dump` tool which ties together the various types and
utilities we've been working on, to allow for inspection of XRay FDR
mode traces both with and without verification.

This change is the final step of the refactoring of D50441.

Reviewers: mboerger, eizan

Subscribers: mgorny, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D51846

llvm-svn: 341887
2018-09-11 00:22:53 +00:00
Dean Michael Berris
665e1fb24b [XRay] Add a BlockVerifier visitor for FDR Records
Summary:
This patch implements a `BlockVerifier` type which enforces the
invariants of the log structure of FDR mode logs on a per-block basis.
This ensures that the data we encounter from an FDR mode log
semantically correct (i.e. that records follow the documented "grammar"
for FDR mode log records).

This is another part of the refactoring of D50441.

This is a slightly modified version of rL341628, avoiding the
`std::tuple<...>` constructor that is not constexpr in C++11.

Reviewers: mboerger, eizan

Subscribers: mgorny, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D51723

llvm-svn: 341769
2018-09-10 02:35:25 +00:00
Dean Michael Berris
1f415314f7 Revert "[XRay] Add a BlockVerifier visitor for FDR Records"
This reverts commit r341628.

llvm-svn: 341631
2018-09-07 03:38:26 +00:00
Dean Michael Berris
073b6320d5 [XRay] Add a BlockVerifier visitor for FDR Records
Summary:
This patch implements a `BlockVerifier` type which enforces the
invariants of the log structure of FDR mode logs on a per-block basis.
This ensures that the data we encounter from an FDR mode log
semantically correct (i.e. that records follow the documented "grammar"
for FDR mode log records).

This is another part of the refactoring of D50441.

Reviewers: mboerger, eizan

Subscribers: mgorny, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D51723

llvm-svn: 341628
2018-09-07 02:25:06 +00:00
Dean Michael Berris
f5133da999 [XRay] Add a BlockIndexer visitor for FDR Records.
Summary:
This change adds a `BlockIndexer` type which maintains pointers to
records that belong to the same process+thread pairs. The indexing
happens with order of appearance of records as they are visited.

This version of the indexer currently only supports FDR version 3 logs,
which contain `BufferExtent` records. We will add support for v2 and v1
logs in follow-up patches.

This is another part of D50441.

Reviewers: eizan, kpw, mboerger

Reviewed By: mboerger

Subscribers: mboerger, mgorny, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D51673

llvm-svn: 341518
2018-09-06 05:55:57 +00:00
Dean Michael Berris
05508f6a1d [XRay] Add a RecordPrinter visitor for FDR Records
Summary:
This change adds a `RecordPrinter` type which does some basic text
serialization of the FDR record instances. This is one component of the
tool we're building to dump the records from an FDR mode log as-is.

This is a small part of D50441.

Reviewers: eizan, kpw

Subscribers: mgorny, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D51672

llvm-svn: 341447
2018-09-05 06:57:23 +00:00
Dean Michael Berris
5bd013d85b [XRay] Use correct type for PID records
Previously we've been reading and writing the wrong types which only
worked in little endian implementations. This time we're writing the
same typed values the runtime is using, and reading them appropriately
as well.

llvm-svn: 341241
2018-08-31 19:32:46 +00:00
Dean Michael Berris
2a9df49e7d [XRay] Fix FunctionRecord serialization
This change makes the writer implementation more consistent with the way
fields are written down to avoid assumptions on bitfield order and
padding. We also fix an inconsistency between the type returned by the
`delta()` accessor to match the data member it's returning.

This is a follow-up to D51289 and D51210.

llvm-svn: 341230
2018-08-31 17:49:59 +00:00
Dean Michael Berris
c41598efcc [XRay] Make the FDRTraceWriter Endian-aware
Before this patch, the FDRTraceWriter would not take endianness into
account when writing data into the output stream.

This is a follow-up to D51289 and D51210.

llvm-svn: 341223
2018-08-31 16:08:38 +00:00
Dean Michael Berris
ddbb57e570 [XRay] FDR Record Producer/Consumer Implementation
Summary:
This patch defines two new base types called `RecordProducer` and
`RecordConsumer` which have default implementations for convenience
(particularly for testing).

A `RecordProducer` implementation has one member function called
`produce()` which serves as a factory constructor for `Record`
instances. This code exercises the `RecordInitializer` code path in the
implementation for `FileBasedRecordProducer`.

A `RecordConsumer` has a single member function called `consume(...)`
which, as the name implies, consumes instances of
`std::unique_ptr<Record>`. We have two implementations, one of which is
used in the test to generate a vector of `std::unique_ptr<Record>`
similar to how the `LogBuilder` implementation works.

We introduce a test in `FDRProducerConsumerTest` which ensures that
records we write through the `FDRTraceWriter` can be loaded by the
`FileBasedRecordProducer`. The record(s) loaded this way are written
again through the `FDRTraceWriter` into a separate string, which we then
compare. This ensures that the read-in bytes to create the `Record`
instances in memory can be replicated when written out through the
`FDRTraceWriter`.

This change depends on D51210 and is part of the refactoring of D50441
into smaller, more focused changes.

Reviewers: eizan, kpw

Subscribers: mgorny, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D51289

llvm-svn: 341180
2018-08-31 08:04:56 +00:00
Dean Michael Berris
dc767da403 [XRay] FDRTraceWriter and FDR Trace Loading
Summary:
This is the first step in the larger refactoring and reduction of
D50441.

This step in the process does the following:

- Introduces more granular types of `Record`s representing the many
  kinds of records written/read by the Flight Data Recorder (FDR) mode
  `Trace` loading function(s).

- Introduces an abstract `RecordVisitor` type meant to handle the
  processing of the various `Record` derived types. This `RecordVisitor`
  has two implementations in this patch: `RecordInitializer` and
  `FDRTraceWriter`.

- We also introduce a convenience interface for building a collection of
  `Record` instances called a `LogBuilder`. This allows us to generate
  sequences of `Record` instances manually (used in unit tests but
  useful otherwise).

- The`FDRTraceWriter` class implements the `RecordVisitor` interface and
  handles the writing of metadata records to a `raw_ostream`. We
  demonstrate that in the unit test, we can generate in-memory FDR mode
  traces using the specific `Record` derived types, which we load
  through the `loadTrace(...)` function yielding valid `Trace` objects.

This patch introduces the required types and concepts for us to start
replacing the logic implemented in the `loadFDRLog` function to use the
more granular types. In subsequent patches, we will introduce more
visitor implementations which isolate the verification, printing,
indexing, production/consumption, and finally the conversion of the FDR
mode logs.

The overarching goal of these changes is to make handling FDR mode logs
better tested, more understandable, more extensible, and more
systematic. This will also allow us to better represent the execution
trace, as we improve the fidelity of the events we represent in an XRay
`Trace` object, which we intend to do after FDR mode log processing is
in better shape.

Reviewers: eizan

Reviewed By: eizan

Subscribers: mgorny, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D51210

llvm-svn: 341029
2018-08-30 07:22:21 +00:00
Dean Michael Berris
a9b91bad26 [XRay][llvm] Load XRay Profiles
Summary:
This change implements the profile loading functionality in LLVM to
support XRay's profiling mode in compiler-rt.

We introduce a type named `llvm::xray::Profile` which allows building a
profile representation. We can load an XRay profile from a file to build
Profile instances, or do it manually through the Profile type's API.

The intent is to get the `llvm-xray` tool to generate `Profile`
instances and use that as the common abstraction through which all
conversion and analysis can be done. In the future we can generate
`Profile` instances from `Trace` instances as well, through conversion
functions.

Some of the key operations supported by the `Profile` API are:

- Path interning (`Profile::internPath(...)`) which returns a unique path
  identifier.

- Block appending (`Profile::addBlock(...)`) to add thread-associated
  profile information.

- Path ID to Path lookup (`Profile::expandPath(...)`) to look up a
  PathID and return the original interned path.

- Block iteration.

A 'Path' in this context represents the function call stack in
leaf-to-root order. This is represented as a path in an internally
managed prefix tree in the `Profile` instance. Having a handle (PathID)
to identify the unique Paths we encounter for a particular Profile
allows us to reduce the amount of memory required to associate profile
data to a particular Path.

This is the first of a series of patches to migrate the `llvm-stacks`
tool towards using a single profile representation.

Depends on D48653.

Reviewers: kpw, eizan

Reviewed By: kpw

Subscribers: kpw, thakis, mgorny, llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D48370

llvm-svn: 341012
2018-08-30 01:43:22 +00:00
Dean Michael Berris
66fff7c74c [XRay] Refactor loadTraceFile(...) into two (NFC)
This patch splits the file trace loading function into two versions, one
that takes a filename and one that takes a `DataExtractor`.

This change is a precursor to larger changes to increase test coverage
for the trace loading implementation.

llvm-svn: 340603
2018-08-24 10:30:37 +00:00
Dean Michael Berris
f9e1fbc647 [XRay] Refactor file header reading (NFC)
Summary:
This patch moves out the definition of the XRay log file header from
binary logs into its own header and implementation file.

This is one part of the refactoring being done in D50441.

Reviewers: eizan

Subscribers: mgorny, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D51086

llvm-svn: 340389
2018-08-22 07:37:55 +00:00
Galina Kistanova
241ffa16b1 Reverted r338825 and all the following tries to fix issues introduced by that commit (r338826, r338827, r338829, r338880).
This commit has broken build bots and has been left unattended for too long.

llvm-svn: 338948
2018-08-04 01:59:12 +00:00
Dean Michael Berris
e289480427 [XRay] Fixup: remove 'noexcept' in defaulted move members
This is to appease stage1 builds using gcc.

Follow-up to D48370.

llvm-svn: 338826
2018-08-03 07:41:34 +00:00
Dean Michael Berris
0c7cbedfef [XRay][llvm] Load XRay Profiles
Summary:
This change implements the profile loading functionality in LLVM to
support XRay's profiling mode in compiler-rt.

We introduce a type named `llvm::xray::Profile` which allows building a
profile representation. We can load an XRay profile from a file to build
Profile instances, or do it manually through the Profile type's API.

The intent is to get the `llvm-xray` tool to generate `Profile`
instances and use that as the common abstraction through which all
conversion and analysis can be done. In the future we can generate
`Profile` instances from `Trace` instances as well, through conversion
functions.

Some of the key operations supported by the `Profile` API are:

- Path interning (`Profile::internPath(...)`) which returns a unique path
  identifier.

- Block appending (`Profile::addBlock(...)`) to add thread-associated
  profile information.

- Path ID to Path lookup (`Profile::expandPath(...)`) to look up a
  PathID and return the original interned path.

- Block iteration.

A 'Path' in this context represents the function call stack in
leaf-to-root order. This is represented as a path in an internally
managed prefix tree in the `Profile` instance. Having a handle (PathID)
to identify the unique Paths we encounter for a particular Profile
allows us to reduce the amount of memory required to associate profile
data to a particular Path.

This is the first of a series of patches to migrate the `llvm-stacks`
tool towards using a single profile representation.

Depends on D48653.

Reviewers: kpw, eizan

Reviewed By: kpw

Subscribers: mgorny, llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D48370

llvm-svn: 338825
2018-08-03 07:18:39 +00:00
Dean Michael Berris
aab4e8c7ac [XRay][compiler-rt] Add PID field to llvm-xray tool and add PID metadata record entry in FDR mode
Summary:
llvm-xray changes:
- account-mode - process-id  {...} shows after thread-id
- convert-mode - process {...} shows after thread
- parses FDR and basic mode pid entries
- Checks version number for FDR log parsing.

Basic logging changes:
- Update header version from 2 -> 3

FDR logging changes:
- Update header version from 2 -> 3
- in writeBufferPreamble, there is an additional PID Metadata record (after thread id record and tsc record)

Test cases changes:
- fdr-mode.cc, fdr-single-thread.cc, fdr-thread-order.cc modified to catch process id output in the log.

Reviewers: dberris

Reviewed By: dberris

Subscribers: hiraditya, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49153

llvm-svn: 336974
2018-07-13 05:38:22 +00:00
David Blaikie
f412868b63 XRayRecord.h: Add missing #include
llvm-svn: 318713
2017-11-21 00:23:19 +00:00
David Blaikie
2b177f1e03 YAML/XRay/std::vector: Fix ODR violation by removing local specialization
There's a generic partial specialization for all std::vector<T> that
does what's desired, so no need for this full specialization that's
causing an ODR violation anyway.

llvm-svn: 318712
2017-11-21 00:23:17 +00:00
Martin Pelikan
33c8e86564 [XRay] convert FDR arg1 log entries
Summary:
A new FDR metadata record will support logging a function call argument;
appending multiple metadata records will represent a sequence of arguments
meaning that "holes" are not representable by the buffer format.  Each
call argument is currently a 64-bit value (useful for "this" pointers and
synchronization objects).

If present, we put this argument to the function call "entry" record it
belongs to, and alter its type to notify the user of its presence.

Reviewers: dberris

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D32840

llvm-svn: 314269
2017-09-27 04:48:03 +00:00
Dean Michael Berris
834a58b0df [XRay][tools] Support tail-call exits before we write them in the runtime
Summary:
This change adds support for explicit tail-exit records to be written by
the XRay runtime. This lets us differentiate the tail exit
records/events in the log, and allows us to treat those exit events
especially in the future. For now we allow printing those out in YAML
(and reading them in).

Reviewers: kpw, pelikan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D37964

llvm-svn: 313514
2017-09-18 06:08:46 +00:00
Dean Michael Berris
08d8030831 [XRay][tools] Support new kinds of instrumentation map entries
Summary:
When extracting the instrumentation map from a binary, we should be able
to recognize the new kinds of instrumentation sleds we've been emitting
with the compiler using -fxray-instrument. This change adds a test for
all the kinds of sleds we currently support (sans the tail-call sled,
which is a bit harder to force in a simple prebuilt input).

Reviewers: kpw, dblaikie

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D36819

llvm-svn: 311305
2017-08-21 00:14:06 +00:00
Dean Michael Berris
bb6b9f32df [XRay][tools] Add option to llvm-xray extract to symbolize functions
Summary:
This allows us to, if the symbol names are available in the binary, be
able to provide the function name in the YAML output.

Reviewers: dblaikie, pelikan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D32153

llvm-svn: 300624
2017-04-18 23:23:54 +00:00
Dean Michael Berris
37f9897224 [XRay] Update FDR log reader to be aware of buffer sizes per thread.
Summary:
It is problematic for this reader that it expects to read data from
several threads, but the header or message format does not define
framing. Since the buffers are reused, we can't rely on skipping
zeroed out data as a synchronization method either.

There is an argument that this is not version compatible with the format
the reader expected previously. I argue that since the writer wrote garbage
past the end of buffer record, there is no currently working reader to
compromise.

The corresponding writer change is posted to D31384.

Reviewers: dberris, pelikan

Reviewed By: dberris

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D31385

llvm-svn: 298983
2017-03-29 06:10:12 +00:00
Dean Michael Berris
cdc62f0bc3 [XRAY] [x86_64] Adding a Flight Data filetype reader to the llvm-xray Trace implementation.
Summary:
The file type packs function trace data onto disk from potentially multiple
threads that are aggregated and flushed during the course of an instrumented
program's runtime.

It is named FDR mode or Flight Data recorder as an analogy to plane
blackboxes, which instrument a running system without access to IO.

The writer code is defined in compiler-rt in xray_fdr_logging.h/cc

Reviewers: rSerge, kcc, dberris

Reviewed By: dberris

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D29697

llvm-svn: 295397
2017-02-17 01:47:16 +00:00
Dean Michael Berris
cc37553db4 [XRay] A graph Class for the llvm-xray graph
Summary:
In preparation for graph comparison and filtering, this is a library for
representing graphs in LLVM. This will enable easier encapsulation and reuse
of graphs in llvm-xray.

Depends on D28999, D28225

Reviewers: dblaikie, dberris

Reviewed By: dberris

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D29005

llvm-svn: 294717
2017-02-10 06:36:08 +00:00
Dean Michael Berris
dd1f0a3f3a Revert "[XRay] A graph Class for the llvm-xray graph"
Broke tests, reverting.

llvm-svn: 294714
2017-02-10 06:05:46 +00:00
Dean Michael Berris
cadcaaab9b [XRay] A graph Class for the llvm-xray graph
Summary:
In preparation for graph comparison and filtering, this is a library for
representing graphs in LLVM. This will enable easier encapsulation and reuse
of graphs in llvm-xray.

Depends on D28999, D28225

Reviewers: dblaikie, dberris

Reviewed By: dberris

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D29005

llvm-svn: 294713
2017-02-10 05:40:37 +00:00
Eugene Zelenko
347afa6803 [MC] Fix some Clang-tidy modernize and Include What You Use warnings in SubtargetFeature; other minor fixes (NFC).
Same changes in files affected by reduced SubtargetFeature.h dependencies.

llvm-svn: 294548
2017-02-09 01:09:54 +00:00
Dean Michael Berris
c91cd5f47b [XRay] Define the InstrumentationMap type
Summary:
This change implements the instrumentation map loading library which can
understand both YAML-defined instrumentation maps, and ELF 64-bit object
files that have the XRay instrumentation map section. We break it out
into a library on its own to allow for other applications to deal with
the XRay instrumentation map defined in XRay-instrumented binaries.

This type provides both raw access to the logical representation of the
instrumentation map entries as well as higher level functions for
converting a function ID into a function address.

At this point we only support ELF64 binaries and YAML-defined XRay
instrumentation maps. Future changes should extend this to support
32-bit ELF binaries, as well as other binary formats (like MachO).

As part of this change we also migrate all uses of the extraction logic
that used to be defined in tools/llvm-xray/ to use this new type and
interface for loading from files. We also remove the flag from the
`llvm-xray` tool that required users to specify the type of the
instrumentation map file being provided to instead make the library
auto-detect the file type.

Reviewers: dblaikie

Subscribers: mgorny, varno, llvm-commits

Differential Revision: https://reviews.llvm.org/D29319

llvm-svn: 293721
2017-02-01 00:05:29 +00:00
Dean Michael Berris
85c0bb36c3 [XRay] Define the library for XRay trace logs
Summary:
In this change we move the definition of the log reading routines from
the tools directory in LLVM to {include/llvm,lib}/XRay. We improve the
documentation a little bit for the publicly accessible headers, and
adjust the top-matter. This also leads to some refactoring and cleanup
in the tooling code.

In particular, we do the following:

  - Rename the class from LogReader to Trace, as it better represents
    the logical set of records as opposed to a log.
  - Use file type detection instead of asking the user to say what
    format the input file is. This allows us to keep the interface
    simple and encapsulate the logic of loading the data appropriately.

In future changes we increase the API surface and write dedicated unit
tests for the XRay library.

Depends on D24376.

Reviewers: dblaikie, echristo

Subscribers: mehdi_amini, mgorny, llvm-commits, varno

Differential Revision: https://reviews.llvm.org/D28345

llvm-svn: 291652
2017-01-11 06:39:09 +00:00