The existing YAML Output::scalarString code path includes a partial and
incorrect implementation of YAML escaping logic. In particular, the logic put
in place in rL321283 escapes non-printable bytes only if they are not part of a
multibyte UTF8 sequence; implicitly this means that all multibyte UTF8
sequences -- printable and non -- are passed through verbatim.
The simplest solution to this is to direct the Output::scalarString method to
use the standalone yaml::escape function, and this _almost_ works, except that
the existing code in that function _over_ escapes: any multibyte UTF8 sequence
is escaped, even printable ones. While this is permitted for YAML, it is also
more aggressive (and hard to read for non-English locales) than necessary,
and the entire point of rL321283 was to back off such aggressive over-escaping.
So in this change, I have both redirected Output::scalarString to use
yaml::escape _and_ modified yaml::escape to optionally restrict its escaping to
non-printables. This preserves behaviour of any existing clients while giving
them a path to more moderate escaping should they desire.
Reviewers: JDevlieghere, thegameg, MatzeB, vladimir.plyashkun
Reviewed By: thegameg
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D44863
llvm-svn: 328661
- Remove use of the opencl and amdopencl environment member of the target triple for the AMDGPU target.
- Use function attribute to communicate to the AMDGPU backend to add implicit arguments for OpenCL kernels for the AMDHSA OS.
Differential Revision: https://reviews.llvm.org/D43736
llvm-svn: 328349
Summary:
This commit changes semantics of createUniqueFile and
createTemporaryFile variants that do not return file descriptors.
Previously they only checked if files exist, therefore being subject
to race conditions. Now they will create an empty file to avoid them.
Functions that do not create a file are now called
getPotentiallyUniqueTempFileName and getPotentiallyUniqueFileName.
Reviewers: klimek, bkramer, krasimir, JDevlieghere, espindola
Reviewed By: klimek
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D36827
llvm-svn: 327851
After r327219 was landed, the bot with expensive checks on GreenDragon
started failing. The problem was missing symbols `regex_t` and
`regmatch_t` in `xlocale/_regex.h`. The latter was included because
after the change in r327219, `random` is needed, which transitively
includes `xlocale.h.` which in turn conditionally includes
`xlocale/_regex.h` when _REGEX_H_ is defined. Because this is the header
guard in `regex_impl.h` and because `regex_impl.h` was included before
the other LLVM includes, `xlocale/_regex.h` was included without the
necessary types being available.
This commit fixes this by moving the include of `regex_impl.h` all the
way down. I also added a comment to stress the significance of its
position.
llvm-svn: 327256
Move the DWARF syntax highlighting into support. This has several
advantages, most notably that this makes the WithColor RAII wrapper
available outside libDebugInfo. Furthermore, several projects all have
their own code for handling colored output. This provides a place to
centralize it.
Differential revision: https://reviews.llvm.org/D44215
llvm-svn: 327108
This is like MemoryBuffer (read-only) and WritableMemoryBuffer
(writable private), but where the underlying file can be modified
after writing. This is useful when you want to open a file, make
some targeted edits, and then write it back out.
Differential Revision: https://reviews.llvm.org/D44230
llvm-svn: 327057
Summary:
Most of the time, compiler statistics can be obtained using a process that
performs a single compilation and terminates such as llc. However, this isn't
always the case. JITs for example, perform multiple compilations over their
lifetime and STATISTIC() will record cumulative values across all of them.
Provide tools like this with the facilities needed to measure individual
compilations by allowing them to reset the STATISTIC() values back to zero using
llvm::ResetStatistics(). It's still the tools responsibility to ensure that they
perform compilations in such a way that the results are meaningful to their
intended use.
Reviewers: qcolombet, rtereshin, bogner, aditya_nandakumar
Reviewed By: bogner
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D44181
llvm-svn: 326981
I think most of the Intel Core CPUs and recent AMD CPUs are unaffected. All the CPUs that have a "subtype" should work. The ones that were broken are the ones that are a "type" with no subtypes.
Fixes PR36619.
llvm-svn: 326840
These two functions iterate over the list of statistics but don't take the lock
that protects the iterators from being invalidated by
StatisticInfo::addStatistic().
So far, this hasn't been an issue since (in-tree at least) these functions are
called by the StatisticInfo destructor so addStatistic() shouldn't be called
anymore. However, we do expose them in the public API.
Note that this only protects against iterator invalidation and does not protect
against ordering issues caused by statistic updates that race with
PrintStatistics()/PrintStatisticsJSON().
Thanks to Roman Tereshin for spotting it
llvm-svn: 326834
Summary:
It can be useful for tools to be able to retrieve the values of variables
declared via STATISTIC() directly without having to emit them and parse
them back. Use cases include:
* Needing to report specific statistics to a test harness
* Wanting to post-process statistics. For example, to produce a percentage of
functions that were fully selected by GlobalISel
Make this possible by adding llvm::GetStatistics() which returns an
iterator_range that can be used to inspect the statistics that have been
touched during execution. When statistics are disabled (NDEBUG and not
LLVM_ENABLE_STATISTICS) this method will return an empty range.
This patch doesn't address the effect of multiple compilations within the same
process. In such situations, the statistics will be cumulative for all
compilations up to the GetStatistics() call.
Reviewers: qcolombet, rtereshin, aditya_nandakumar, bogner
Reviewed By: rtereshin, bogner
Subscribers: llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D43901
This re-commit fixes a missing include of <vector> which it seems clang didn't
mind but G++ and MSVC objected to. It seems that, clang was ok with std::vector
only being forward declared at the point of use since it was fully defined
eventually but G++/MSVC both rejected it at the point of use.
llvm-svn: 326738
Despite building cleanly on my machine in three separate configs, it's failing on pretty much all bots due to missing includes among other things. Investigating.
llvm-svn: 326726
Summary:
It can be useful for tools to be able to retrieve the values of variables
declared via STATISTIC() directly without having to emit them and parse
them back. Use cases include:
* Needing to report specific statistics to a test harness
* Wanting to post-process statistics. For example, to produce a percentage of
functions that were fully selected by GlobalISel
Make this possible by adding llvm::GetStatistics() which returns an
iterator_range that can be used to inspect the statistics that have been
touched during execution. When statistics are disabled (NDEBUG and not
LLVM_ENABLE_STATISTICS) this method will return an empty range.
This patch doesn't address the effect of multiple compilations within the same
process. In such situations, the statistics will be cumulative for all
compilations up to the GetStatistics() call.
Reviewers: qcolombet, rtereshin, aditya_nandakumar, bogner
Reviewed By: rtereshin, bogner
Subscribers: llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D43901
llvm-svn: 326723
Summary:
For various reasons, CMake's detection mechanism for `backtrace()`
returns an absolute path `/usr/lib/libexecinfo.so` on FreeBSD and
NetBSD.
Since `tools/llvm-config/CMakeLists.txt` only checks if system
libraries start with `-`, this causes `llvm-config --system-libs` to
produce the following incorrect output:
```
-lrt -l/usr/lib/libexecinfo.so -ltinfo -lpthread -lz -lm
```
Fix it by removing the path and the `lib` prefix, to make it look like a
regular short library name, suitable for appending to a `-l` link flag.
This also fixes the `Bindings/Go/go.test` test case, since that always
died with "unable to find library -l/usr/lib/libexecinfo.so".
Reviewers: chandlerc, emaste, joerg, krytarowski
Reviewed By: krytarowski
Subscribers: hans, bdrewery, mgorny, hintonda, llvm-commits
Differential Revision: https://reviews.llvm.org/D42702
llvm-svn: 326358
Until this patch, only `powerpc` and `ppc32` were recognized as valid
PowerPC 32-bit architectures in a target triple. This was incompatible
with the triple `ppc-apple-darwin` as returned for libObject. I found
out about this when working on a test case using a binary generated on
an old PowerBook G4.
We had the choice of either fix this in the Mach-O object parser or
in the Triple implementation. I chose the latter because it feels like
the most canonical place.
Differential revision: https://reviews.llvm.org/D43760
llvm-svn: 326182
This patch removes the HashString function from StringExtraces and
replaces its uses with calls to djbHash from DJB.h.
This change is *almost* NFC. While the algorithm is identical, the
djbHash implementation in StringExtras used 0 as its default seed while
the implementation in DJB uses 5381. The latter has been shown to result
in less collisions and improved avalanching and is used by the DWARF
accelerator tables.
Because some test were implicitly relying on the hash order, I've
reverted to using zero as a seed for the following two files:
lld/include/lld/Core/SymbolTable.h
llvm/lib/Support/StringMap.cpp
Differential revision: https://reviews.llvm.org/D43615
llvm-svn: 326091
It looks like some of our tests depend on the ordering of hashed values.
I'm reverting my changes while I try to reproduce and fix this locally.
Failing builds:
lab.llvm.org:8011/builders/lld-x86_64-darwin13/builds/18388
lab.llvm.org:8011/builders/clang-cmake-x86_64-sde-avx512-linux/builds/6743
lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/builds/15607
llvm-svn: 326082
This removes the HashString function from StringExtraces and replaces
its uses with calls to djbHash from DJB.h
This is *almost* NFC. While the algorithm is identical, the djbHash
implementation in StringExtras used 0 as its seed while the
implementation in DJB uses 5381. The latter has been shown to result in
less collisions and improved avalanching.
https://reviews.llvm.org/D43615
(cherry picked from commit 77f7f965bc9499a9ae768a296ca5a1f7347d1d2c)
llvm-svn: 326081
The issue was that the has function was generating different results depending
on the signedness of char on the host platform. This commit fixes the issue by
explicitly using an unsigned char type to prevent sign extension and
adds some extra tests.
The original commit message was:
This patch implements a variant of the DJB hash function which folds the
input according to the algorithm in the Dwarf 5 specification (Section
6.1.1.4.5), which in turn references the Unicode Standard (Section 5.18,
"Case Mappings").
To achieve this, I have added a llvm::sys::unicode::foldCharSimple
function, which performs this mapping. The implementation of this
function was generated from the CaseMatching.txt file from the Unicode
spec using a python script (which is also included in this patch). The
script tries to optimize the function by coalescing adjecant mappings
with the same shift and stride (terms I made up). Theoretically, it
could be made a bit smarter and merge adjecant blocks that were
interrupted by only one or two characters with exceptional mapping, but
this would save only a couple of branches, while it would greatly
complicate the implementation, so I deemed it was not worth it.
Since we assume that the vast majority of the input characters will be
US-ASCII, the folding hash function has a fast-path for handling these,
and only whips out the full decode+fold+encode logic if we encounter a
character outside of this range. It might be possible to implement the
folding directly on utf8 sequences, but this would also bring a lot of
complexity for the few cases where we will actually need to process
non-ascii characters.
Reviewers: JDevlieghere, aprantl, probinson, dblaikie
Subscribers: mgorny, hintonda, echristo, clayborg, vleschuk, llvm-commits
Differential Revision: https://reviews.llvm.org/D42740
llvm-svn: 325732
This is the second part of recommit of r325224. The previous part was
committed in r325426, which deals with C++ memory allocation. Solution
for C memory allocation involved functions `llvm::malloc` and similar.
This was a fragile solution because it caused ambiguity errors in some
cases. In this commit the new functions have names like `llvm::safe_malloc`.
The relevant part of original comment is below, updated for new function
names.
Analysis of fails in the case of out of memory errors can be tricky on
Windows. Such error emerges at the point where memory allocation function
fails, but manifests itself when null pointer is used. These two points
may be distant from each other. Besides, next runs may not exhibit
allocation error.
In some cases memory is allocated by a call to some of C allocation
functions, malloc, calloc and realloc. They are used for interoperability
with C code, when allocated object has variable size and when it is
necessary to avoid call of constructors. In many calls the result is not
checked for null pointer. To simplify checks, new functions are defined
in the namespace 'llvm': `safe_malloc`, `safe_calloc` and `safe_realloc`.
They behave as corresponding standard functions but produce fatal error if
allocation fails. This change replaces the standard functions like 'malloc'
in the cases when the result of the allocation function is not checked
for null pointer.
Finally, there are plain C code, that uses malloc and similar functions. If
the result is not checked, assert statement is added.
Differential Revision: https://reviews.llvm.org/D43010
llvm-svn: 325551
Summary: We probably mostly get this right due to family/model/stepping mapping to CPU names. But we should detect it explicitly.
Reviewers: RKSimon, echristo, dim, spatel
Reviewed By: dim
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D43418
llvm-svn: 325439
This is partial recommit of r325224, reverted in 325227. The relevant
part of original comment is below.
Analysis of fails in the case of out of memory errors can be tricky on
Windows. Such error emerges at the point where memory allocation function
fails, but manifests itself when null pointer is used. These two points
may be distant from each other. Besides, next runs may not exhibit
allocation error.
Usual programming practice does not require checking result of 'operator
new' because it throws 'std::bad_alloc' in the case of allocation error.
However, LLVM is usually built with exceptions turned off, so 'new' can
return null pointer. This change installs custom new handler, which causes
fatal error in the case of out of memory. The handler is installed
automatically prior to call to 'main' during construction of a static
object defined in 'lib/Support/ErrorHandling.cpp'. If the application does
not use this file, the handler may be installed manually by a call to
'llvm::install_out_of_memory_new_handler', declared in
'include/llvm/Support/ErrorHandling.h".
Differential Revision: https://reviews.llvm.org/D43010
llvm-svn: 325426
Summary: extractBits assumes that `!this->isSingleWord() implies !Result.isSingleWord()`, which may not necessarily be true. Handle both cases.
Reviewers: RKSimon
Subscribers: sanjoy, llvm-commits, hiraditya
Differential Revision: https://reviews.llvm.org/D43363
llvm-svn: 325311
There is a latent Windows kernel bug, the exact trigger
conditions are not well understood, which can cause a file
to be correctly written, but unable to be correctly read.
The workaround appears to be simply calling FlushFileBuffers.
Differential Revision: https://reviews.llvm.org/D42925
llvm-svn: 325274
Analysis of fails in the case of out of memory errors can be tricky on
Windows. Such error emerges at the point where memory allocation function
fails, but manifests itself when null pointer is used. These two points
may be distant from each other. Besides, next runs may not exhibit
allocation error.
Usual programming practice does not require checking result of 'operator
new' because it throws 'std::bad_alloc' in the case of allocation error.
However, LLVM is usually built with exceptions turned off, so 'new' can
return null pointer. This change installs custom new handler, which causes
fatal error in the case of out of memory. The handler is installed
automatically prior to call to 'main' during construction of a static
object defined in 'lib/Support/ErrorHandling.cpp'. If the application does
not use this file, the handler may be installed manually by a call to
'llvm::install_out_of_memory_new_handler', declared in
'include/llvm/Support/ErrorHandling.h".
There are calls to C allocation functions, malloc, calloc and realloc.
They are used for interoperability with C code, when allocated object has
variable size and when it is necessary to avoid call of constructors. In
many calls the result is not checked against null pointer. To simplify
checks, new functions are defined in the namespace 'llvm' with the
same names as these C function. These functions produce fatal error if
allocation fails. User should use 'llvm::malloc' instead of 'std::malloc'
in order to use the safe variant. This change replaces 'std::malloc'
in the cases when the result of allocation function is not checked against
null pointer.
Finally, there are plain C code, that uses malloc and similar functions. If
the result is not checked, assert statements are added.
Differential Revision: https://reviews.llvm.org/D43010
llvm-svn: 325224
Summary:
This patch implements a variant of the DJB hash function which folds the
input according to the algorithm in the Dwarf 5 specification (Section
6.1.1.4.5), which in turn references the Unicode Standard (Section 5.18,
"Case Mappings").
To achieve this, I have added a llvm::sys::unicode::foldCharSimple
function, which performs this mapping. The implementation of this
function was generated from the CaseMatching.txt file from the Unicode
spec using a python script (which is also included in this patch). The
script tries to optimize the function by coalescing adjecant mappings
with the same shift and stride (terms I made up). Theoretically, it
could be made a bit smarter and merge adjecant blocks that were
interrupted by only one or two characters with exceptional mapping, but
this would save only a couple of branches, while it would greatly
complicate the implementation, so I deemed it was not worth it.
Since we assume that the vast majority of the input characters will be
US-ASCII, the folding hash function has a fast-path for handling these,
and only whips out the full decode+fold+encode logic if we encounter a
character outside of this range. It might be possible to implement the
folding directly on utf8 sequences, but this would also bring a lot of
complexity for the few cases where we will actually need to process
non-ascii characters.
Reviewers: JDevlieghere, aprantl, probinson, dblaikie
Subscribers: mgorny, hintonda, echristo, clayborg, vleschuk, llvm-commits
Differential Revision: https://reviews.llvm.org/D42740
llvm-svn: 325107
Currently, each LLVM timer can be only printed once, as the act of
printing clears the timer.
Moreover, the current printing mechanism implicitly assumes that the
timer is stopped -- and prints zero otherwise.
This patch relaxes this assumption and makes printing statistics
multiple time a possibility.
Differential Revision: https://reviews.llvm.org/D43136
llvm-svn: 324788
This is a support change for a CFE change (https://reviews.llvm.org/D42978)
that allows march and -target-cpu to list the valid targets in a note. The changes
are limited to the ARM/AArch64, since this is the only target that gets the CPU
list from LLVM.
llvm-svn: 324623
This is a bit faster in theory, in practice it's cold code that's only
active in !NDEBUG, so it probably doesn't make a difference. This is one
of the last users of our homegrown Atomic.h.
llvm-svn: 323999
This patch moves the DJB hash to support. This is consistent with other
hashing algorithms living there. The hash is used by the DWARF
accelerator tables. We're doing this now because the hashing function is
needed by dsymutil and we don't want to link against libBinaryFormat.
Differential revision: https://reviews.llvm.org/D42594
llvm-svn: 323616
This makes wasm32-unknown-unknown-wasm the default, which supports
the .o file writer and the new linking ABI. To enable s2wasm-compatible
output, use the wasm32-unknown-unknown-elf triple.
llvm-svn: 323220
The original change was made based on a misunderstanding that
-DCMAKE_BUILD_TYPE=RelWithDebugInfo would produce the same executable
as -DCMAKE_BUILD_TYPE=Release modulo debug info. Turned out that's not
true -- it at least disables some optimizations such as function inlining.
llvm-svn: 323161
Try to detect the terminal color support by checking the value of the
TERM environment variable. This is not great, but it's better than
nothing when terminfo library isn't available, which may still be the
case on some Linux distributions.
Differential Revision: https://reviews.llvm.org/D42055
llvm-svn: 322962
This adds a new instrinsic to support the rdpid instruction. The implementation is a bit weird because the intrinsic is defined as always returning 32-bits, but the assembler support thinks the instruction produces a 64-bit register in 64-bit mode. But really it zeros the upper 32 bits. So I had to add separate patterns where 64-bit mode uses an extract_subreg.
Differential Revision: https://reviews.llvm.org/D42205
llvm-svn: 322910
