This should fix build breaks for 'development' mode. The other modes
were unaffected - 'release' because it doesn't use TFUtils.cpp, and the
mixed mode because the AOT compiled code brings in the necessary include
dirs anyway.
It turns out that during training, the time required to parse the
textual protobuf of a training log is about the same as the time it
takes to compile the module generating that log. Using binary protobufs
instead elides that cost almost completely.
Differential Revision: https://reviews.llvm.org/D106157
With this patch, `FLANG_BUILD_NEW_DRIVER` is set to `On` by default
(i.e. the new driver is enabled). Note that the new driver depends on
Clang and hence with this change you will need to add `clang` to
`LLVM_ENABLE_PROJECTS`.
If you don't want to build the new driver, set `FLANG_BUILD_NEW_DRIVER`
to `Off`. This way you won't be required to include `clang` in
`LLVM_ENABLE_PROJECTS`.
Differential Revision: https://reviews.llvm.org/D101842
Using `cmake` >=3.20 results in many warnings about this new policy. This change silences the warnings by explicitly declaring use of the "OLD" behavior.
This policy currently affects only one place: the `tablegen()` function in `TableGen.cmake`.
Differential Revision: https://reviews.llvm.org/D101083
This was prompted by D95727, which had the side-effect to break the
'release' mode build bot for ML-driven policies. The problem is that now
the pre-compiled object files don't get transitively carried through as
'source' anymore; that being said, the previous way of consuming them
was problematic, because it was only working for static builds; in
dynamic builds, the whole tf_xla_runtime was linked, which is
undesirable.
The alternative is to treat tf_xla_runtime as an archive, which then
leads to the desired effect.
Differential Revision: https://reviews.llvm.org/D99829
Call `get_errc_messages` only if `LLVM_INCLUDE_TESTS` was set.
Reviewed By: zero9178, abhina.sreeskantharajan, jhenderson
Differential Revision: https://reviews.llvm.org/D98861
Visual Studios implementation of the C++ Standard Library does not use strerror to produce a message for std::error_code unlike other standard libraries such as libstdc++ or libc++ that might be used.
This patch adds a cmake script that through running a C++ program gets the error messages for the POSIX error codes and passes them onto lit through an optional config parameter.
If the config parameter is not set, or getting the messages failed, due to say a cross compiling configuration without an emulator, it will fall back to using pythons strerror functions.
Differential Revision: https://reviews.llvm.org/D98278
The lit test suite uses python 3.6 features. Rather than a strange
python syntax error upon running the lit tests, we will require the
correct version in CMake.
Reviewed By: serge-sans-paille, yln
Differential Revision: https://reviews.llvm.org/D95635
As suggested by Nico in https://reviews.llvm.org/D96049, move check for libproc
from CMakeLists to config-ix.cmake
Reviewed By: thakis
Differential Revision: https://reviews.llvm.org/D96713
Note: Also removes the CMAKE_REQUIRED_LIBRARIES which doesn't appear to
be necessary.
In addition to wall time etc. this should allow us to get less noisy
values for time measurements.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D96049
This turns on the new pass manager by default for the optimization pipeline in
Clang and ThinLTO in various LLD backends. This also makes uses of `opt
-instcombine` use the new pass manager (unless specifically opted out).
This does not affect the backend target-dependent codegen pipeline.
If this causes regressions, you can opt out of the new pass manager
either via the -DENABLE_EXPERIMENTAL_NEW_PASS_MANAGER=OFF CMake flag
while building LLVM, or via various compiler flags, e.g.
-flegacy-pass-manager for Clang or -Wl,--lto-legacy-pass-manager for
ELF LLD. Please file bugs for any regressions.
Major differences:
* The inliner works slightly differently
* -O1 does some amount of inlining
* LCSSA and LoopSimplify are run before all loop passes
* Loop unswitching is implemented slightly differently
* A new SpeculateAroundPHIs pass is added to the pipeline
https://lists.llvm.org/pipermail/llvm-dev/2021-January/148098.html
Reviewed By: asbirlea, ychen, MaskRay, echristo
Differential Revision: https://reviews.llvm.org/D95380
LLVM_TARGETS_TO_BUILD accepts both "host" or "Native" for auto-selecting
the target from the environment. However the way "Native" was plumbed
would lead to the JIT environment being disabled. This patch is making
"Native" works just as "host".
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D95837
There are builders that do not have python 3.6. Revert until this situation can be rectified
This reverts commit 0703b0753c40dad30f1683403f6600bd2cb42055.
Previously, CMake would find any version of Python3. However, the project
claims to require 3.6 or greater, and 3.6 features are being used.
Reviewed By: yln
Differential Revision: https://reviews.llvm.org/D95635
cmake_minimum_required(VERSION) calls cmake_policy(VERSION),
which sets all policies up to VERSION to NEW.
LLVM started requiring CMake 3.13 last year, so we can remove
a bunch of code setting policies prior to 3.13 to NEW as it
no longer has any effect.
Reviewed By: phosek, #libunwind, #libc, #libc_abi, ldionne
Differential Revision: https://reviews.llvm.org/D94374
Remove the hack adding /usr/local paths on FreeBSD and DragonFlyBSD.
It does not seem to be necessary today, and it breaks cross builds.
Differential Revision: https://reviews.llvm.org/D94491
This is needed for cross-compiling LLVM from Cygwin, but it had gotten
deleted in rG2724d9e12960cc1d93eeabbfc9aa1bffffa041cc
Reviewed By: compnerd
Differential Revision: https://reviews.llvm.org/D92336
This allows us to use its value everywhere, rather than just clang. Some
other places, like opt and lld, will use its value soon.
Rename it internally to LLVM_ENABLE_NEW_PASS_MANAGER.
The #define for it is now in llvm-config.h.
The initial land accidentally set the value of
LLVM_ENABLE_NEW_PASS_MANAGER to the string
ENABLE_EXPERIMENTAL_NEW_PASS_MANAGER instead of its value.
Reviewed By: rnk, hans
Differential Revision: https://reviews.llvm.org/D92072
This allows us to use its value everywhere, rather than just clang. Some
other places, like opt and lld, will use its value soon.
The #define for it is now in llvm-config.h.
Reviewed By: rnk, hans
Differential Revision: https://reviews.llvm.org/D92072
No longer rely on an external tool to build the llvm component layout.
Instead, leverage the existing `add_llvm_componentlibrary` cmake function and
introduce `add_llvm_component_group` to accurately describe component behavior.
These function store extra properties in the created targets. These properties
are processed once all components are defined to resolve library dependencies
and produce the header expected by llvm-config.
Differential Revision: https://reviews.llvm.org/D90848
Allow overriding the default set of flags used to enable UBSan when
building llvm.
This can be used to test new checks or opt out of certain checks.
Differential Revision: https://reviews.llvm.org/D89439
The change in 0ba98433971f changed the behaviour of the build when
using an XL build compiler because `-G` is not a pure linker option:
it also implies `-shared`. This was accounted for in the base CMake
configuration, so an analysis of the change from 0ba98433971f in
relation to a build using Clang (where `-shared` is introduced by CMake)
would not identify the issue. This patch resolves this particular issue
by adding `-shared` alongside `-Wl,-G`.
At the same time, the investigation reveals that several aspects of the
various build configurations are not operating in the manner originally
intended.
The other issue related to the `-G` linker option in the build is that
the removal of it (to avoid unnecessary use of run-time linking) is not
effective for the build using the Clang compiler. This patch addresses
this by adjusting the regular expressions used to remove the broadly-
applied `-G`.
Finally, the issue of specifying the export list with `-Wl,` instead of
a compiler option is flagged with a FIXME comment.
Reviewed By: daltenty, amyk
Differential Revision: https://reviews.llvm.org/D90041
1. Emit error for -G driver option on AIX
2. Adjust cmake file to use -Wl,-G instead of -G
On AIX, legacy XL compiler uses -G to produce a shared object enabled
for use with the run-time linker, which has different meanings from what
it is used for in Clang. And in Clang, other targets do not have -G map
to another functionality in their legacy compiler. So this error is more
important when we are on AIX.
Differential Revision: https://reviews.llvm.org/D89897
Also have CMake fails if the user provides a TENSORFLOW_C_LIB_PATH but
we can't find TensorFlow at this path.
At the moment the CMake script tries to figure if TensorFlow is
available on the system and enables support for it. This is in general
not desirable to customize build features this way and instead it is
preferable to let the user opt-in explicitly into the features they want
to enable. This is in line with other optional external dependencies
like Z3.
There are a few reasons to this but amongst others:
- reproducibility: making features "magically" enabled based on whether
we find a package on the system or not makes it harder to handle bug
reports from users.
- user control: they can't have TensorFlow on the system and build LLVM
without TensorFlow right now. They also would suddenly distribute LLVM
with a different set of features unknowingly just because their build
machine environment would change subtly.
Right now this is motivated by a user reporting build failures on their system:
.../mesa-git/llvm-git/src/llvm-project/llvm/lib/Analysis/TFUtils.cpp:23:10: fatal error: tensorflow/c/c_api.h: No such file or directory
23 | #include "tensorflow/c/c_api.h"
| ^~~~~~
It looks like we detected TensorFlow at configure time but couldn't set all the paths correctly.
Differential Revision: https://reviews.llvm.org/D88371
Configure default value of `LLVM_ENABLE_WARNINGS` in `HandleLLVMOptions.cmake`.
`LLVM_ENABLE_WARNINGS` is documented as ON by default, but `HandleLLVMOptions` assumes the default has been set somewhere else. If it has not been explicitly set, then `HandleLLVMOptions` implicitly uses OFF as a default.
This removes the various `option()` declarations in favor of a single declaration in `HandleLLVMOptions`. This will prevent the unwanted use of `-w` that is mentioned in a couple of the comments.
Reviewed By: DavidTruby, #libunwind, JDevlieghere, compnerd
Differential Revision: https://reviews.llvm.org/D87243
LLVM has bumped the minimum required CMake version to 3.13.4, so this has become dead code.
Reviewed By: #libc, ldionne
Differential Revision: https://reviews.llvm.org/D87189
In GNU ld, gold and LLD, --no-allow-shlib-undefined is the default when
linking an executable. The option disallows unresolved symbols in shared objects.
(gold and LLD catch fewer cases than GNU ld. See D57385 for details)
See D57569 why it is bad idea to use --allow-shlib-undefined for executables [a].
GNU ld traditionally copied DT_NEEDED entries transitively. This was
deemed not good, so GNU ld 2.22 defaulted to --no-copy-dt-needed-entries.
gold and LLD always behave like --no-copy-dt-needed-entries.
rL221530 added -Wl,-allow-shlib-undefined to make some old releases of GNU ld's
--no-copy-dt-needed-entries to actually work.
Due to [a] and [b], this patch drops -Wl,-allow-shlib-undefined.
[b]: In a -DBUILD_SHARED_LIBS=on build, `--as-needed --allow-shlib-undefined`
can unexpectedly suppress some .dynsym entries. The issue can cause
mlir-cpu-runner to fail at runtime. Note, on Debian, gcc newer than (gcc-9-20190125-2) enable
--as-needed by default.
See https://sourceware.org/bugzilla/show_bug.cgi?id=26551 for a reduced example.
Reviewed By: mehdi_amini, echristo
Differential Revision: https://reviews.llvm.org/D86839
Currently it is hard to avoid having LLVM link to the system install of
ncurses, since it uses check_library_exists to find e.g. libtinfo and
not find_library or find_package.
With this change the ncurses lib is found with find_library, which also
considers CMAKE_PREFIX_PATH. This solves an issue for the spack package
manager, where we want to use the zlib installed by spack, and spack
provides the CMAKE_PREFIX_PATH for it.
This is a similar change as https://reviews.llvm.org/D79219, which just
landed in master.
Patch By: haampie
Differential Revision: https://reviews.llvm.org/D85820
This patch optionally replaces the CRT allocator (i.e., malloc and free) with rpmalloc (mixed public domain licence/MIT licence) or snmalloc (MIT licence) or mimalloc (MIT licence). Please note that the source code for these allocators must be available outside of LLVM's tree.
To enable, use `cmake ... -DLLVM_INTEGRATED_CRT_ALLOC=D:/git/rpmalloc -DLLVM_USE_CRT_RELEASE=MT` where `D:/git/rpmalloc` has already been git clone'd from `https://github.com/mjansson/rpmalloc`. The same applies to snmalloc and mimalloc.
When enabled, the allocator will be embeded (statically linked) into the LLVM tools & libraries. This currently only works with the static CRT (/MT), although using the dynamic CRT (/MD) could potentially work as well in the future.
When enabled, this changes the memory stack from:
new/delete -> MS VC++ CRT malloc/free -> HeapAlloc -> VirtualAlloc
to:
new/delete -> {rpmalloc|snmalloc|mimalloc} -> VirtualAlloc
The goal of this patch is to bypass the application's global heap - which is thread-safe thus inducing locking - and instead take advantage of a modern lock-free, thread cache, allocator. On a 6-core Xeon Skylake we observe a 2.5x decrease in execution time when linking a large scale application with LLD and ThinLTO (12 min 20 sec -> 5 min 34 sec), when all hardware threads are being used (using LLD's flag /opt:lldltojobs=all). On a dual 36-core Xeon Skylake with all hardware threads used, we observe a 24x decrease in execution time (1 h 2 min -> 2 min 38 sec) when linking a large application with LLD and ThinLTO. Clang build times also see a decrease in the range 5-10% depending on the configuration.
Differential Revision: https://reviews.llvm.org/D71786
This change extend the CMake files with the necessary additions
to build LLVM for z/OS.
Reviewed By: hubert.reinterpretcast
Differential Revision: https://reviews.llvm.org/D83866
We can just use the definition from config.h. This means we need to move
a few lines around in CMakeLists.txt - the TF_AOT detection needs to be
before the spot we process the config.h.cmake files.
Differential Revision: https://reviews.llvm.org/D84349
Summary:
This allows users of the llvm library discover whether llvm was built
with the tensorflow c API dependency, which helps if using the TFUtils
wrapper, for example.
We don't do the same for the LLVM_HAVE_TF_AOT flag, because that does
not expose any API.
Reviewers: mehdi_amini, davidxl
Subscribers: mgorny, aaron.ballman, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D83746
This reverts commit 9908a3b9f521c954cbf6adcec35b14b2f6c8da49.
The fix was to exclude the content of TFUtils.h (automatically
included in the LLVM_Analysis module, when LLVM_ENABLE_MODULES is enabled).
Differential Revision: https://reviews.llvm.org/D82817
Summary:
This is an experimental ML-based native size estimator, necessary for
computing partial rewards during -Oz inliner policy training. Data
extraction for model training will be provided in a separate patch.
RFC: http://lists.llvm.org/pipermail/llvm-dev/2020-April/140763.html
Reviewers: davidxl, jdoerfert
Subscribers: mgorny, hiraditya, mgrang, arphaman, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D82817
