1
0
mirror of https://github.com/RPCS3/llvm-mirror.git synced 2024-10-20 03:23:01 +02:00
Commit Graph

217 Commits

Author SHA1 Message Date
Teresa Johnson
6fd4796f76 Resolution-based LTO API.
Summary:
This introduces a resolution-based LTO API. The main advantage of this API over
existing APIs is that it allows the linker to supply a resolution for each
symbol in each object, rather than the combined object as a whole. This will
become increasingly important for use cases such as ThinLTO which require us
to process symbol resolutions in a more complicated way than just adjusting
linkage.

Patch by Peter Collingbourne.

Reviewers: rafael, tejohnson, mehdi_amini

Subscribers: lhames, tejohnson, mehdi_amini, llvm-commits

Differential Revision: https://reviews.llvm.org/D20268

Address review comments

llvm-svn: 278330
2016-08-11 12:56:40 +00:00
Peter Collingbourne
b54bee8ab7 Move legacy LTO interface headers to legacy/ directory.
Differential Revision: https://reviews.llvm.org/D22173

llvm-svn: 275476
2016-07-14 21:21:16 +00:00
Peter Collingbourne
0442bd0563 ThinLTO: Do not take into account whether a definition has multiple copies when promoting.
We currently do not touch a symbol's linkage in the case where a definition
has a single copy. However, this code is effectively unnecessary: either
the definition is not exported, in which case the internalize phase sets
its linkage to internal, or it is exported, in which case we need to promote
linkage to weak. Those two cases are already handled by existing code.

I believe that the only real functional change here is in the case where we
have a single definition which does not prevail (e.g. because the definition
in a native object file prevails). In that case we now lower linkage to
available_externally following the existing code path for that case.

As a result we can remove the isExported function parameter from the
thinLTOResolveWeakForLinkerInIndex function.

Differential Revision: http://reviews.llvm.org/D21883

llvm-svn: 274784
2016-07-07 18:31:51 +00:00
Davide Italiano
e42bdc124e [UpdateCompilerUsed] API rename and cleanup, suggested by Rafaael.
* UpdateCompilerUsed() -> updateCompilerUsed()
* ThinLTO doesn't use the API so we can remove the include
* Clean up unused #include <functional> from the header
* Rename #ifdef guard comment to be correct.

llvm-svn: 273461
2016-06-22 19:50:42 +00:00
Davide Italiano
bdd047956c [LTO] Move UpdateCompilerUsed.h from lib/ to include/
I plan to use it in lld soon.

Differential Revision:  http://reviews.llvm.org/D21575

llvm-svn: 273380
2016-06-22 04:52:43 +00:00
Benjamin Kramer
9809f36d76 Move instances of std::function.
Or replace with llvm::function_ref if it's never stored. NFC intended.

llvm-svn: 272513
2016-06-12 16:13:55 +00:00
Benjamin Kramer
d415569b3b Apply most suggestions of clang-tidy's performance-unnecessary-value-param
Avoids unnecessary copies. All changes audited & pass tests with asan.
No functional change intended.

llvm-svn: 272190
2016-06-08 19:09:22 +00:00
Teresa Johnson
655b4d9f20 [ThinLTO] Refactor ODR resolution and internalization (NFC)
Move the now index-based ODR resolution and internalization routines out
of ThinLTOCodeGenerator.cpp and into either LTO.cpp (index-based
analysis) or FunctionImport.cpp (index-driven optimizations).
This is to enable usage by other linkers.

llvm-svn: 270698
2016-05-25 14:03:11 +00:00
Teresa Johnson
5cc2580600 [ThinLTO] Change ODR resolution and internalization to be index-based
Summary:
This patch changes the ODR resolution and internalization to be based on
updates to the Index, which are consumed by the backend portion of the
transformations.

It will be followed by an NFC change to move these out of libLTO's
ThinLTOCodeGenerator so that it can be used by other linkers
(gold and lld) and by ThinLTO distributed backends.

The global summary-based portions use callbacks so that the client can
determine the prevailing copy and other information in a client-specific
way. Eventually, with the API being developed in D20268, these may be
modified to use information such as symbol resolutions, supplied by the
clients to the API.

Reviewers: joker-eph

Subscribers: joker.eph, pcc, llvm-commits

Differential Revision: http://reviews.llvm.org/D20290

llvm-svn: 270584
2016-05-24 17:24:25 +00:00
Teresa Johnson
9184fab6a9 [ThinLTO] Refactor module loader handling into new LTO file (NFC)
Moved the ModuleLoader and supporting helper loadModuleFromBuffer out of
ThinLTOCodeGenerator and into new LTO.h/LTO.cpp files. This is in
preparation for a patch that will utilize these in the gold-plugin.

Note that there are some other pending patches (D20268 and D20290) that
also plan to refactor common interfaces and functionality into this same
pair of new files.

llvm-svn: 270509
2016-05-23 22:54:06 +00:00
Rafael Espindola
22e87bbb08 Delete Reloc::Default.
Having an enum member named Default is quite confusing: Is it distinct
from the others?

This patch removes that member and instead uses Optional<Reloc> in
places where we have a user input that still hasn't been maped to the
default value, which is now clear has no be one of the remaining 3
options.

llvm-svn: 269988
2016-05-18 22:04:49 +00:00
Rafael Espindola
32a5521c87 White space cleanup.
llvm-svn: 269919
2016-05-18 13:10:27 +00:00
Mehdi Amini
3be11cb6c5 Remove unwanted fprintf debug
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 269536
2016-05-14 04:41:14 +00:00
Teresa Johnson
e31605427f [ThinLTO] Add option to emit imports files for distributed backends
Summary:
Add support for emission of plaintext lists of the imported files for
each distributed backend compilation. Used for distributed build file
staging.

Invoked with new gold-plugin thinlto-emit-imports-files option, which is
only valid with thinlto-index-only (i.e. for distributed builds), or
from llvm-lto with new -thinlto-action=emitimports value.

Depends on D19556.

Reviewers: joker.eph

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D19636

llvm-svn: 269067
2016-05-10 15:54:09 +00:00
Teresa Johnson
4147326797 Restore "[ThinLTO] Emit individual index files for distributed backends"
This restores commit r268627:
    Summary:
    When launching ThinLTO backends in a distributed build (currently
    supported in gold via the thinlto-index-only plugin option), emit
    an individual index file for each backend process as described here:
    http://lists.llvm.org/pipermail/llvm-dev/2016-April/098272.html

    ...

    Differential Revision: http://reviews.llvm.org/D19556

Address msan failures by avoiding std::prev on map.end(), the
theory is that this is causing issues due to some known UB problems
in __tree.

llvm-svn: 269059
2016-05-10 13:48:23 +00:00
Mehdi Amini
5bcf351ca9 ThinLTOCodeGenerator: ignore 0 values for the cache settings.
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 268890
2016-05-09 05:16:30 +00:00
Mehdi Amini
193aba341e Degrade assertions to a warning in LTOCodeGenerator for preserved linkonce
The assertions were assuming that the linker will not ask to preserve
a global that is internal or available_externally, as it does not
really make sense. In practice this break the bootstrap of clang,
I degrade to a warning for now.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 268671
2016-05-05 20:05:33 +00:00
Vitaly Buka
a10eb05536 Revert "[ThinLTO] Emit individual index files for distributed backends"
MemorySanitizer: use-of-uninitialized-value in lib/Bitcode/Writer/BitcodeWriter.cpp:364:70
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/12544/steps/check-llvm%20msan/logs/stdio

This reverts commit 0c4a898ea550699d1b2f4fe3767251c8f9a48d52.

llvm-svn: 268660
2016-05-05 18:31:00 +00:00
Teresa Johnson
960ebc706f [ThinLTO] Emit individual index files for distributed backends
Summary:
When launching ThinLTO backends in a distributed build (currently
supported in gold via the thinlto-index-only plugin option), emit
an individual index file for each backend process as described here:
http://lists.llvm.org/pipermail/llvm-dev/2016-April/098272.html

The individual index file encodes the summary and module information
required for implementing the importing/exporting decisions made
for a given module in the thin link step.
This is in place of the current mechanism that uses the combined index
to make importing decisions in each back end independently. It is an
enabler for doing global summary based optimizations in the thin link
step (which will be recorded in the individual index files), and reduces
the size of the index that must be sent to each backend process, and
the amount of work to scan it in the backends.

Rather than create entirely new ModuleSummaryIndex structures (and all
the included unique_ptrs) for each backend index file, a map is created
to record all of the GUID and summary pointers needed for a particular
index file. The IndexBitcodeWriter walks this map instead of the full
index (hiding the details of managing the appropriate summary iteration
in a new iterator subclass). This is more efficient than walking the
entire combined index and filtering out just the needed summaries during
each backend bitcode index write.

Depends on D19481.

Reviewers: joker.eph

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D19556

llvm-svn: 268627
2016-05-05 13:44:56 +00:00
Mehdi Amini
08040b380e Provide some default values for the ThinLTO Cache pruning
This control how the cache is pruned. The cache still has to
be explicitely enabled/disabled by providing a path.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 268393
2016-05-03 15:17:50 +00:00
Mehdi Amini
cc7d938331 Revert "Support "preserving" the summary information when using setModule() API in LTOCodeGenerator"
This reverts commit r267665.
ASAN shows that there is a use of undefined value.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267668
2016-04-27 05:11:44 +00:00
Mehdi Amini
6e6426ce69 Support "preserving" the summary information when using setModule() API in LTOCodeGenerator
Another attempt at r267655...

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267665
2016-04-27 04:24:10 +00:00
Mehdi Amini
4c94cd20e1 Revert "Support "preserving" the summary information when using setModule() API in LTOCodeGenerator"
This reverts commit r267657, r267656, and r267655.
The test does not pass on multiple bots, I'm unsure why yet but let's unbreak them.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267664
2016-04-27 03:34:28 +00:00
Mehdi Amini
a75f9ab43e Support "preserving" the summary information when using setModule() API in LTOCodeGenerator
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267655
2016-04-27 01:46:48 +00:00
Mehdi Amini
c68b6482c1 Add an internalization step to the ThinLTOCodeGenerator
Keeping as much as possible internal/private is
known to help the optimizer. Let's try to benefit from
this in ThinLTO.
Note: this is early work, but is enough to build clang (and
all the LLVM tools). I still need to write some lit-tests...

Differential Revision: http://reviews.llvm.org/D19103

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267317
2016-04-24 03:18:01 +00:00
Mehdi Amini
3078624124 ThinLTO: initialize variables
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266964
2016-04-21 06:43:41 +00:00
Duncan P. N. Exon Smith
f163b980cd LTO: Verify the input even if optimize() isn't called
Clients may call writeMergedModules before calling optimize, or call
compileOptimized without calling optimize.  Make sure they don't sneak
past the verifier.  This adds LTOCodeGenerator::verifyMergedModuleOnce,
and calls it from writeMergedModule, optimize, and codegenOptimized.

I couldn't find a good way to test this.  I tried writing broken IR to
send into llvm-lto, but LTOCodeGenerator doesn't understand textual IR,
and assembler runs the verifier itself anyway.  Checking in
valid-but-doesn't-verify bitcode here doesn't seem valuable.

llvm-svn: 266894
2016-04-20 17:48:22 +00:00
Mehdi Amini
9ff867f98c [NFC] Header cleanup
Removed some unused headers, replaced some headers with forward class declarations.

Found using simple scripts like this one:
clear && ack --cpp -l '#include "llvm/ADT/IndexedMap.h"' | xargs grep -L 'IndexedMap[<]' | xargs grep -n --color=auto 'IndexedMap'

Patch by Eugene Kosov <claprix@yandex.ru>

Differential Revision: http://reviews.llvm.org/D19219

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266595
2016-04-18 09:17:29 +00:00
Rafael Espindola
e2d87a8c44 Keep only the splitCodegen version that takes a factory.
This makes it much easier to see that all created TargetMachines are
equivalent.

llvm-svn: 266564
2016-04-17 18:42:27 +00:00
Mehdi Amini
f1caa14180 Refactor the Internalize stage of libLTO in a separate file (NFC)
This is intended to be shared by the ThinLTOCodeGenerator.

Note that there is a change in the way the verifier is run, previously
it was ran as a Pass on the merged module during internalization.
While now the verifier is called explicitely on the merged module
outside of the internalize "pass pipeline".

What remains strange in the API is the fact that `DisableVerify` in
the API does not disable this initial verifier.

Differential Revision: http://reviews.llvm.org/D19000

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266047
2016-04-12 06:34:10 +00:00
Mehdi Amini
97f744e831 Use StringSet instead of StringMap where it makes sense to in LTOCodeGenerator (NFC)
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266046
2016-04-12 06:33:44 +00:00
Benjamin Kramer
7cb557e97a [ThinLTO] Fix uninitialized flags.
Found by msan. Patch by Adrian Kuegel!

llvm-svn: 265133
2016-04-01 11:49:59 +00:00
Mehdi Amini
a7205432cb Add a libLTO API to stop/restart ThinLTO between optimizations and CodeGen
This allows the linker to instruct ThinLTO to perform only the
optimization part or only the codegen part of the process.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265113
2016-04-01 06:47:02 +00:00
Teresa Johnson
89e5e1dadb [ThinLTO] Renaming of function index to module summary index (NFC)
(Resubmitting after fixing missing file issue)

With the changes in r263275, there are now more than just functions in
the summary. Completed the renaming of data structures (started in
r263275) to reflect the wider scope. In particular, changed the
FunctionIndex* data structures to ModuleIndex*, and renamed related
variables and comments. Also renamed the files to reflect the changes.

A companion clang patch will immediately succeed this patch to reflect
this renaming.

llvm-svn: 263513
2016-03-15 00:04:37 +00:00
Teresa Johnson
6632706d71 Revert "[ThinLTO] Renaming of function index to module summary index (NFC)"
This reverts commit r263490. Missed a file.

llvm-svn: 263493
2016-03-14 21:18:10 +00:00
Teresa Johnson
7f5c7bac22 [ThinLTO] Renaming of function index to module summary index (NFC)
With the changes in r263275, there are now more than just functions in
the summary. Completed the renaming of data structures (started in
r263275) to reflect the wider scope. In particular, changed the
FunctionIndex* data structures to ModuleIndex*, and renamed related
variables and comments. Also renamed the files to reflect the changes.

A companion clang patch will immediately succeed this patch to reflect
this renaming.

llvm-svn: 263490
2016-03-14 21:05:56 +00:00
Mehdi Amini
ea71be934f libLTO: add a ThinLTOCodeGenerator on the model of LTOCodeGenerator.
This is intended to provide a parallel (threaded) ThinLTO scheme
for linker plugin use through the libLTO C API.

The intent of this patch is to provide a first implementation as a
proof-of-concept and allows linker to start supporting ThinLTO by
definiing the libLTO C API. Some part of the libLTO API are left
unimplemented yet. Following patches will add support for these.

The current implementation can link all clang/llvm binaries.

Differential Revision: http://reviews.llvm.org/D17066

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 262977
2016-03-09 01:37:22 +00:00
Petr Pavlu
324853823b [LTO] Fix error reporting from lto_module_create_in_local_context()
Function lto_module_create_in_local_context() would previously
rely on the default LLVMContext being created for it by
LTOModule::makeLTOModule(). This context exits the program on
error and is not arranged to update sLastStringError in
tools/lto/lto.cpp.

Function lto_module_create_in_local_context() now creates an
LLVMContext by itself, sets it up correctly to its needs and then
passes it to LTOModule::createInLocalContext() which takes
ownership of the context and keeps it present for the lifetime of
the returned LTOModule.

Function LTOModule::makeLTOModule() is modified to take a
reference to LLVMContext (instead of a pointer) and no longer
creates a default context when nullptr is passed to it. Method
LTOModule::createInContext() that takes a pointer to LLVMContext
is removed because it allows to pass a nullptr to it. Instead
LTOModule::createFromBuffer() (that takes a reference to
LLVMContext) should be used.

Differential Revision: http://reviews.llvm.org/D17715

llvm-svn: 262330
2016-03-01 13:13:49 +00:00
Reid Kleckner
37e33df9b2 Remove unused MC includes from LTOModule.h
llvm-svn: 259115
2016-01-28 23:21:12 +00:00
Benjamin Kramer
2d2ebf092c Make header self-contained.
llvm-svn: 259060
2016-01-28 17:48:29 +00:00
Tobias Edler von Koch
c19c96e06f [LTO] Restore original linkage of externals prior to splitting
Summary:
This is a companion patch for http://reviews.llvm.org/D16124.

Internalized symbols increase the size of strongly-connected components in
SCC-based module splitting and thus reduce the amount of parallelism. This
patch records the original linkage of non-local symbols prior to
internalization and then restores it just before splitting/CodeGen. This is
also useful for cases where the linker requires symbols to remain external, for
instance, so they can be placed according to linker script rules.

It's currently under its own flag (-restore-globals) but should eventually
share a common flag with D16124.

Reviewers: joker.eph, pcc

Subscribers: slarin, llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D16229

llvm-svn: 258100
2016-01-18 23:24:54 +00:00
Teresa Johnson
d80a73c588 Rename variables to reflect linker split (NFC)
Renamed variables to be more reflective of whether they are
an instance of Linker, IRLinker or ModuleLinker. Also fix a stale
comment.

llvm-svn: 256011
2015-12-18 19:28:59 +00:00
Steven Wu
1b8163c1b8 Fix the order of destructors in LibLTOCodeGenerator
Summary:
The order of destructors in LTOCodeGenerator gets changed in r254696.
It is possible for LTOCodeGenerator to have a MergedModule created in
the OwnedContext, in which case the module must be destructed before
the context.

Reviewers: rafael, dexonsmith

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D15346

llvm-svn: 255092
2015-12-09 03:37:51 +00:00
Rafael Espindola
b4f7892e3a Modernize the C++ APIs for creating LTO modules.
This is a continuation of r253367.

These functions return is owned by the caller, so they return
std::unique_ptr now.

The call can fail, so the return is wrapped in ErrorOr.

They have a context where to report diagnostics, so they don't need to
take a string out parameter.

With this there are no call to getGlobalContext in lib/LTO.

llvm-svn: 254721
2015-12-04 16:14:31 +00:00
Rafael Espindola
b0476044d6 Move a call to getGlobalContext out of lib/LTO.
llvm-svn: 254696
2015-12-04 02:42:28 +00:00
Rafael Espindola
116a2b8530 Simplify since this function never fails.
llvm-svn: 254667
2015-12-03 23:56:42 +00:00
Rafael Espindola
49b4ffd0b8 Delete the setModule method from the Linker.
It was only used from LTO for a debug feature, and LTO can just create
another linker.

It is pretty odd to have a method to reset the module in the middle of a
link. It would make IdentifiedStructTypes inconsistent with the Module
for example.

llvm-svn: 254434
2015-12-01 18:41:30 +00:00
Tobias Edler von Koch
0ee12e019e [LTO] Add option to emit assembly from LTOCodeGenerator
This adds a new API, LTOCodeGenerator::setFileType, to choose the output file
format for LTO CodeGen. A corresponding change to use this new API from
llvm-lto and a test case is coming in a separate commit.

Differential Revision: http://reviews.llvm.org/D14554

llvm-svn: 253622
2015-11-19 23:59:24 +00:00
Yunzhong Gao
b3793fa262 Switch lto codegen to using diagnostic handlers.
This patch removes the std::string& argument from a number of C++ LTO API calls
and instead makes them use the installed diagnostic handler. This would also
improve consistency of diagnostic handling infrastructure: if an LTO client used
lto_codegen_set_diagnostic_handler() to install a custom error handler, we do
not want some error messages to go through the custom error handler, and some
other error messages to go into sLastErrorString.

llvm-svn: 253367
2015-11-17 19:48:12 +00:00
Mehdi Amini
f559a9a15a Add a method to the BitcodeReader to parse only the identification block
Summary: Mimic parseTriple(); and exposes it to LTOModule.cpp

Reviewers: dexonsmith, rafael

Subscribers: llvm-commits

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 252442
2015-11-09 02:46:41 +00:00
Duncan P. N. Exon Smith
a136fd65ad Reapply "LTO: Disable extra verify runs in release builds"
This reverts commit r247730, effectively reapplying r247729.  This time
I have an lld commit ready to follow.

llvm-svn: 247735
2015-09-15 23:05:59 +00:00
Duncan P. N. Exon Smith
af74674cca Revert "LTO: Disable extra verify runs in release builds"
This temporarily reverts commit r247729, as it caused lld build
failures.  I'll recommit once I have an lld patch ready-to-go.

llvm-svn: 247730
2015-09-15 22:47:38 +00:00
Duncan P. N. Exon Smith
c2011f0a8c LTO: Disable extra verify runs in release builds
The verifier currently runs three times in LTO: (1) after parsing, (2)
at the beginning of the optimization pipeline, and (3) at the end of it.

The first run is important, since we're not sure where the bitcode comes
from and it's nice to validate it, but in release builds the extra runs
aren't appropriate.

This commit:
  - Allows these runs to be disabled in LTOCodeGenerator.
  - Adds command-line options to llvm-lto.
  - Adds command-line options to libLTO.dylib, and disables the verifier
    by default in release builds (based on NDEBUG).

This shaves about 3.5% off the runtime of ld64 when linking
verify-uselistorder with -flto -g.

rdar://22509081

llvm-svn: 247729
2015-09-15 22:26:11 +00:00
Duncan P. N. Exon Smith
54c273cc35 LTO: Cleanup parameter names and header docs, NFC
Follow LLVM style for the parameter names (`CamelCase` not `camelCase`),
and surface the header docs in doxygen.  No functionality change
intended.

llvm-svn: 246509
2015-08-31 23:44:06 +00:00
Peter Collingbourne
b346721555 CodeGen: Introduce splitCodeGen and teach LTOCodeGenerator to use it.
llvm::splitCodeGen is a function that implements the core of parallel LTO
code generation. It uses llvm::SplitModule to split the module into linkable
partitions and spawning one code generation thread per partition. The function
produces multiple object files which can be linked in the usual way.

This has been threaded through to LTOCodeGenerator (and llvm-lto for testing
purposes). Separate patches will add parallel LTO support to the gold plugin
and lld.

Differential Revision: http://reviews.llvm.org/D12260

llvm-svn: 246236
2015-08-27 23:37:36 +00:00
Peter Collingbourne
1d5cde4afc LTO: Simplify merged module ownership.
This change moves LTOCodeGenerator's ownership of the merged module to a
field of type std::unique_ptr<Module>. This helps simplify parts of the code
and clears the way for the module to be consumed by LLVM CodeGen (see D12132
review comments).

Differential Revision: http://reviews.llvm.org/D12205

llvm-svn: 245891
2015-08-24 22:22:53 +00:00
Peter Collingbourne
772d0abe3e LTO: Maintain target triple, FeatureStr and CGOptLevel in the module or LTOCodeGenerator.
This makes it easier to create new TargetMachines on demand.

llvm-svn: 245781
2015-08-22 02:25:53 +00:00
Peter Collingbourne
db573c5328 LTO: Change signature of LTOCodeGenerator::setCodePICModel() to take a Reloc::Model.
This allows us to remove a bunch of code in LTOCodeGenerator and llvm-lto
and has the side effect of improving error handling in the libLTO C API.

llvm-svn: 245756
2015-08-21 22:57:17 +00:00
Peter Collingbourne
e7628d5684 LTO: Simplify ownership of LTOCodeGenerator::TargetMach.
llvm-svn: 245671
2015-08-21 04:45:57 +00:00
Peter Collingbourne
1a4134b22f LTO: Simplify ownership of LTOCodeGenerator::CodegenOptions.
llvm-svn: 245670
2015-08-21 04:45:55 +00:00
Peter Collingbourne
d3c303721f Teach LTOModule to emit linker flags for dllexported symbols, plus interface cleanup.
This change unifies how LTOModule and the backend obtain linker flags
for globals: via a new TargetLoweringObjectFile member function named
emitLinkerFlagsForGlobal. A new function LTOModule::getLinkerOpts() returns
the list of linker flags as a single concatenated string.

This change affects the C libLTO API: the function lto_module_get_*deplibs now
exposes an empty list, and lto_module_get_*linkeropts exposes a single element
which combines the contents of all observed flags. libLTO should never have
tried to parse the linker flags; it is the linker's job to do so. Because
linkers will need to be able to parse flags in regular object files, it
makes little sense for libLTO to have a redundant mechanism for doing so.

The new API is compatible with the old one. It is valid for a user to specify
multiple linker flags in a single pragma directive like this:

 #pragma comment(linker, "/defaultlib:foo /defaultlib:bar")

The previous implementation would not have exposed
either flag via lto_module_get_*deplibs (as the test in
TargetLoweringObjectFileCOFF::getDepLibFromLinkerOpt was case sensitive)
and would have exposed "/defaultlib:foo /defaultlib:bar" as a single flag via
lto_module_get_*linkeropts. This may have been a bug in the implementation,
but it does give us a chance to fix the interface.

Differential Revision: http://reviews.llvm.org/D10548

llvm-svn: 241010
2015-06-29 22:04:09 +00:00
Alexander Kornienko
f993659b8f Revert r240137 (Fixed/added namespace ending comments using clang-tidy. NFC)
Apparently, the style needs to be agreed upon first.

llvm-svn: 240390
2015-06-23 09:49:53 +00:00
Alexander Kornienko
40cb19d802 Fixed/added namespace ending comments using clang-tidy. NFC
The patch is generated using this command:

tools/clang/tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \
  -checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \
  llvm/lib/


Thanks to Eugene Kosov for the original patch!

llvm-svn: 240137
2015-06-19 15:57:42 +00:00
Peter Collingbourne
554dcad83e LTO: Introduce LTOModule::getSymbolGV().
llvm-svn: 239993
2015-06-18 05:10:06 +00:00
Peter Collingbourne
7f6337343d Make the C++ LTO API easier to use from C++ clients.
Start using C++ types such as StringRef and MemoryBuffer in the C++ LTO
API. In doing so, clarify the ownership of the native object file: the caller
now owns it, not the LTOCodeGenerator. The C libLTO library has been modified
to use a derived class of LTOCodeGenerator that owns the object file.

Differential Revision: http://reviews.llvm.org/D10114

llvm-svn: 238776
2015-06-01 20:08:30 +00:00
Duncan P. N. Exon Smith
697f734b82 LTO: Add API to choose whether to embed uselists
Reverse libLTO's default behaviour for preserving use-list order in
bitcode, and add API for controlling it.  The default setting is now
`false` (don't preserve them), which is consistent with `clang`'s
default behaviour.

Users of libLTO should call `lto_codegen_should_embed_uselists(CG,true)`
prior to calling `lto_codegen_write_merged_modules()` whenever the
output file isn't part of the production workflow in order to reproduce
results with subsequent calls to `llc`.

(I haven't added tests since `llvm-lto` (the test tool for LTO) doesn't
support bitcode output, and even if it did: there isn't actually a good
way to test whether a tool has passed the flag.  If the order is already
"natural" (if the order will already round-trip) then no use-list
directives are emitted at all.  At some point I'll circle back to add
tests to `llvm-as` (etc.) that they actually respect the flag, at which
point I can somehow add a test here as well.)

llvm-svn: 235943
2015-04-27 23:38:54 +00:00
Duncan P. N. Exon Smith
a9647df35e LTO: Simplify code generator initialization
Simplify `LTOCodeGenerator` initialization by initializing simple fields
at their definition.

llvm-svn: 235939
2015-04-27 23:19:26 +00:00
Manman Ren
b67a75becd [LTO API] add lto_codegen_set_should_internalize.
When debugging LTO issues with ld64, we use -save-temps to save the merged
optimized bitcode file, then invoke ld64 again on the single bitcode file.
The saved bitcode file is already internalized, so we can call
lto_codegen_set_should_internalize and skip running internalization again.

rdar://20227235

llvm-svn: 235211
2015-04-17 17:10:09 +00:00
Rafael Espindola
aeb03deb16 Use raw_pwrite_stream in the object writer/streamer.
The ELF object writer will take advantage of that in the next commit.

llvm-svn: 234950
2015-04-14 22:14:34 +00:00
Peter Collingbourne
3bdbf14413 libLTO, llvm-lto, gold: Introduce flag for controlling optimization level.
This change also introduces a link-time optimization level of 1. This
optimization level runs only the globaldce pass as well as cleanup passes for
passes that run at -O0, specifically simplifycfg which cleans up lowerbitsets.

http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20150316/266951.html

llvm-svn: 232769
2015-03-19 22:01:00 +00:00
Manman Ren
0fc198ef3f [LTO API] fix memory leakage introduced at r230290.
r230290 released the LLVM module but not the LTOModule.

rdar://19024554

llvm-svn: 230544
2015-02-25 21:20:53 +00:00
Manman Ren
87a073ae1a [LTO API] add lto_codegen_set_module to set the destination module.
When debugging LTO issues with ld64, we use -save-temps to save the merged
optimized bitcode file, then invoke ld64 again on the single bitcode file to
speed up debugging code generation passes and ld64 stuff after code generation.

llvm linking a single bitcode file via lto_codegen_add_module will generate a
different bitcode file from the single input. With the newly-added
lto_codegen_set_module, we can make sure the destination module is the same as
the input.

lto_codegen_set_module will transfer the ownship of the module to code
generator.

rdar://19024554

llvm-svn: 230290
2015-02-24 00:45:56 +00:00
Manman Ren
ecc02c6b0a [LTO API] split lto_codegen_compile to lto_codegen_optimize and
lto_codegen_compile_optimized. Also add lto_api_version.

Before this commit, we can only dump the optimized bitcode after running
lto_codegen_compile, but it includes some impacts of running codegen passes,
one example is StackProtector pass. We will get assertion failure when running
llc on the optimized bitcode, because StackProtector is effectively run twice.

After splitting lto_codegen_compile, the linker can choose to dump the bitcode
before running lto_codegen_compile_optimized.

lto_api_version is added so ld64 can check for runtime-availability of the new
API.

rdar://19565500

llvm-svn: 228000
2015-02-03 18:39:15 +00:00
David Blaikie
7499cbae4c Remove StringMap::GetOrCreateValue in favor of StringMap::insert
Having two ways to do this doesn't seem terribly helpful and
consistently using the insert version (which we already has) seems like
it'll make the code easier to understand to anyone working with standard
data structures. (I also updated many references to the Entry's
key and value to use first() and second instead of getKey{Data,Length,}
and get/setValue - for similar consistency)

Also removes the GetOrCreateValue functions so there's less surface area
to StringMap to fix/improve/change/accommodate move semantics, etc.

llvm-svn: 222319
2014-11-19 05:49:42 +00:00
Duncan P. N. Exon Smith
906524d748 libLTO: Allow LTOModule to own a context
llvm-svn: 221728
2014-11-11 23:08:05 +00:00
Duncan P. N. Exon Smith
7a6397a4f1 libLTO: Allow LTOCodeGenerator to own a context
llvm-svn: 221726
2014-11-11 23:03:29 +00:00
Arnold Schwaighofer
574ccf8f46 Add an option to the LTO code generator to disable vectorization during LTO
We used to always vectorize (slp and loop vectorize) in the LTO pass pipeline.

r220345 changed it so that we used the PassManager's fields 'LoopVectorize' and
'SLPVectorize' out of the desire to be able to disable vectorization using the
cl::opt flags 'vectorize-loops'/'slp-vectorize' which the before mentioned
fields default to.
Unfortunately, this turns off vectorization because those fields
default to false.
This commit adds flags to the LTO library to disable lto vectorization which
reconciles the desire to optionally disable vectorization during LTO and
the desired behavior of defaulting to enabled vectorization.

We really want tools to set PassManager flags directly to enable/disable
vectorization and not go the route via cl::opt flags *in*
PassManagerBuilder.cpp.

llvm-svn: 220652
2014-10-26 21:50:58 +00:00
Rafael Espindola
67b4b608c2 Update the error handling of lib/Linker.
Instead of passing a std::string&, use the new diagnostic infrastructure.

llvm-svn: 220608
2014-10-25 04:06:10 +00:00
David Blaikie
c90d424c3a unique_ptrify LTOCodeGenerator::NativeObjectFile
llvm-svn: 216927
2014-09-02 18:21:06 +00:00
Craig Topper
d133367f2d Fix some cases were ArrayRefs were being passed by reference. Also remove 'const' from some other ArrayRef uses since its implicitly const already.
llvm-svn: 216524
2014-08-27 05:25:00 +00:00
Craig Topper
65775cc03d Repace SmallPtrSet with SmallPtrSetImpl in function arguments to avoid needing to mention the size.
llvm-svn: 216158
2014-08-21 05:55:13 +00:00
Rafael Espindola
08aa78de63 Don't own the buffer in object::Binary.
Owning the buffer is somewhat inflexible. Some Binaries have sub Binaries
(like Archive) and we had to create dummy buffers just to handle that. It is
also a bad fit for IRObjectFile where the Module wants to own the buffer too.

Keeping this ownership would make supporting IR inside native objects
particularly painful.

This patch focuses in lib/Object. If something elsewhere used to own an Binary,
now it also owns a MemoryBuffer.

This patch introduces a few new types.

* MemoryBufferRef. This is just a pair of StringRefs for the data and name.
  This is to MemoryBuffer as StringRef is to std::string.
* OwningBinary. A combination of Binary and a MemoryBuffer. This is needed
  for convenience functions that take a filename and return both the
  buffer and the Binary using that buffer.

The C api now uses OwningBinary to avoid any change in semantics. I will start
a new thread to see if we want to change it and how.

llvm-svn: 216002
2014-08-19 18:44:46 +00:00
Craig Topper
aa7422b5a6 Revert "Repace SmallPtrSet with SmallPtrSetImpl in function arguments to avoid needing to mention the size."
Getting a weird buildbot failure that I need to investigate.

llvm-svn: 215870
2014-08-18 00:24:38 +00:00
Craig Topper
227456e133 Repace SmallPtrSet with SmallPtrSetImpl in function arguments to avoid needing to mention the size.
llvm-svn: 215868
2014-08-17 23:47:00 +00:00
Rafael Espindola
4dd4e8c9ed Return a std::uinque_ptr. Every caller was already using one.
llvm-svn: 215858
2014-08-17 22:37:39 +00:00
Benjamin Kramer
da144ed5a2 Canonicalize header guards into a common format.
Add header guards to files that were missing guards. Remove #endif comments
as they don't seem common in LLVM (we can easily add them back if we decide
they're useful)

Changes made by clang-tidy with minor tweaks.

llvm-svn: 215558
2014-08-13 16:26:38 +00:00
Rafael Espindola
d0cd4f5275 Implement LTOModule on top of IRObjectFile.
IRObjectFile provides all the logic for producing mangled names and getting
symbols from inline assembly.

LTOModule then adds logic for linking specific tasks, like constructing
llvm.compiler_user or extracting linker options from the bitcode.

The rule of the thumb is that IRObjectFile has the functionality that is
needed by both LTO and llvm-ar.

llvm-svn: 212349
2014-07-04 18:40:36 +00:00
Rafael Espindola
75feeb47c0 Avoid mangling names twice. No functionality change.
llvm-svn: 212348
2014-07-04 16:37:02 +00:00
Rafael Espindola
0f593fe8ae Change LTOModule`s getTargetTriple and setTargetTriple to use c++ types.
llvm-svn: 212343
2014-07-04 14:19:41 +00:00
Alp Toker
a27c4dc68d Sink undesirable LTO functions into the old C API
We want to encourage users of the C++ LTO API to reuse memory buffers instead
of repeatedly opening and reading the same file contents.

This reverts commit r212305 and implements a tidier scheme.

llvm-svn: 212308
2014-07-04 00:58:41 +00:00
Peter Collingbourne
81fd228ab6 Modify LTOModule::isTargetMatch to take a StringRef instead of a MemoryBuffer.
llvm-svn: 212305
2014-07-03 23:49:28 +00:00
Peter Collingbourne
83a67bf953 LTO: rename the various makeLTOModule overloads.
This rename makes it easier to identify the specific overload being called
in each particular case and makes future refactorings easier.

Differential Revision: http://reviews.llvm.org/D4370

llvm-svn: 212302
2014-07-03 23:28:00 +00:00
Rafael Espindola
39ed527ccb Use std::unique_ptr to manage memory. No functionality change.
llvm-svn: 212299
2014-07-03 22:43:03 +00:00
Rafael Espindola
f3c045d9ca Rename member variable to try to fix the bots.
llvm-svn: 207915
2014-05-03 15:28:13 +00:00
Rafael Espindola
6884239aa9 Move LTOModule and LTOCodeGenerator to the llvm namespace.
llvm-svn: 207911
2014-05-03 14:59:52 +00:00
Rafael Espindola
dc579d7111 Style fix: don't duplicate the method names.
llvm-svn: 207910
2014-05-03 14:46:47 +00:00
Tom Roeder
a8ef44c3ed Add an -mattr option to the gold plugin to support subtarget features in LTO
This adds support for an -mattr option to the gold plugin and to llvm-lto. This
allows the caller to specify details of the subtarget architecture, like +aes,
or +ssse3 on x86.  Note that this requires a change to the include/llvm-c/lto.h
interface: it adds a function lto_codegen_set_attr and it increments the
version of the interface.

llvm-svn: 207279
2014-04-25 21:46:51 +00:00
Craig Topper
c2260fc0ab [C++11] More 'nullptr' conversion. In some cases just using a boolean check instead of comparing to nullptr.
llvm-svn: 206252
2014-04-15 06:32:26 +00:00
Duncan P. N. Exon Smith
7f2af7c18b Revert "Reapply "LTO: add API to set strategy for -internalize""
This reverts commit r199244.

Conflicts:
	include/llvm-c/lto.h
	include/llvm/LTO/LTOCodeGenerator.h
	lib/LTO/LTOCodeGenerator.cpp

llvm-svn: 205471
2014-04-02 22:05:57 +00:00
Ahmed Charles
52ce0c101e Replace OwningPtr<T> with std::unique_ptr<T>.
This compiles with no changes to clang/lld/lldb with MSVC and includes
overloads to various functions which are used by those projects and llvm
which have OwningPtr's as parameters. This should allow out of tree
projects some time to move. There are also no changes to libs/Target,
which should help out of tree targets have time to move, if necessary.

llvm-svn: 203083
2014-03-06 05:51:42 +00:00
Chandler Carruth
a799485f06 [Layering] Sink Linker.h into a Linker subdirectory to make it
consistent with every other sub-library header in LLVM.

llvm-svn: 203065
2014-03-06 03:42:23 +00:00
Manman Ren
5a1cf3b255 LTO API: add lto_module_create_from_memory_with_path.
This function adds an extra path argument to lto_module_create_from_memory.
The path argument will be passed to makeBuffer to make sure the MemoryBuffer
has a name and the created module has a module identifier.

This is mainly for emitting warning messages from the linker. When we emit
warning message on a module, we can use the module identifier.

rdar://15985737

llvm-svn: 201114
2014-02-10 23:26:14 +00:00
Yunzhong Gao
12926dc854 Adding new LTO APIs to parse metadata nodes and extract linker options and
dependent libraries from a bitcode module.

Differential Revision: http://llvm-reviews.chandlerc.com/D2343

llvm-svn: 199759
2014-01-21 18:31:27 +00:00
Quentin Colombet
fa288f1121 [LTO] Add a hook to map LLVM diagnostics into the clients of LTO.
Add a hook in the C API of LTO so that clients of the code generator can set
their own handler for the LLVM diagnostics.
The handler is defined like this:
typedef void (*lto_diagnostic_handler_t)(lto_codegen_diagnostic_severity_t
severity, const char *diag, void *ctxt)
- severity says how bad this is.
- diag is a string that contains the diagnostic message.
- ctxt is the registered context for this handler.

This hook is more general than the lto_get_error_message, since this function
keeps only the latest message and can only be queried when something went wrong
(no warning for instance).

<rdar://problem/15517596>

llvm-svn: 199338
2014-01-15 22:04:35 +00:00
Duncan P. N. Exon Smith
bb847bd59e Reapply "LTO: add API to set strategy for -internalize"
Reapply r199191, reverted in r199197 because it carelessly broke
Other/link-opts.ll.  The problem was that calling
createInternalizePass("main") would select
createInternalizePass(bool("main")) instead of
createInternalizePass(ArrayRef<const char *>("main")).  This commit
fixes the bug.

The original commit message follows.

Add API to LTOCodeGenerator to specify a strategy for the -internalize
pass.

This is a new attempt at Bill's change in r185882, which he reverted in
r188029 due to problems with the gold linker.  This puts the onus on the
linker to decide whether (and what) to internalize.

In particular, running internalize before outputting an object file may
change a 'weak' symbol into an internal one, even though that symbol
could be needed by an external object file --- e.g., with arclite.

This patch enables three strategies:

- LTO_INTERNALIZE_FULL: the default (and the old behaviour).
- LTO_INTERNALIZE_NONE: skip -internalize.
- LTO_INTERNALIZE_HIDDEN: only -internalize symbols with hidden
  visibility.

LTO_INTERNALIZE_FULL should be used when linking an executable.

Outputting an object file (e.g., via ld -r) is more complicated, and
depends on whether hidden symbols should be internalized.  E.g., for
ld -r, LTO_INTERNALIZE_NONE can be used when -keep_private_externs, and
LTO_INTERNALIZE_HIDDEN can be used otherwise.  However,
LTO_INTERNALIZE_FULL is inappropriate, since the output object file will
eventually need to link with others.

lto_codegen_set_internalize_strategy() sets the strategy for subsequent
calls to lto_codegen_write_merged_modules() and lto_codegen_compile*().

<rdar://problem/14334895>

llvm-svn: 199244
2014-01-14 18:52:17 +00:00
NAKAMURA Takumi
068c8352f7 Revert r199191, "LTO: add API to set strategy for -internalize"
Please update also Other/link-opts.ll, in next time.

llvm-svn: 199197
2014-01-14 09:40:18 +00:00
Duncan P. N. Exon Smith
95dadb39e4 LTO: add API to set strategy for -internalize
Add API to LTOCodeGenerator to specify a strategy for the -internalize
pass.

This is a new attempt at Bill's change in r185882, which he reverted in
r188029 due to problems with the gold linker.  This puts the onus on the
linker to decide whether (and what) to internalize.

In particular, running internalize before outputting an object file may
change a 'weak' symbol into an internal one, even though that symbol
could be needed by an external object file --- e.g., with arclite.

This patch enables three strategies:

- LTO_INTERNALIZE_FULL: the default (and the old behaviour).
- LTO_INTERNALIZE_NONE: skip -internalize.
- LTO_INTERNALIZE_HIDDEN: only -internalize symbols with hidden
  visibility.

LTO_INTERNALIZE_FULL should be used when linking an executable.

Outputting an object file (e.g., via ld -r) is more complicated, and
depends on whether hidden symbols should be internalized.  E.g., for
ld -r, LTO_INTERNALIZE_NONE can be used when -keep_private_externs, and
LTO_INTERNALIZE_HIDDEN can be used otherwise.  However,
LTO_INTERNALIZE_FULL is inappropriate, since the output object file will
eventually need to link with others.

lto_codegen_set_internalize_strategy() sets the strategy for subsequent
calls to lto_codegen_write_merged_modules() and lto_codegen_compile*().

<rdar://problem/14334895>

llvm-svn: 199191
2014-01-14 06:37:26 +00:00
Duncan P. N. Exon Smith
3b0b19af25 LTO: whitespace changes
llvm-svn: 198954
2014-01-10 20:24:35 +00:00
Rafael Espindola
4dc5af8bc2 Move the llvm mangler to lib/IR.
This makes it available to tools that don't link with target (like llvm-ar).

llvm-svn: 198708
2014-01-07 21:19:40 +00:00
Chandler Carruth
87f14b4eec Re-sort all of the includes with ./utils/sort_includes.py so that
subsequent changes are easier to review. About to fix some layering
issues, and wanted to separate out the necessary churn.

Also comment and sink the include of "Windows.h" in three .inc files to
match the usage in Memory.inc.

llvm-svn: 198685
2014-01-07 11:48:04 +00:00
Rafael Espindola
30bfebf7cc Use a more direct check for finding out the file type.
No functionality change.

llvm-svn: 196811
2013-12-09 20:26:40 +00:00
Justin Bogner
f42cbce2da Protect user-supplied runtime library functions in LTO
Add user-supplied C runtime and compiler-rt library functions to
llvm.compiler.used to protect them from premature optimization by
passes like -globalopt and -ipsccp.  Calls to (seemingly unused)
runtime library functions can be added by -instcombine and instruction
lowering.

Patch by Duncan Exon Smith, thanks!

Fixes <rdar://problem/14740087>

llvm-svn: 194514
2013-11-12 21:44:01 +00:00
Rafael Espindola
24353f2de2 Use LTO_SYMBOL_SCOPE_DEFAULT_CAN_BE_HIDDEN instead of the "dso list".
There are two ways one could implement hiding of linkonce_odr symbols in LTO:
* LLVM tells the linker which symbols can be hidden if not used from native
  files.
* The linker tells LLVM which symbols are not used from other object files,
  but will be put in the dso symbol table if present.

GOLD's API is the second option. It was implemented almost 1:1 in llvm by
passing the list down to internalize.

LLVM already had partial support for the first option. It is also very similar
to how ld64 handles hiding these symbols when *not* doing LTO.

This patch then
* removes the APIs for the DSO list.
* marks LTO_SYMBOL_SCOPE_DEFAULT_CAN_BE_HIDDEN all linkonce_odr unnamed_addr
  global values and other linkonce_odr whose address is not used.
* makes the gold plugin responsible for handling the API mismatch.

llvm-svn: 193800
2013-10-31 20:51:58 +00:00
Rafael Espindola
a88c415234 Optimize linkonce_odr unnamed_addr functions during LTO.
Generalize the API so we can distinguish symbols that are needed just for a DSO
symbol table from those that are used from some native .o.

The symbols that are only wanted for the dso symbol table can be dropped if
llvm can prove every other dso has a copy (linkonce_odr) and the address is not
important (unnamed_addr).

llvm-svn: 191922
2013-10-03 18:29:09 +00:00
Rafael Espindola
83e2fc9be0 Fix option parsing in the gold plugin.
This was broken when options were moved up in r191680. No test because this is
specific LLVMgold.so/libLTO.so.

Patch by Tom Roeder!

llvm-svn: 191829
2013-10-02 14:36:23 +00:00
Rafael Espindola
c7fe26d6e4 Move command line options to the users of libLTO. Fixes --enable-shared build.
Patch by Richard Sandiford.

llvm-svn: 191680
2013-09-30 16:39:19 +00:00
Peter Collingbourne
8969240f9e Move LTO support library to a component, allowing it to be tested
more reliably across platforms.  Patch by Tom Roeder!

llvm-svn: 191343
2013-09-24 23:52:22 +00:00