1
0
mirror of https://github.com/RPCS3/llvm-mirror.git synced 2024-11-24 19:52:54 +01:00
Commit Graph

444 Commits

Author SHA1 Message Date
Peter Collingbourne
286be4f135 Bitcode: Change module reader functions to return an llvm::Expected.
Differential Revision: https://reviews.llvm.org/D26562

llvm-svn: 286752
2016-11-13 07:00:17 +00:00
Peter Collingbourne
a426ef0167 Bitcode: Change getModuleSummaryIndex() to return an llvm::Expected.
Differential Revision: https://reviews.llvm.org/D26539

llvm-svn: 286624
2016-11-11 19:50:39 +00:00
Peter Collingbourne
3a8f902dd8 Bitcode: Clean up error handling for certain bitcode query functions.
The functions getBitcodeTargetTriple(), isBitcodeContainingObjCCategory(),
getBitcodeProducerString() and hasGlobalValueSummary() now return errors
via their return value rather than via the diagnostic handler.

To make this work, re-implement these functions using non-member functions
so that they can be used without the LLVMContext required by BitcodeReader.

Differential Revision: https://reviews.llvm.org/D26532

llvm-svn: 286623
2016-11-11 19:50:24 +00:00
Teresa Johnson
7d18c2cf0a Split Bitcode/ReaderWriter.h into separate reader and writer headers
Summary:
Split ReaderWriter.h which contains the APIs into both the BitReader and
BitWriter libraries into BitcodeReader.h and BitcodeWriter.h.

This is to address Chandler's concern about sharing the same API header
between multiple libraries (BitReader and BitWriter). That concern is
why we create a single bitcode library in our downstream build of clang,
which led to r286297 being reverted as it added a dependency that
created a cycle only when there is a single bitcode library (not two as
in upstream).

Reviewers: mehdi_amini

Subscribers: dlj, mehdi_amini, llvm-commits

Differential Revision: https://reviews.llvm.org/D26502

llvm-svn: 286566
2016-11-11 05:34:58 +00:00
Mehdi Amini
3d0dd0ec2c Make the Error class constructor protected
This is forcing to use Error::success(), which is in a wide majority
of cases a lot more readable.

Differential Revision: https://reviews.llvm.org/D26481

llvm-svn: 286561
2016-11-11 04:28:40 +00:00
Peter Collingbourne
bcab72e19e Bitcode: Change the materializer interface to return llvm::Error.
Differential Revision: https://reviews.llvm.org/D26439

llvm-svn: 286382
2016-11-09 17:49:19 +00:00
Eli Friedman
f6fe3c8a24 Don't store Twine in a local variable.
Fixes post-commit review comment from r286177.

llvm-svn: 286275
2016-11-08 19:43:56 +00:00
Peter Collingbourne
a4bf768dd2 IR, Bitcode: Change bitcode reader to no longer own its memory buffer.
Unique ownership is just one possible ownership pattern for the memory buffer
underlying the bitcode reader. In practice, as this patch shows, ownership can
often reside at a higher level. With the upcoming change to allow multiple
modules in a single bitcode file, it will no longer be appropriate for
modules to generally have unique ownership of their memory buffer.

The C API exposes the ownership relation via the LLVMGetBitcodeModuleInContext
and LLVMGetBitcodeModuleInContext2 functions, so we still need some way for
the module to own the memory buffer. This patch does so by adding an owned
memory buffer field to Module, and using it in a few other places where it
is convenient.

Differential Revision: https://reviews.llvm.org/D26384

llvm-svn: 286214
2016-11-08 06:03:43 +00:00
Eli Friedman
cded552ec7 [LTO] Add error message on IO error in compileOptimizedToFile.
(No testcase because it's difficult to force an error here.)

Differential Revision: https://reviews.llvm.org/D26371

llvm-svn: 286177
2016-11-07 23:43:07 +00:00
Teresa Johnson
440d83db71 [ThinLTO] Disable importing and other cross-module optis at -O0
Summary:
There is no point to importing at -O0, since we won't inline. We should
also disable other cross-module optimizations.

(Plan to backport this fix to the 3.9 branch to fix PR30774)

Reviewers: pcc

Subscribers: johanengelen, mehdi_amini

Differential Revision: https://reviews.llvm.org/D25918

llvm-svn: 285648
2016-10-31 22:12:21 +00:00
Teresa Johnson
310ea0ff26 [ThinLTO] Correctly resolve linkonce when importing aliasee
Summary:
When we have an aliasee that is linkonce, while we can't convert
the non-prevailing copies to available_externally, we still need to
convert the prevailing copy to weak. If a reference to the aliasee
is exported, not converting a copy to weak will result in undefined
references when the linkonce is removed in its original module.

Add a new test and update existing tests.

Reviewers: mehdi_amini

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D26076

llvm-svn: 285512
2016-10-30 05:15:23 +00:00
Evgeniy Stepanov
2be3986f95 Utility functions for appending to llvm.used/llvm.compiler.used.
llvm-svn: 285143
2016-10-25 23:53:31 +00:00
Rafael Espindola
ba3f65d754 fix warning
llvm-svn: 285064
2016-10-25 12:28:26 +00:00
Rafael Espindola
f8f8e59cdd Make the LTO comdat api more symbol table friendly.
In an IR symbol table I would expect the comdats to be represented as:

- A table of strings, one for each comdat name.
- Each symbol has an optional index into that table.

The natural api for accessing that would be

InputFile:
ArrayRef<StringRef> getComdatTable() const;

Symbol:
int getComdatIndex() const;

This patch implements an API as close to that as possible.  The
implementation on top of the current IRObjectFile is a bit hackish,
but should map just fine over a symbol table and is very convenient to
use.

llvm-svn: 285061
2016-10-25 12:02:03 +00:00
Pavel Labath
6b2a0c490c Remove TimeValue usage from llvm/Support
Summary:
This is a follow-up to D25416. It removes all usages of TimeValue from
llvm/Support library (except for the actual TimeValue declaration), and replaces
them with appropriate usages of std::chrono. To facilitate this, I have added
small utility functions for converting time points and durations into appropriate
OS-specific types (FILETIME, struct timespec, ...).

Reviewers: zturner, mehdi_amini

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D25730

llvm-svn: 284966
2016-10-24 10:59:17 +00:00
Teresa Johnson
73f09465fc [ThinLTO] Default backend threads to heavyweight_hardware_concurrency
Summary:
Changes default backend parallelism from thread::hardware_concurrency to
the new llvm::heavyweight_hardware_concurrency, which for X86 Linux
defaults to the number of physical cores (and will fall back to
thread::hardware_concurrency otherwise). This avoid oversubscribing
the physical cores using hyperthreading.

Reviewers: mehdi_amini, pcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D25775

llvm-svn: 284618
2016-10-19 17:35:01 +00:00
Michal Gorny
691da67fd3 [cmake] Declare LLVM_CMAKE_PATH for use in subprojects
Declare the LLVM_CMAKE_PATH to the source directory location of CMake
files, in order to make it possible to easily use them in subprojects.
Such a variable is already declared in most of LLVM projects
(and inconsistently mixed with direct source tree references), including
Clang, LLDB, compiler-rt, libcxx... Declaring it inside main LLVM tree
makes it possible to avoid having to declare fallback values or use
conditionals in those projects.

It should be noted that in some of the subprojects LLVM_CMAKE_PATH is
used to reference generated LLVMConfig.cmake file. However, these
references are conditional to stand-alone builds and explicitly
including this file is unnecessary in combined builds.

Differential Revision: https://reviews.llvm.org/D25724

llvm-svn: 284581
2016-10-19 12:18:34 +00:00
Benjamin Kramer
641730af4a Reduce global namespace pollution. NFC.
llvm-svn: 284521
2016-10-18 19:39:31 +00:00
Peter Collingbourne
2f78370807 LTO: Use the correct mangler function in LTOCodeGenerator::applyScopeRestrictions().
We need to use the overload of Mangler::getNameWithPrefix that takes a
GlobalValue in order to mangle in the stdcall stack byte count for Windows
targets.

Differential Revision: https://reviews.llvm.org/D25529

llvm-svn: 284040
2016-10-12 20:12:19 +00:00
Teresa Johnson
de773ccc50 [ThinLTO] Don't link module level assembly when importing
Module inline asm was always being linked/concatenated
when running the IRLinker. This is correct for full LTO but not when
we are importing for ThinLTO, as it can result in multiply defined
symbols when the module asm defines a global symbol.

In order to test with llvm-lto2, I had to work around PR30396,
where a symbol that is defined in module assembly but defined in the
LLVM IR appears twice. Added workaround to llvm-lto2 with a FIXME.

Fixes PR30610.

Reviewers: mehdi_amini

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D25359

llvm-svn: 284030
2016-10-12 18:39:29 +00:00
Mehdi Amini
885692c663 ThinLTO: don't perform incremental LTO on module without a hash
Clang always emit a hash for ThinLTO, but as other frontend are
starting to use ThinLTO, this could be a serious bug.

Differential Revision: https://reviews.llvm.org/D25379

llvm-svn: 283655
2016-10-08 04:44:23 +00:00
Mehdi Amini
e74b55c5f5 ThinLTO: handles modules with empty summaries
We need to add an entry in the combined-index for modules that have
a hash but otherwise empty summary, this is needed so that we can
get the hash for the module.

Also, if no entry is present in the combined index for a module, we
need to skip it when trying to compute a cache entry.

Differential Revision: https://reviews.llvm.org/D25300

llvm-svn: 283654
2016-10-08 04:44:18 +00:00
Mehdi Amini
c7f513e186 Recommit "Use StringRef in LTOModule implementation (NFC)""
This reverts commit r283456 and reapply r282997, with explicitly
zeroing the struct member to workaround a bug in MSVC2013 with
zero-initialization: https://connect.microsoft.com/VisualStudio/feedback/details/802160

llvm-svn: 283581
2016-10-07 19:05:14 +00:00
Mehdi Amini
a18dc1c7ea Revert "Use StringRef in LTOModule implementation (NFC)"
This reverts commit r282997, a windows bot is asserting in
one test apparently.

llvm-svn: 283456
2016-10-06 15:12:22 +00:00
Mehdi Amini
d309b57e6e Use StringRef in LTOCodegenerator (NFC)
llvm-svn: 282998
2016-10-01 01:18:23 +00:00
Mehdi Amini
f1666198ae Use StringRef in LTOModule implementation (NFC)
llvm-svn: 282997
2016-10-01 01:18:16 +00:00
Peter Collingbourne
85f81c10fd Add explanatory comment.
llvm-svn: 282678
2016-09-29 03:29:28 +00:00
Peter Collingbourne
887d502d5e LTO: Fix use-after-scope error.
llvm-svn: 282665
2016-09-29 01:28:36 +00:00
Dehao Chen
8c4cb5e152 Refactor the ProfileSummaryInfo to use doInitialization and doFinalization to handle Module update.
Summary: This refactors the change in r282616

Reviewers: davidxl, eraman, mehdi_amini

Subscribers: mehdi_amini, davide, llvm-commits

Differential Revision: https://reviews.llvm.org/D25041

llvm-svn: 282630
2016-09-28 21:00:58 +00:00
Dehao Chen
efea2f3ad1 Fix the bug introduced in r282616.
llvm-svn: 282618
2016-09-28 18:54:36 +00:00
Piotr Padlewski
3152e057e1 [thinlto] Basic thinlto fdo heuristic
Summary:
This patch improves thinlto importer
by importing 3x larger functions that are called from hot block.

I compared performance with the trunk on spec, and there
were about 2% on povray and 3.33% on milc. These results seems
to be consistant and match the results Teresa got with her simple
heuristic. Some benchmarks got slower but I think they are just
noisy (mcf, xalancbmki, omnetpp)- running the benchmarks again with
more iterations to confirm. Geomean of all benchmarks including the noisy ones
were about +0.02%.

I see much better improvement on google branch with Easwaran patch
for pgo callsite inlining (the inliner actually inline those big functions)
Over all I see +0.5% improvement, and I get +8.65% on povray.
So I guess we will see much bigger change when Easwaran patch will land
(it depends on new pass manager), but it is still worth putting this to trunk
before it.

Implementation details changes:
- Removed CallsiteCount.
- ProfileCount got replaced by Hotness
- hot-import-multiplier is set to 3.0 for now,
didn't have time to tune it up, but I see that we get most of the interesting
functions with 3, so there is no much performance difference with higher, and
binary size doesn't grow as much as with 10.0.

Reviewers: eraman, mehdi_amini, tejohnson

Subscribers: mehdi_amini, llvm-commits

Differential Revision: https://reviews.llvm.org/D24638

llvm-svn: 282437
2016-09-26 20:37:32 +00:00
Peter Collingbourne
fbbb4ed4f1 Add qualification to fix MSVC build.
llvm-svn: 282313
2016-09-23 23:23:23 +00:00
Peter Collingbourne
cef2ce70e8 LTO: Simplify caching interface.
The NativeObjectOutput class has a design problem: it mixes up the caching
policy with the interface for output streams, which makes the client-side
code hard to follow and would for example make it harder to replace the
cache implementation in an arbitrary client.

This change separates the two aspects by moving the caching policy
to a separate field in Config, replacing NativeObjectOutput with a
NativeObjectStream class which only deals with streams and does not need to
be overridden by most clients and introducing an AddFile callback for adding
files (e.g. from the cache) to the link.

Differential Revision: https://reviews.llvm.org/D24622

llvm-svn: 282299
2016-09-23 21:33:43 +00:00
Teresa Johnson
2856a03dce [ThinLTO] Emit files for distributed builds for all modules
With the new LTO API in r278338, we stopped emitting the individual
index files and imports files for some modules in the distributed backend
case (thinlto-index-only plugin option).

Specifically, this is when the linker decides not to include a module in the
link, because it was in an archive library and did not have a strong
reference to it. Not creating the expected output files makes the
distributed build system implementation more difficult, in terms of
checking for the expected outputs of the thin link, and scheduling the
backend jobs. To address this, the gold-plugin will write dummy empty
.thinlto.bc and .imports files for modules not included in the link
(which LTO never sees).

Augmented a gold v1.12+ test, since that version of gold has the handling
for notifying on modules not being included in the link.

llvm-svn: 282100
2016-09-21 19:12:05 +00:00
Teresa Johnson
0f1566e03c [ThinLTO] Always emit a summary when compiling in ThinLTO mode
Summary:
Emit an empty summary section, instead of no summary section, when
there are no global variables in the index. This ensures that LTO
will treat these files as ThinLTO inputs, instead of as regular
LTO inputs.

In addition to not being what the user likely intended when
compiling with -flto=thin, the current behavior is problematic for
distributed build systems that expect to get ThinLTO index and imports
files back for each input compiled with -flto=thin. Combining into
a single regular LTO module also reduces the backend parallelism.
And in the case where the index was suppressed due to uses in
inline assembly, combining into a single LTO module could provoke
renaming of duplicates that we were trying to prevent by suppressing
the index.

This change required a couple of fixes to handle the empty summary
section.

Reviewers: mehdi_amini

Subscribers: mehdi_amini, llvm-commits, pcc

Differential Revision: https://reviews.llvm.org/D24779

llvm-svn: 282037
2016-09-20 23:07:17 +00:00
Davide Italiano
5be981977a [lib/LTO] Try harder to reduce code duplication. NFCI.
llvm-svn: 281843
2016-09-17 22:32:42 +00:00
Davide Italiano
b91039268d [LTO] Add ability to parse AA pipelines.
This is supposed to be a drop in replacement for what lld
provides via --lto-newpm-aa-pipeline.

llvm-svn: 281774
2016-09-16 21:03:21 +00:00
Davide Italiano
6e57a52f32 [LTO] Prevent asm references to be dropped from the output.
Differential Revision:  https://reviews.llvm.org/D24617

llvm-svn: 281741
2016-09-16 16:05:25 +00:00
Teresa Johnson
b6ffcf7fbd [LTO] Fix handling of mixed (regular and thin) mode LTO
Summary:
In runThinLTO we start the task numbering for ThinLTO backend
tasks depending on whether there was also a regular LTO object
(CombinedModule). However, the CombinedModule is moved at
the end of runRegularLTO, so we need to save this information and
pass it into runThinLTO. Otherwise the AddOutput callback to the client
will use the same task number for both the regular LTO object
and the first ThinLTO object, which in gold-plugin caused only
one to be end up in the output filename array and therefore passed
back to gold for the final native link.

Reviewers: pcc, mehdi_amini

Subscribers: mehdi_amini, kromanova

Differential Revision: https://reviews.llvm.org/D24643

llvm-svn: 281725
2016-09-16 13:54:19 +00:00
Eric Christopher
fc58955e10 Move the Mangler from the AsmPrinter down to TLOF and clean up the
TLOF API accordingly.

llvm-svn: 281708
2016-09-16 07:33:15 +00:00
Mehdi Amini
5f7f123920 [LTO] Fix commons handling
Previously the prevailing information was not honored, and commons
symbols could override a strong definition. This patch fixes it and
propose the following semantic for commons: the client should mark
as prevailing the commons that it expects the LTO implementation to
merge (i.e. take the maximum size and alignment).
It implies that commons are allowed to have multiple prevailing
definitions.

Differential Revision: https://reviews.llvm.org/D24545

llvm-svn: 281538
2016-09-14 21:05:04 +00:00
Davide Italiano
1946df91a4 [lib/LTO] Fix a typo. NFC.
llvm-svn: 281517
2016-09-14 18:48:43 +00:00
Davide Italiano
a12abfaeba [LTO] Don't pass SF_Undefined symbols to the IRmover.
This should fix PR 30363.

llvm-svn: 281366
2016-09-13 18:45:13 +00:00
Davide Italiano
9cc7f6f6fa [lib/LTO] Add a way to run a custom pipeline
Differential Revision:  https://reviews.llvm.org/D24095

llvm-svn: 280830
2016-09-07 17:46:16 +00:00
Davide Italiano
76aa4454b1 [LTO] Rename variables to be more explicative.
Thanks to Mehdi for the suggestion!

llvm-svn: 280772
2016-09-07 01:08:31 +00:00
Mehdi Amini
2f275228ad [LTO] Constify (NFC)
llvm-svn: 280687
2016-09-06 03:23:45 +00:00
Davide Italiano
1ae4416813 [lib/LTO] Simplify. No functional change intended.
llvm-svn: 280503
2016-09-02 16:37:31 +00:00
Davide Italiano
e4f4bf3d65 [lib/LTO] Simplify a bit. NFCI.
llvm-svn: 280396
2016-09-01 18:34:47 +00:00
Davide Italiano
6c9ddd7257 [lib/LTO] Factor out logic for running passes.
This is in preparation for adding an option
to run a custom pipeline with the new PM. It's
currently used in lld.

llvm-svn: 280258
2016-08-31 17:02:44 +00:00
Teresa Johnson
bad28dab29 [LTO] Remove extraneous output
Remove some debugging output to stderr that snuck in with r279576.

llvm-svn: 279974
2016-08-29 15:33:01 +00:00
Teresa Johnson
864ef41e7f [LTO] Don't create a new common unless merged has different size
Summary:
This addresses a regression in common handling from the new LTO
API in r278338. Only create a new common if the size is different.
The type comparison against an array type fails when the size is
different but not an array. GlobalMerge does not handle the
array types as well and we lose some global merging opportunities.

Reviewers: mehdi_amini

Subscribers: junbuml, llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D23955

llvm-svn: 279911
2016-08-27 04:41:22 +00:00
Teresa Johnson
048eeff265 [ThinLTO] Move loading of cache entry to client
Summary:
Have the cache pass back the path to the cache entry when it
is ready to be loaded, instead of a buffer.

For gold-plugin we can simply pass this file back to gold directly,
which avoids expensive writing of a separate tmp file. Ensure
the cache entry is not deleted on cleanup by adjusting the setting
of the IsTemporary flags.

Moved the loading of the buffer into llvm-lto2 to maintain current
behavior.

Reviewers: mehdi_amini

Subscribers: llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D23946

llvm-svn: 279883
2016-08-26 23:29:14 +00:00
Rafael Espindola
26c1b18156 Make writeToResolutionFile a static helper.
llvm-svn: 279859
2016-08-26 20:19:35 +00:00
Davide Italiano
f463cb5866 [lib/LTO] Add an assertion to catch invalid opt levels.
llvm-svn: 279823
2016-08-26 15:22:59 +00:00
Mehdi Amini
54cea79c7b [ThinLTO] Add caching to the new LTO API
Add the ability to plug a cache on the LTO API.
I tried to write such that a linker implementation can
control the cache backend. This is intrusive and I'm
not totally happy with it, but I can't figure out a
better design right now.

Differential Revision: https://reviews.llvm.org/D23599

llvm-svn: 279576
2016-08-23 21:30:12 +00:00
Mehdi Amini
1a6d8014ee Stop always creating and running an LTO compilation if there is not a single LTO object
Summary:
I assume there was a use case, so maybe this strawman patch will help
clarifying if it is legit.
In any case the current situation is not legit: a ThinLTO compilation
should not trigger an unexpected full LTO compilation.
Right now, adding a --save-temps option triggers this and makes the
number of output differs.

Reviewers: tejohnson

Subscribers: pcc, llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D23600

llvm-svn: 279550
2016-08-23 18:39:12 +00:00
Mehdi Amini
6dfb67d640 [ThinLTO] Make sure the Context used for the ThinLTO backend has all the appropriate options
An important performance setting on the LLVMContext for LTO is
enableDebugTypeODRUniquing(), this adds an automatic merging of
debug information in the context based on type ids.

Also, the lto::Config includes a diagnostic handler that needs to
be set on the Context, as well as the setDiscardValueNames() setting.

llvm-svn: 279532
2016-08-23 16:53:34 +00:00
Davide Italiano
7033df4633 [LTOCodeGenerator] Reduce code duplication. NFCI.
llvm-svn: 279514
2016-08-23 12:32:57 +00:00
Mehdi Amini
91a6c36524 [LTO] Constify the Module Hook function (NFC)
It use to be non-const for the sole purpose of custom handling of
commons symbol. This is moved now in the regular LTO handling now
and such we can constify the callback.

llvm-svn: 279438
2016-08-22 16:17:40 +00:00
Mehdi Amini
2007e493a9 [LTO] Handles commons in monolithic LTO
The gold-plugin was doing this internally, now the API is handling
commons correctly based on the given resolution.

Differential Revision: https://reviews.llvm.org/D23739

llvm-svn: 279417
2016-08-22 06:25:46 +00:00
Mehdi Amini
d00c9cd31e [LTO] Add a "CodeGenOnly" option. Allows the client to skip the optimizer.
Summary: Slowly getting on par with libLTO

Reviewers: tejohnson

Subscribers: llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D23615

llvm-svn: 279416
2016-08-22 06:25:41 +00:00
Mehdi Amini
6efdc4d959 [LTO] Remove dead-code: collectUsedGlobalVariables has been moved to Thin and LTO specifc path (NFC)
llvm-svn: 279261
2016-08-19 15:35:44 +00:00
Chandler Carruth
f68dd1e089 [PM] Rework the new PM support for building the ModuleSummaryIndex to
directly produce the index as the value type result.

This requires making the index movable which is straightforward. It
greatly simplifies things by allowing us to completely avoid the builder
API and the layers of abstraction inherent there. Instead both pass
managers can directly construct these when run by value. They still
won't be constructed truly eagerly thanks to the optional in the legacy
PM. The code that directly builds the index can also just share a direct
function.

A notable change here is that the result type of the analysis for the
new PM is no longer a reference type. This was really problematic when
making changes to how we handle result types to make our interface
requirements *much* more strict and precise. But I think this is an
overall improvement.

Differential Revision: https://reviews.llvm.org/D23701

llvm-svn: 279216
2016-08-19 07:49:19 +00:00
Mehdi Amini
cbc92ad10a [LTO] Move callback member from base class to the derived where it is used (NFC)
llvm-svn: 279212
2016-08-19 06:10:03 +00:00
Mehdi Amini
7d534f1d55 [LTO] Add a move to inialize member in ctor initialization list (NFC)
llvm-svn: 279210
2016-08-19 05:56:37 +00:00
Mehdi Amini
cb1ea8721b [LTO] Promote before performing weak resolution
Summary:
This was reversed compared to ThinLTOCodeGenerator for some reason,
and lead to an increased code-size on my tests. I figured that the
weak resolution may internalize a linkonce function, which will be
promoted immediately (and renamed), before being internalized again.

Reviewers: tejohnson

Subscribers: pcc, llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D23632

llvm-svn: 279021
2016-08-18 00:59:24 +00:00
Mehdi Amini
ebde3ba665 [LTO] Change addSaveTemps API: do not add dot to the supplied prefix path
Summary:
It does not play well with directories (end up with a bunch of hidden
files).
Also, do not strip the 0 suffix for the first task, especially since
0 can be used by ThinLTO as well now.

Reviewers: tejohnson

Subscribers: mehdi_amini, pcc, llvm-commits

Differential Revision: https://reviews.llvm.org/D23612

llvm-svn: 279014
2016-08-18 00:12:33 +00:00
Mehdi Amini
e80ee44a77 [LTO] Introduce an Output class to wrap the output stream creation (NFC)
Summary:
While NFC for now, this will allow more flexibility on the client side
to hold state necessary to back up the stream.
Also when adding caching, this class will grow in complexity.

Note I blindly modified the gold-plugin as I can't compile it.

Reviewers: tejohnson

Subscribers: mehdi_amini, llvm-commits

Differential Revision: https://reviews.llvm.org/D23542

llvm-svn: 278907
2016-08-17 06:23:09 +00:00
Mehdi Amini
87829dabc8 [LTO] Simplify APIs and constify (NFC)
Summary:
Multiple APIs were taking a StringMap for the ImportLists containing
the entries for for all the modules while operating on a single entry
for the current module. Instead we can pass the desired ModuleImport
directly. Also some of the APIs were not const, I believe just to be
able to use operator[] on the StringMap.

Reviewers: tejohnson

Subscribers: llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D23537

llvm-svn: 278776
2016-08-16 05:46:05 +00:00
Mehdi Amini
e4d1ed48d7 [LTO] Rename variables with meaningul names, i.e. more than one character (NFC)
llvm-svn: 278766
2016-08-16 00:44:46 +00:00
Teresa Johnson
5f3335a592 [ThinLTO] Fix temp file dumping, enable via llvm-lto and test it
Summary:
Fixed a bug in ThinLTOCodeGenerator's temp file dumping. The Twine
needs to be passed directly as an argument, or a copy saved into a
std::string.

It doesn't seem there are any consumers of this, so I added a new option
to llvm-lto to enable saving of temp files during ThinLTO, and augmented
a test to use it to check post-import but pre-opt bitcode.

Reviewers: mehdi_amini

Subscribers: llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D23525

llvm-svn: 278761
2016-08-15 23:24:57 +00:00
Teresa Johnson
2110ec3312 Fix type truncation warnings
Avoid type truncation warnings from a 32-bit bot due to size_t not
being unsigned long long, by converting the variables and constants to
unsigned. This was introduced by r278338 and caused warnings here:
http://bb.pgr.jp/builders/i686-mingw32-RA-on-linux/builds/15527/steps/build_llvmclang/logs/warnings%20%287%29

llvm-svn: 278406
2016-08-11 20:38:39 +00:00
Teresa Johnson
bd574a6d8f Restore "Resolution-based LTO API."
This restores commit r278330, with fixes for a few bot failures:
- Fix a late change I had made to the save temps output file that I
  missed due to existing files sitting on my disk
- Fix a bunch of Windows bot failures with "ambiguous call to overloaded
  function" due to confusion between llvm::make_unique vs
  std::make_unique (preface the new make_unique calls with "llvm::")
- Attempt to fix a modules bot failure by adding a missing include
  to LTO/Config.h.

Original change:

Resolution-based LTO API.

Summary:
This introduces a resolution-based LTO API. The main advantage of this API over
existing APIs is that it allows the linker to supply a resolution for each
symbol in each object, rather than the combined object as a whole. This will
become increasingly important for use cases such as ThinLTO which require us
to process symbol resolutions in a more complicated way than just adjusting
linkage.

Patch by Peter Collingbourne.

Reviewers: rafael, tejohnson, mehdi_amini

Subscribers: lhames, tejohnson, mehdi_amini, llvm-commits

Differential Revision: https://reviews.llvm.org/D20268

llvm-svn: 278338
2016-08-11 14:58:12 +00:00
Teresa Johnson
638d6367b4 Revert "Resolution-based LTO API."
This reverts commit r278330.

I made a change to the save temps output that is causing issues with the
bots. Didn't realize this because I had older output files sitting on
disk in my test output directory.

llvm-svn: 278331
2016-08-11 13:03:56 +00:00
Teresa Johnson
6fd4796f76 Resolution-based LTO API.
Summary:
This introduces a resolution-based LTO API. The main advantage of this API over
existing APIs is that it allows the linker to supply a resolution for each
symbol in each object, rather than the combined object as a whole. This will
become increasingly important for use cases such as ThinLTO which require us
to process symbol resolutions in a more complicated way than just adjusting
linkage.

Patch by Peter Collingbourne.

Reviewers: rafael, tejohnson, mehdi_amini

Subscribers: lhames, tejohnson, mehdi_amini, llvm-commits

Differential Revision: https://reviews.llvm.org/D20268

Address review comments

llvm-svn: 278330
2016-08-11 12:56:40 +00:00
Peter Collingbourne
b54bee8ab7 Move legacy LTO interface headers to legacy/ directory.
Differential Revision: https://reviews.llvm.org/D22173

llvm-svn: 275476
2016-07-14 21:21:16 +00:00
Dehao Chen
9b4dc6d522 New pass manager for LICM.
Summary: Port LICM to the new pass manager.

Reviewers: davidxl, silvas

Subscribers: krasin, vitalybuka, silvas, davide, sanjoy, llvm-commits, mehdi_amini

Differential Revision: http://reviews.llvm.org/D21772

llvm-svn: 275222
2016-07-12 22:37:48 +00:00
Vitaly Buka
d3a08f0254 Revert "New pass manager for LICM."
Summary: This reverts commit r275118.

Subscribers: sanjoy, mehdi_amini

Differential Revision: http://reviews.llvm.org/D22259

llvm-svn: 275156
2016-07-12 06:25:32 +00:00
Dehao Chen
75ac849ab9 New pass manager for LICM.
Summary: Port LICM to the new pass manager.

Reviewers: davidxl, silvas

Subscribers: silvas, davide, sanjoy, llvm-commits, mehdi_amini

Differential Revision: http://reviews.llvm.org/D21772

llvm-svn: 275118
2016-07-11 22:45:24 +00:00
Peter Collingbourne
0442bd0563 ThinLTO: Do not take into account whether a definition has multiple copies when promoting.
We currently do not touch a symbol's linkage in the case where a definition
has a single copy. However, this code is effectively unnecessary: either
the definition is not exported, in which case the internalize phase sets
its linkage to internal, or it is exported, in which case we need to promote
linkage to weak. Those two cases are already handled by existing code.

I believe that the only real functional change here is in the case where we
have a single definition which does not prevail (e.g. because the definition
in a native object file prevails). In that case we now lower linkage to
available_externally following the existing code path for that case.

As a result we can remove the isExported function parameter from the
thinLTOResolveWeakForLinkerInIndex function.

Differential Revision: http://reviews.llvm.org/D21883

llvm-svn: 274784
2016-07-07 18:31:51 +00:00
Peter Collingbourne
dcafb3d672 ThinLTO: Remove check for multiple modules before applying weak resolutions.
This check is not only unnecessary, it can produce the wrong result. If we
are linking a single module and it has an exported linkonce symbol, we need
to promote to weak in order to avoid PR19901-style problems.

Differential Revision: http://reviews.llvm.org/D21917

llvm-svn: 274722
2016-07-07 01:51:11 +00:00
Davide Italiano
e42bdc124e [UpdateCompilerUsed] API rename and cleanup, suggested by Rafaael.
* UpdateCompilerUsed() -> updateCompilerUsed()
* ThinLTO doesn't use the API so we can remove the include
* Clean up unused #include <functional> from the header
* Rename #ifdef guard comment to be correct.

llvm-svn: 273461
2016-06-22 19:50:42 +00:00
Davide Italiano
bdd047956c [LTO] Move UpdateCompilerUsed.h from lib/ to include/
I plan to use it in lld soon.

Differential Revision:  http://reviews.llvm.org/D21575

llvm-svn: 273380
2016-06-22 04:52:43 +00:00
Davide Italiano
af24ebdbb8 [PM] Port MergedLoadStoreMotion to the new pass manager, take two.
This is indeed a much cleaner approach (thanks to Daniel Berlin
for pointing out), and also David/Sean for review.

Differential Revision:  http://reviews.llvm.org/D21454

llvm-svn: 273032
2016-06-17 19:10:09 +00:00
Davide Italiano
f3b1808d94 [PM] Revert the port of MergeLoadStoreMotion to the new pass manager.
Daniel Berlin expressed some real concerns about the port and proposed
and alternative approach. I'll revert this for now while working on a
new patch, which I hope to put up for review shortly. Sorry for the churn.

llvm-svn: 272925
2016-06-16 17:40:53 +00:00
David Majnemer
c6df3d773b Remove the ScalarReplAggregates pass
Nearly all the changes to this pass have been done while maintaining and
updating other parts of LLVM.  LLVM has had another pass, SROA, which
has superseded ScalarReplAggregates for quite some time.

Differential Revision: http://reviews.llvm.org/D21316

llvm-svn: 272737
2016-06-15 00:19:09 +00:00
Sean Silva
e007734fd3 [PM] Port MemCpyOpt to the new PM.
The need for all these Lookup* functions is just because of calls to
getAnalysis inside methods (i.e. not at the top level) of the
runOnFunction method. They should be straightforward to clean up when
the old PM is gone.

llvm-svn: 272615
2016-06-14 02:44:55 +00:00
Davide Italiano
5434c2f38a [PM] Port MergedLoadStoreMotion to the new pass manager.
llvm-svn: 272606
2016-06-14 00:49:23 +00:00
Benjamin Kramer
9809f36d76 Move instances of std::function.
Or replace with llvm::function_ref if it's never stored. NFC intended.

llvm-svn: 272513
2016-06-12 16:13:55 +00:00
Sean Silva
eb63d26915 [PM] Port ReversePostOrderFunctionAttrs to the new PM
Below are my super rough notes when porting. They can probably serve as
a basic guide for porting other passes to the new PM. As I port more
passes I'll expand and generalize this and make a proper
docs/HowToPortToNewPassManager.rst document. There is also missing
documentation for general concepts and API's in the new PM which will
require some documentation.
Once there is proper documentation in place we can put up a list of
passes that have to be ported and game-ify/crowdsource the rest of the
porting (at least of the middle end; the backend is still unclear).

I will however be taking personal responsibility for ensuring that the
LLD/ELF LTO pipeline is ported in a timely fashion. The remaining passes
to be ported are (do something like
`git grep "<the string in the bullet point below>"` to find the pass):

General Scalar:
[ ] Simplify the CFG
[ ] Jump Threading
[ ] MemCpy Optimization
[ ] Promote Memory to Register
[ ] MergedLoadStoreMotion
[ ] Lazy Value Information Analysis

General IPO:
[ ] Dead Argument Elimination
[ ] Deduce function attributes in RPO

Loop stuff / vectorization stuff:
[ ] Alignment from assumptions
[ ] Canonicalize natural loops
[ ] Delete dead loops
[ ] Loop Access Analysis
[ ] Loop Invariant Code Motion
[ ] Loop Vectorization
[ ] SLP Vectorizer
[ ] Unroll loops

Devirtualization / CFI:
[ ] Cross-DSO CFI
[ ] Whole program devirtualization
[ ] Lower bitset metadata

CGSCC passes:
[ ] Function Integration/Inlining
[ ] Remove unused exception handling info
[ ] Promote 'by reference' arguments to scalars

Please let me know if you are interested in working on any of the passes
in the above list (e.g. reply to the post-commit thread for this patch).
I'll probably be tackling "General Scalar" and "General IPO" first FWIW.

Steps as I port "Deduce function attributes in RPO"
---------------------------------------------------

(note: if you are doing any work based on these notes, please leave a
note in the post-commit review thread for this commit with any
improvements / suggestions / incompleteness you ran into!)

Note: "Deduce function attributes in RPO" is a module pass.

1. Do preparatory refactoring.

Do preparatory factoring. In this case all I had to do was to pull out a static helper (r272503).
(TODO: give more advice here e.g. if pass holds state or something)

2. Rename the old pass class.

llvm/lib/Transforms/IPO/FunctionAttrs.cpp
Rename class ReversePostOrderFunctionAttrs -> ReversePostOrderFunctionAttrsLegacyPass
in preparation for adding a class ReversePostOrderFunctionAttrs as the pass in the new PM.
(edit: actually wait what? The new class name will be
ReversePostOrderFunctionAttrsPass, so it doesn't conflict. So this step is
sort of useless churn).

llvm/include/llvm/InitializePasses.h
llvm/lib/LTO/LTOCodeGenerator.cpp
llvm/lib/Transforms/IPO/IPO.cpp
llvm/lib/Transforms/IPO/FunctionAttrs.cpp
Rename initializeReversePostOrderFunctionAttrsPass -> initializeReversePostOrderFunctionAttrsLegacyPassPass
(note that the "PassPass" thing falls out of `s/ReversePostOrderFunctionAttrs/ReversePostOrderFunctionAttrsLegacyPass/`)
Note that the INITIALIZE_PASS macro is what creates this identifier name, so renaming the class requires this renaming too.

Note that createReversePostOrderFunctionAttrsPass does not need to be
renamed since its name is not generated from the class name.

3. Add the new PM pass class.

In the new PM all passes need to have their
declaration in a header somewhere, so you will often need to add a header.
In this case
llvm/include/llvm/Transforms/IPO/FunctionAttrs.h is already there because
PostOrderFunctionAttrsPass was already ported.
The file-level comment from the .cpp file can be used as the file-level
comment for the new header. You may want to tweak the wording slightly
from "this file implements" to "this file provides" or similar.

Add declaration for the new PM pass in this header:

    class ReversePostOrderFunctionAttrsPass
        : public PassInfoMixin<ReversePostOrderFunctionAttrsPass> {
    public:
      PreservedAnalyses run(Module &M, AnalysisManager<Module> &AM);
    };

Its name should end with `Pass` for consistency (note that this doesn't
collide with the names of most old PM passes). E.g. call it
`<name of the old PM pass>Pass`.

Also, move the doxygen comment from the old PM pass to the declaration of
this class in the header.
Also, include the declaration for the new PM class
`llvm/Transforms/IPO/FunctionAttrs.h` at the top of the file (in this case,
it was already done when the other pass in this file was ported).

Now define the `run` method for the new class.
The main things here are:
a) Use AM.getResult<...>(M) to get results instead of `getAnalysis<...>()`

b) If the old PM pass would have returned "false" (i.e. `Changed ==
false`), then you should return PreservedAnalyses::all();

c) In the old PM getAnalysisUsage method, observe the calls
   `AU.addPreserved<...>();`.

   In the case `Changed == true`, for each preserved analysis you should do
   call `PA.preserve<...>()` on a PreservedAnalyses object and return it.
   E.g.:

       PreservedAnalyses PA;
       PA.preserve<CallGraphAnalysis>();
       return PA;

Note that calls to skipModule/skipFunction are not supported in the new PM
currently, so optnone and optimization bisect support do not work. You can
just drop those calls for now.

4. Add the pass to the new PM pass registry to make it available in opt.

In llvm/lib/Passes/PassBuilder.cpp add a #include for your header.
`#include "llvm/Transforms/IPO/FunctionAttrs.h"`
In this case there is already an include (from when
PostOrderFunctionAttrsPass was ported).

Add your pass to llvm/lib/Passes/PassRegistry.def
In this case, I added
`MODULE_PASS("rpo-functionattrs", ReversePostOrderFunctionAttrsPass())`
The string is from the `INITIALIZE_PASS*` macros used in the old pass
manager.

Then choose a test that uses the pass and use the new PM `-passes=...` to
run it.
E.g. in this case there is a test that does:
; RUN: opt < %s -basicaa -functionattrs -rpo-functionattrs -S | FileCheck %s
I have added the line:
; RUN: opt < %s -aa-pipeline=basic-aa -passes='require<targetlibinfo>,cgscc(function-attrs),rpo-functionattrs' -S | FileCheck %s
The `-aa-pipeline=basic-aa` and
`require<targetlibinfo>,cgscc(function-attrs)` are what is needed to run
functionattrs in the new PM (note that in the new PM "functionattrs"
becomes "function-attrs" for some reason). This is just pulled from
`readattrs.ll` which contains the change from when functionattrs was ported
to the new PM.
Adding rpo-functionattrs causes the pass that was just ported to run.

llvm-svn: 272505
2016-06-12 07:48:51 +00:00
Benjamin Kramer
d415569b3b Apply most suggestions of clang-tidy's performance-unnecessary-value-param
Avoids unnecessary copies. All changes audited & pass tests with asan.
No functional change intended.

llvm-svn: 272190
2016-06-08 19:09:22 +00:00
Teresa Johnson
f1d37e4c3b [ThinLTO] Resolve LinkOnceAny
Summary:
Ensure we keep prevailing copy of LinkOnceAny by converting it to
WeakAny.

Rename odr_resolution test to the now more appropriate weak_resolution
(weak in the linker sense includes linkonce).

Reviewers: joker.eph

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D20634

llvm-svn: 270850
2016-05-26 14:16:52 +00:00
Teresa Johnson
655b4d9f20 [ThinLTO] Refactor ODR resolution and internalization (NFC)
Move the now index-based ODR resolution and internalization routines out
of ThinLTOCodeGenerator.cpp and into either LTO.cpp (index-based
analysis) or FunctionImport.cpp (index-driven optimizations).
This is to enable usage by other linkers.

llvm-svn: 270698
2016-05-25 14:03:11 +00:00
Teresa Johnson
19d7a21c83 [ThinLTO] Handle empty exports list for module.
We might have an empty exports list for a module and shouldn't assert
when checking if a symbol is exported.

llvm-svn: 270600
2016-05-24 19:12:48 +00:00
Teresa Johnson
751b460b85 [ThinLTO] Fix bot failures by avoiding threaded map modifications
Ensure multiple threads won't have to modify map, probable cause of
bot failures from r270584 such as:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/13095

llvm-svn: 270595
2016-05-24 18:44:01 +00:00
Teresa Johnson
5cc2580600 [ThinLTO] Change ODR resolution and internalization to be index-based
Summary:
This patch changes the ODR resolution and internalization to be based on
updates to the Index, which are consumed by the backend portion of the
transformations.

It will be followed by an NFC change to move these out of libLTO's
ThinLTOCodeGenerator so that it can be used by other linkers
(gold and lld) and by ThinLTO distributed backends.

The global summary-based portions use callbacks so that the client can
determine the prevailing copy and other information in a client-specific
way. Eventually, with the API being developed in D20268, these may be
modified to use information such as symbol resolutions, supplied by the
clients to the API.

Reviewers: joker-eph

Subscribers: joker.eph, pcc, llvm-commits

Differential Revision: http://reviews.llvm.org/D20290

llvm-svn: 270584
2016-05-24 17:24:25 +00:00
Teresa Johnson
9184fab6a9 [ThinLTO] Refactor module loader handling into new LTO file (NFC)
Moved the ModuleLoader and supporting helper loadModuleFromBuffer out of
ThinLTOCodeGenerator and into new LTO.h/LTO.cpp files. This is in
preparation for a patch that will utilize these in the gold-plugin.

Note that there are some other pending patches (D20268 and D20290) that
also plan to refactor common interfaces and functionality into this same
pair of new files.

llvm-svn: 270509
2016-05-23 22:54:06 +00:00
Rafael Espindola
22e87bbb08 Delete Reloc::Default.
Having an enum member named Default is quite confusing: Is it distinct
from the others?

This patch removes that member and instead uses Optional<Reloc> in
places where we have a user input that still hasn't been maped to the
default value, which is now clear has no be one of the remaining 3
options.

llvm-svn: 269988
2016-05-18 22:04:49 +00:00
Mehdi Amini
b84ff32c91 ThinLTO: sort inputs and schedule by decreasing size
This is a compile time optimization: keeping a large file to process
at the end hurts parallelism.
The heurisitic used right now is the input buffer size, however we
may want to consider the number of functions to import or the
different number of files to load for importing as well.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 269684
2016-05-16 19:33:07 +00:00
Mehdi Amini
8334dfa2b3 ThinLTO caching: reload cached file with mmap and drop heap-allocated memory buffer
This is reducing pressure on the OS memory system, and is NFC
when not using a cache.
I measure a 10x memory consumption reduction when linking opt
with full debug info.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 269682
2016-05-16 19:11:59 +00:00