Tools working with object files on Darwin (e.g. lipo) may need to know
properties like the CPU type and subtype of a bitcode file. The logic of
converting a triple to a Mach-O CPU_(SUB_)TYPE should be provided by
LLVM instead of relying on tools to re-implement it.
Differential Revision: https://reviews.llvm.org/D75067
In order to correctly pass options to LLVM, including options containing
spaces which are used as delimiters for multiple options in
lto_codegen_debug_options, add a new API:
lto_codegen_debug_options_array.
Unfortunately, tools/lto has no testing infrastructure yet, so there are
no tests associated with this patch.
Differential Revision: https://reviews.llvm.org/D70463
Summary:
This is needed to implemented the same approach as lld (implemented in r338434)
for how to handling symbols that can be generated by LTO code generator
but not present in the symbol table for linker that uses legacy C APIs.
libLTO is in charge of providing the list of symbols. Linker is in
charge of implementing the eager loading from static libraries using
the list of symbols.
rdar://problem/52853974
Reviewers: tejohnson, bd1976llvm, deadalnix, espindola
Reviewed By: tejohnson
Subscribers: emaste, arichardson, hiraditya, MaskRay, dang, kledzik, mehdi_amini, inglorion, jkorous, dexonsmith, ributzka, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D67568
llvm-svn: 372021
Dependent libraries support for the legacy api was committed in a
broken state (see: https://reviews.llvm.org/D60274). This was missed
due to the painful nature of having to integrate the changes into a
linker in order to test. This change implements support for dependent
libraries in the legacy LTO api:
- I have removed the current api function, which returns a single
string, and added functions to access each dependent library
specifier individually.
- To reduce the testing pain, I have made the api functions as thin as
possible to maximize coverage from llvm-lto.
- When doing ThinLTO the system linker will load the modules lazily
when scanning the input files. Unfortunately, when modules are
lazily loaded there is no access to module level named metadata. To
fix this I have added api functions that allow querying the IRSymtab
for the dependent libraries. I hope to expand the api in the future
so that, eventually, all the information needed by a client linker
during scan can be retrieved from the IRSymtab.
Differential Revision: https://reviews.llvm.org/D62935
llvm-svn: 363140
The export file of libLTO should has all the interfaces declared in
llvm-c/lto.h and llvm-c/Disassembler.h but LLVMCreateDisasmCPUFeatures
is missing from the list. Export the C API to be consistant.
llvm-svn: 343124
The original was reverted due to an apparent build-bot test failure,
but it looks like this is just a flaky test.
Also added a C-interface function for large values, and updated
llvm-lto's --thinlto-cache-max-size-bytes switch to take a type larger
than int.
The maximum cache size in terms of bytes is a 64-bit number. However,
the methods to set it only took unsigned previously, which meant that
the maximum cache size could not be specified above 4GB. That's quite
small compared to the output of some projects, so it makes sense to
provide the ability to set larger values in that field.
We also needed a C-interface function that provides a greater range
than the existing thinlto_codegen_set_cache_size_bytes, which also only
takes an unsigned, so this change also adds
hinlto_codegen_set_cache_size_megabytes.
Reviewed by: mehdi_amini, tejohnson, steven_wu
Differential Revision: https://reviews.llvm.org/D52023
llvm-svn: 342366
Also added a C-interface function for large values, and updated
llvm-lto's --thinlto-cache-max-size-bytes switch to take a type larger
than int.
The maximum cache size in terms of bytes is a 64-bit number. However,
the methods to set it only took unsigned previously, which meant that
the maximum cache size could not be specified above 4GB. That's quite
small compared to the output of some projects, so it makes sense to
provide the ability to set larger values in that field.
We also needed a C-interface function that provides a greater range
than the existing thinlto_codegen_set_cache_size_bytes, which also only
takes an unsigned, so this change also adds
hinlto_codegen_set_cache_size_megabytes.
Reviewed by: mehdi_amini, tejohnson, steven_wu
Differential Revision: https://reviews.llvm.org/D52023
llvm-svn: 342233
- thinlto_codegen_set_cache_size_bytes to control the absolute size of cache directory.
- thinlto_codegen_set_cache_size_files the size and amount of files in cache directory.
These functions have been supported in C++ LTO API for a long time, but were absent in C LTO API.
Differential Revision: https://reviews.llvm.org/D42446
llvm-svn: 326537
Summary:
The motivation is to support better the -object_path_lto option on
Darwin. The linker needs to write down the generate object files on
disk for later use by lldb or dsymutil (debug info are not present
in the final binary). We're moving this into libLTO so that we can
be smarter when a cache is enabled and hard-link when possible
instead of duplicating the files.
Reviewers: tejohnson, deadalnix, pcc
Subscribers: dexonsmith, llvm-commits
Differential Revision: https://reviews.llvm.org/D27507
llvm-svn: 289631
The linker supports a feature to force load an object from a static
archive if it defines an Objective-C category.
This API supports this feature by looking at every section in the
module to find if a category is defined in the module.
llvm-svn: 275125
This allows the linker to instruct ThinLTO to perform only the
optimization part or only the codegen part of the process.
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265113
This is intended to provide a parallel (threaded) ThinLTO scheme
for linker plugin use through the libLTO C API.
The intent of this patch is to provide a first implementation as a
proof-of-concept and allows linker to start supporting ThinLTO by
definiing the libLTO C API. Some part of the libLTO API are left
unimplemented yet. Following patches will add support for these.
The current implementation can link all clang/llvm binaries.
Differential Revision: http://reviews.llvm.org/D17066
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 262977
Specifically, remove the dependent library interface and replace the existing
linker option interface with a new one that returns a single list of flags.
Differential Revision: http://reviews.llvm.org/D10820
llvm-svn: 241018
Summary:
lto_codegen_set_should_embed_uselists is introduced in r235943 but not
added to lto.exports. Add to export list to expose the API.
Reviewers: dexonsmith
Subscribers: rafael, llvm-commits
Differential Revision: http://reviews.llvm.org/D10658
llvm-svn: 240442
When debugging LTO issues with ld64, we use -save-temps to save the merged
optimized bitcode file, then invoke ld64 again on the single bitcode file.
The saved bitcode file is already internalized, so we can call
lto_codegen_set_should_internalize and skip running internalization again.
rdar://20227235
llvm-svn: 235211
When debugging LTO issues with ld64, we use -save-temps to save the merged
optimized bitcode file, then invoke ld64 again on the single bitcode file to
speed up debugging code generation passes and ld64 stuff after code generation.
llvm linking a single bitcode file via lto_codegen_add_module will generate a
different bitcode file from the single input. With the newly-added
lto_codegen_set_module, we can make sure the destination module is the same as
the input.
lto_codegen_set_module will transfer the ownship of the module to code
generator.
rdar://19024554
llvm-svn: 230290
lto_codegen_compile_optimized. Also add lto_api_version.
Before this commit, we can only dump the optimized bitcode after running
lto_codegen_compile, but it includes some impacts of running codegen passes,
one example is StackProtector pass. We will get assertion failure when running
llc on the optimized bitcode, because StackProtector is effectively run twice.
After splitting lto_codegen_compile, the linker can choose to dump the bitcode
before running lto_codegen_compile_optimized.
lto_api_version is added so ld64 can check for runtime-availability of the new
API.
rdar://19565500
llvm-svn: 228000
Export symbols in libLTO.dylib for the local context-related functions
added in r221733 (`LTO_API_VERSION=11`)... and add the missing
definition for `lto_codegen_create_in_local_context()`.
llvm-svn: 224567
This function adds an extra path argument to lto_module_create_from_memory.
The path argument will be passed to makeBuffer to make sure the MemoryBuffer
has a name and the created module has a module identifier.
This is mainly for emitting warning messages from the linker. When we emit
warning message on a module, we can use the module identifier.
rdar://15985737
llvm-svn: 201114
Add a hook in the C API of LTO so that clients of the code generator can set
their own handler for the LLVM diagnostics.
The handler is defined like this:
typedef void (*lto_diagnostic_handler_t)(lto_codegen_diagnostic_severity_t
severity, const char *diag, void *ctxt)
- severity says how bad this is.
- diag is a string that contains the diagnostic message.
- ctxt is the registered context for this handler.
This hook is more general than the lto_get_error_message, since this function
keeps only the latest message and can only be queried when something went wrong
(no warning for instance).
<rdar://problem/15517596>
llvm-svn: 199338
There are two ways one could implement hiding of linkonce_odr symbols in LTO:
* LLVM tells the linker which symbols can be hidden if not used from native
files.
* The linker tells LLVM which symbols are not used from other object files,
but will be put in the dso symbol table if present.
GOLD's API is the second option. It was implemented almost 1:1 in llvm by
passing the list down to internalize.
LLVM already had partial support for the first option. It is also very similar
to how ld64 handles hiding these symbols when *not* doing LTO.
This patch then
* removes the APIs for the DSO list.
* marks LTO_SYMBOL_SCOPE_DEFAULT_CAN_BE_HIDDEN all linkonce_odr unnamed_addr
global values and other linkonce_odr whose address is not used.
* makes the gold plugin responsible for handling the API mismatch.
llvm-svn: 193800
----
Add new API lto_codegen_compile_parallel().
This API is proposed by Nick Kledzik. The semantic is:
--------------------------------------------------------------------------
Generate code for merged module into an array of native object files. On
success returns a pointer to an array of NativeObjectFile. The count
parameter returns the number of elements in the array. Each element is
a pointer/length for a generated mach-o/ELF buffer. The buffer is owned
by the lto_code_gen_t and will be freed when lto_codegen_dispose() is called,
or lto_codegen_compile() is called again. On failure, returns NULL
(check lto_get_error_message() for details).
extern const struct NativeObjectFile*
lto_codegen_compile_parallel(lto_code_gen_t cg, size_t *count);
---------------------------------------------------------------------------
This API is currently only called on OSX platform. Linux or other Unixes
using GNU gold are not supposed to call this function, because on these systems,
object files are fed back to linker via disk file instead of memory buffer.
In this commit, lto_codegen_compile_parallel() simply calls
lto_codegen_compile() to return a single object file. In the near future,
this function is the entry point for compilation with partition. Linker can
blindly call this function even if partition is turned off; in this case,
compiler will return only one object file.
llvm-svn: 189386
This API is proposed by Nick Kledzik. The semantic is:
--------------------------------------------------------------------------
Generate code for merged module into an array of native object files. On
success returns a pointer to an array of NativeObjectFile. The count
parameter returns the number of elements in the array. Each element is
a pointer/length for a generated mach-o/ELF buffer. The buffer is owned
by the lto_code_gen_t and will be freed when lto_codegen_dispose() is called,
or lto_codegen_compile() is called again. On failure, returns NULL
(check lto_get_error_message() for details).
extern const struct NativeObjectFile*
lto_codegen_compile_parallel(lto_code_gen_t cg, size_t *count);
---------------------------------------------------------------------------
This API is currently only called on OSX platform. Linux or other Unixes
using GNU gold are not supposed to call this function, because on these systems,
object files are fed back to linker via disk file instead of memory buffer.
In this commit, lto_codegen_compile_parallel() simply calls
lto_codegen_compile() to return a single object file. In the near future,
this function is the entry point for compilation with partition. Linker can
blindly call this function even if partition is turned off; in this case,
compiler will return only one object file.
llvm-svn: 189297
In order to appease people (in Apple) who accuse me for committing "huge change" (?) without proper review.
Thank Eric for fixing a compile-warning.
llvm-svn: 188204
1. Add some helper classes for partitions. They are designed in a
way such that the top-level LTO driver will not see much difference
with or without partitioning.
2. Introduce work-dir. Now all intermediate files generated during
LTO phases will be saved under work-dir. User can specify the workdir
via -lto-workdir=/path/to/dir. By default the work-dir will be
erased before linker exit. To keep the workdir, do -lto-keep, or -lto-keep=1.
TODO: Erase the workdir, if the linker exit prematurely.
We are currently not able to remove directory on signal. The support
routines simply ignore directory.
3. Add one new API lto_codegen_get_files_need_remove().
Linker and LTO plugin will communicate via this API about which files
(including directories) need to removed before linker exit.
llvm-svn: 188188
The linker will call `lto_codegen_add_must_preserve_symbol' on all globals that
should be kept around. The linker will pretend that a dylib is being created.
<rdar://problem/12528059>
llvm-svn: 169770
This function sets the `_exportDynamic' ivar. When that's set, we export all
symbols (e.g. we don't run the internalize pass). This is equivalent to the
`--export-dynamic' linker flag in GNU land:
--export-dynamic
When creating a dynamically linked executable, add all symbols to the dynamic
symbol table. The dynamic symbol table is the set of symbols which are visible
from dynamic objects at run time. If you do not use this option, the dynamic
symbol table will normally contain only those symbols which are referenced by
some dynamic object mentioned in the link. If you use dlopen to load a dynamic
object which needs to refer back to the symbols defined by the program, rather
than some other dynamic object, then you will probably need to use this option
when linking the program itself.
The Darwin linker will support this via the `-export_dynamic' flag. We should
modify clang to support this via the `-rdynamic' flag.
llvm-svn: 169656
Necessary to give disassembler users (like darwin's otool) a possibility to
dlopen libLTO and still initialize the required LLVM bits. This used to go
through libMCDisassembler but that's a gross layering violation, the MC layer
can't pull in functions from the targets. Adding a function to libLTO is a bit
of a hack but not worse than exposing other disassembler bits from libLTO.
Fixes PR14362.
llvm-svn: 168545
Per the October 12, 2012 Proposal for annotated disassembly output sent out by
Jim Grosbach this set of changes implements this for X86 and arm. The llvm-mc
tool now has a -mdis option to produced the marked up disassembly and a couple
of small example test cases have been added.
rdar://11764962
llvm-svn: 166445
so we don't want it to show up in the stable 3.1 interface.
While at it, add a comment about why LTOCodeGenerator manually creates the
internalize pass.
llvm-svn: 154807
native linking export files, including running sed to prepend underscores
on darwin, and make use of it in libLTO and libEnhancedDisassembly.
Remove the leading underscores from library export files so that they
work with the new EXPORTED_SYMBOL_FILE support.
llvm-svn: 101399
Add lto_codegen_set_assembler_path() API which allows the linker to specify the
path to the assembler tool to run. When assembler is used (instead of compiler)
different command line options are used.
Add LTO_API_VERSION #define so clients (linkers) can conditionalize use of new APIs.
llvm-svn: 72823
