RPCS3/llvm-mirror
1
0
Fork 0
mirror of https://github.com/RPCS3/llvm-mirror.git synced 2024-11-23 19:23:23 +01:00
Code Issues Projects Releases Wiki Activity
747bfa048d
llvm-mirror/test/ThinLTO/X86/distributed_import.ll

97 lines
3.7 KiB
LLVM
Raw Normal View History

[ThinLTO] Add support for emitting minimized bitcode for thin link Summary: The cumulative size of the bitcode files for a very large application can be huge, particularly with -g. In a distributed build environment, all of these files must be sent to the remote build node that performs the thin link step, and this can exceed size limits. The thin link actually only needs the summary along with a bitcode symbol table. Until we have a proper bitcode symbol table, simply stripping the debug metadata results in significant size reduction. Add support for an option to additionally emit minimized bitcode modules, just for use in the thin link step, which for now just strips all debug metadata. I plan to add a cc1 option so this can be invoked easily during the compile step. However, care must be taken to ensure that these minimized thin link bitcode files produce the same index as with the original bitcode files, as these original bitcode files will be used in the backends. Specifically: 1) The module hash used for caching is typically produced by hashing the written bitcode, and we want to include the hash that would correspond to the original bitcode file. This is because we want to ensure that changes in the stripped portions affect caching. Added plumbing to emit the same module hash in the minimized thin link bitcode file. 2) The module paths in the index are constructed from the module ID of each thin linked bitcode, and typically is automatically generated from the input file path. This is the path used for finding the modules to import from, and obviously we need this to point to the original bitcode files. Added gold-plugin support to take a suffix replacement during the thin link that is used to override the identifier on the MemoryBufferRef constructed from the loaded thin link bitcode file. The assumption is that the build system can specify that the minimized bitcode file has a name that is similar but uses a different suffix (e.g. out.thinlink.bc instead of out.o). Added various tests to ensure that we get identical index files out of the thin link step. Reviewers: mehdi_amini, pcc Subscribers: Prazek, llvm-commits Differential Revision: https://reviews.llvm.org/D31027 llvm-svn: 298638
2017-03-23 20:47:39 +01:00
; Test distributed build thin link output from llvm-lto2
[ThinLTO] Handle distributed backend case when doing renaming Summary: The recent change I made to consult the summary when deciding whether to rename (to handle inline asm) in r285513 broke the distributed build case. In a distributed backend we will only have a portion of the combined index, specifically for imported modules we only have the summaries for any imported definitions. When renaming on import we were asserting because no summary entry was found for a local reference being linked in (def wasn't imported). We only need to consult the summary for a renaming decision for the exporting module. For imports, we would have prevented importing any references to NoRename values already. Reviewers: mehdi_amini Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D26250 llvm-svn: 285871
2016-11-03 02:07:16 +01:00
[ThinLTO] Add support for emitting minimized bitcode for thin link Summary: The cumulative size of the bitcode files for a very large application can be huge, particularly with -g. In a distributed build environment, all of these files must be sent to the remote build node that performs the thin link step, and this can exceed size limits. The thin link actually only needs the summary along with a bitcode symbol table. Until we have a proper bitcode symbol table, simply stripping the debug metadata results in significant size reduction. Add support for an option to additionally emit minimized bitcode modules, just for use in the thin link step, which for now just strips all debug metadata. I plan to add a cc1 option so this can be invoked easily during the compile step. However, care must be taken to ensure that these minimized thin link bitcode files produce the same index as with the original bitcode files, as these original bitcode files will be used in the backends. Specifically: 1) The module hash used for caching is typically produced by hashing the written bitcode, and we want to include the hash that would correspond to the original bitcode file. This is because we want to ensure that changes in the stripped portions affect caching. Added plumbing to emit the same module hash in the minimized thin link bitcode file. 2) The module paths in the index are constructed from the module ID of each thin linked bitcode, and typically is automatically generated from the input file path. This is the path used for finding the modules to import from, and obviously we need this to point to the original bitcode files. Added gold-plugin support to take a suffix replacement during the thin link that is used to override the identifier on the MemoryBufferRef constructed from the loaded thin link bitcode file. The assumption is that the build system can specify that the minimized bitcode file has a name that is similar but uses a different suffix (e.g. out.thinlink.bc instead of out.o). Added various tests to ensure that we get identical index files out of the thin link step. Reviewers: mehdi_amini, pcc Subscribers: Prazek, llvm-commits Differential Revision: https://reviews.llvm.org/D31027 llvm-svn: 298638
2017-03-23 20:47:39 +01:00
; Generate bitcode files with summary, as well as minimized bitcode without
; the debug metadata for the thin link.
; RUN: opt -thinlto-bc %s -thin-link-bitcode-file=%t1.thinlink.bc -o %t1.bc
; RUN: opt -thinlto-bc %p/Inputs/distributed_import.ll -thin-link-bitcode-file=%t2.thinlink.bc -o %t2.bc
ThinLTO Minimized Bitcode File Size Reduction Summary: Currently the ThinLTO minimized bitcode file only strip the debug info, but there is still a lot of information in the minimized bit code file that will be not used for thin linker. In this patch, most of the extra information is striped to reduce the minimized bitcode file. Now only ModuleVersion, ModuleInfo, ModuleGlobalValueSummary, ModuleHash, Symtab and Strtab are left. Now the minimized bitcode file size is reduced to 15%-30% of the debug info stripped bitcode file size. Reviewers: danielcdh, tejohnson, pcc Reviewed By: pcc Subscribers: mehdi_amini, aprantl, inglorion, eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D35334 llvm-svn: 308760
2017-07-21 19:25:20 +02:00
; RUN: llvm-bcanalyzer -dump %t1.thinlink.bc | FileCheck --check-prefix=THINLINKBITCODE %s
[ThinLTO] Add support for emitting minimized bitcode for thin link Summary: The cumulative size of the bitcode files for a very large application can be huge, particularly with -g. In a distributed build environment, all of these files must be sent to the remote build node that performs the thin link step, and this can exceed size limits. The thin link actually only needs the summary along with a bitcode symbol table. Until we have a proper bitcode symbol table, simply stripping the debug metadata results in significant size reduction. Add support for an option to additionally emit minimized bitcode modules, just for use in the thin link step, which for now just strips all debug metadata. I plan to add a cc1 option so this can be invoked easily during the compile step. However, care must be taken to ensure that these minimized thin link bitcode files produce the same index as with the original bitcode files, as these original bitcode files will be used in the backends. Specifically: 1) The module hash used for caching is typically produced by hashing the written bitcode, and we want to include the hash that would correspond to the original bitcode file. This is because we want to ensure that changes in the stripped portions affect caching. Added plumbing to emit the same module hash in the minimized thin link bitcode file. 2) The module paths in the index are constructed from the module ID of each thin linked bitcode, and typically is automatically generated from the input file path. This is the path used for finding the modules to import from, and obviously we need this to point to the original bitcode files. Added gold-plugin support to take a suffix replacement during the thin link that is used to override the identifier on the MemoryBufferRef constructed from the loaded thin link bitcode file. The assumption is that the build system can specify that the minimized bitcode file has a name that is similar but uses a different suffix (e.g. out.thinlink.bc instead of out.o). Added various tests to ensure that we get identical index files out of the thin link step. Reviewers: mehdi_amini, pcc Subscribers: Prazek, llvm-commits Differential Revision: https://reviews.llvm.org/D31027 llvm-svn: 298638
2017-03-23 20:47:39 +01:00
; First perform the thin link on the normal bitcode file.
llvm-lto2: Move the LTO::run() action behind a subcommand. Move LTO::run() to a "run" subcommand so that we can introduce new subcommands for testing different parts of the LTO implementation. This doesn't use llvm::cl subcommands because it doesn't appear to be currently possible to pass an argument not associated with a subcommand to a subcommand (e.g. -lto-use-new-pm, -mcpu=yonah). Differential Revision: https://reviews.llvm.org/D31410 llvm-svn: 299967
2017-04-11 20:12:00 +02:00
; RUN: llvm-lto2 run %t1.bc %t2.bc -o %t.o -save-temps \
[ThinLTO] Handle distributed backend case when doing renaming Summary: The recent change I made to consult the summary when deciding whether to rename (to handle inline asm) in r285513 broke the distributed build case. In a distributed backend we will only have a portion of the combined index, specifically for imported modules we only have the summaries for any imported definitions. When renaming on import we were asserting because no summary entry was found for a local reference being linked in (def wasn't imported). We only need to consult the summary for a renaming decision for the exporting module. For imports, we would have prevented importing any references to NoRename values already. Reviewers: mehdi_amini Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D26250 llvm-svn: 285871
2016-11-03 02:07:16 +01:00
; RUN: -thinlto-distributed-indexes \
; RUN: -r=%t1.bc,g, \
[ThinLTO] Enable importing of aliases as copy of aliasee Summary: This implements a missing feature to allow importing of aliases, which was previously disabled because alias cannot be available_externally. We instead import an alias as a copy of its aliasee. Some additional work was required in the IndexBitcodeWriter for the distributed build case, to ensure that the aliasee has a value id in the distributed index file (i.e. even when it is not being imported directly). This is a performance win in codes that have many aliases, e.g. C++ applications that have many constructor and destructor aliases. Reviewers: pcc Subscribers: mehdi_amini, inglorion, eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D40747 llvm-svn: 320895
2017-12-16 01:18:12 +01:00
; RUN: -r=%t1.bc,analias, \
[ThinLTO] Handle distributed backend case when doing renaming Summary: The recent change I made to consult the summary when deciding whether to rename (to handle inline asm) in r285513 broke the distributed build case. In a distributed backend we will only have a portion of the combined index, specifically for imported modules we only have the summaries for any imported definitions. When renaming on import we were asserting because no summary entry was found for a local reference being linked in (def wasn't imported). We only need to consult the summary for a renaming decision for the exporting module. For imports, we would have prevented importing any references to NoRename values already. Reviewers: mehdi_amini Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D26250 llvm-svn: 285871
2016-11-03 02:07:16 +01:00
; RUN: -r=%t1.bc,f,px \
[ThinLTO] Enable importing of aliases as copy of aliasee Summary: This implements a missing feature to allow importing of aliases, which was previously disabled because alias cannot be available_externally. We instead import an alias as a copy of its aliasee. Some additional work was required in the IndexBitcodeWriter for the distributed build case, to ensure that the aliasee has a value id in the distributed index file (i.e. even when it is not being imported directly). This is a performance win in codes that have many aliases, e.g. C++ applications that have many constructor and destructor aliases. Reviewers: pcc Subscribers: mehdi_amini, inglorion, eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D40747 llvm-svn: 320895
2017-12-16 01:18:12 +01:00
; RUN: -r=%t2.bc,g,px \
; RUN: -r=%t2.bc,analias,px \
; RUN: -r=%t2.bc,aliasee,px
Add another missing -enable-import-metadata to test r320895 modified a test so that it needs -enable-import-metadata which is false by default for NDEBUG, found another place that needs this added. llvm-svn: 320903
2017-12-16 02:35:36 +01:00
; RUN: opt -function-import -import-all-index -enable-import-metadata -summary-file %t1.bc.thinlto.bc %t1.bc -o %t1.out
[ThinLTO] Enable importing of aliases as copy of aliasee Summary: This implements a missing feature to allow importing of aliases, which was previously disabled because alias cannot be available_externally. We instead import an alias as a copy of its aliasee. Some additional work was required in the IndexBitcodeWriter for the distributed build case, to ensure that the aliasee has a value id in the distributed index file (i.e. even when it is not being imported directly). This is a performance win in codes that have many aliases, e.g. C++ applications that have many constructor and destructor aliases. Reviewers: pcc Subscribers: mehdi_amini, inglorion, eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D40747 llvm-svn: 320895
2017-12-16 01:18:12 +01:00
; RUN: opt -function-import -import-all-index -summary-file %t2.bc.thinlto.bc %t2.bc -o %t2.out
; RUN: llvm-dis -o - %t1.out | FileCheck %s --check-prefix=IMPORT
; RUN: llvm-dis -o - %t2.out | FileCheck %s --check-prefix=EXPORT
[ThinLTO] Add support for emitting minimized bitcode for thin link Summary: The cumulative size of the bitcode files for a very large application can be huge, particularly with -g. In a distributed build environment, all of these files must be sent to the remote build node that performs the thin link step, and this can exceed size limits. The thin link actually only needs the summary along with a bitcode symbol table. Until we have a proper bitcode symbol table, simply stripping the debug metadata results in significant size reduction. Add support for an option to additionally emit minimized bitcode modules, just for use in the thin link step, which for now just strips all debug metadata. I plan to add a cc1 option so this can be invoked easily during the compile step. However, care must be taken to ensure that these minimized thin link bitcode files produce the same index as with the original bitcode files, as these original bitcode files will be used in the backends. Specifically: 1) The module hash used for caching is typically produced by hashing the written bitcode, and we want to include the hash that would correspond to the original bitcode file. This is because we want to ensure that changes in the stripped portions affect caching. Added plumbing to emit the same module hash in the minimized thin link bitcode file. 2) The module paths in the index are constructed from the module ID of each thin linked bitcode, and typically is automatically generated from the input file path. This is the path used for finding the modules to import from, and obviously we need this to point to the original bitcode files. Added gold-plugin support to take a suffix replacement during the thin link that is used to override the identifier on the MemoryBufferRef constructed from the loaded thin link bitcode file. The assumption is that the build system can specify that the minimized bitcode file has a name that is similar but uses a different suffix (e.g. out.thinlink.bc instead of out.o). Added various tests to ensure that we get identical index files out of the thin link step. Reviewers: mehdi_amini, pcc Subscribers: Prazek, llvm-commits Differential Revision: https://reviews.llvm.org/D31027 llvm-svn: 298638
2017-03-23 20:47:39 +01:00
; Save the generated index files.
; RUN: cp %t1.bc.thinlto.bc %t1.bc.thinlto.bc.orig
; RUN: cp %t2.bc.thinlto.bc %t2.bc.thinlto.bc.orig
; Copy the minimized bitcode to the regular bitcode path so the module
; paths in the index are the same (save the regular bitcode for use again
; further down).
; RUN: cp %t1.bc %t1.bc.sv
; RUN: cp %t1.thinlink.bc %t1.bc
; RUN: cp %t2.bc %t2.bc.sv
; RUN: cp %t2.thinlink.bc %t2.bc
; Next perform the thin link on the minimized bitcode files, and compare dumps
; of the resulting indexes to the above dumps to ensure they are identical.
; RUN: rm -f %t1.bc.thinlto.bc %t2.bc.thinlto.bc
llvm-lto2: Move the LTO::run() action behind a subcommand. Move LTO::run() to a "run" subcommand so that we can introduce new subcommands for testing different parts of the LTO implementation. This doesn't use llvm::cl subcommands because it doesn't appear to be currently possible to pass an argument not associated with a subcommand to a subcommand (e.g. -lto-use-new-pm, -mcpu=yonah). Differential Revision: https://reviews.llvm.org/D31410 llvm-svn: 299967
2017-04-11 20:12:00 +02:00
; RUN: llvm-lto2 run %t1.bc %t2.bc -o %t.o -save-temps \
[ThinLTO] Add support for emitting minimized bitcode for thin link Summary: The cumulative size of the bitcode files for a very large application can be huge, particularly with -g. In a distributed build environment, all of these files must be sent to the remote build node that performs the thin link step, and this can exceed size limits. The thin link actually only needs the summary along with a bitcode symbol table. Until we have a proper bitcode symbol table, simply stripping the debug metadata results in significant size reduction. Add support for an option to additionally emit minimized bitcode modules, just for use in the thin link step, which for now just strips all debug metadata. I plan to add a cc1 option so this can be invoked easily during the compile step. However, care must be taken to ensure that these minimized thin link bitcode files produce the same index as with the original bitcode files, as these original bitcode files will be used in the backends. Specifically: 1) The module hash used for caching is typically produced by hashing the written bitcode, and we want to include the hash that would correspond to the original bitcode file. This is because we want to ensure that changes in the stripped portions affect caching. Added plumbing to emit the same module hash in the minimized thin link bitcode file. 2) The module paths in the index are constructed from the module ID of each thin linked bitcode, and typically is automatically generated from the input file path. This is the path used for finding the modules to import from, and obviously we need this to point to the original bitcode files. Added gold-plugin support to take a suffix replacement during the thin link that is used to override the identifier on the MemoryBufferRef constructed from the loaded thin link bitcode file. The assumption is that the build system can specify that the minimized bitcode file has a name that is similar but uses a different suffix (e.g. out.thinlink.bc instead of out.o). Added various tests to ensure that we get identical index files out of the thin link step. Reviewers: mehdi_amini, pcc Subscribers: Prazek, llvm-commits Differential Revision: https://reviews.llvm.org/D31027 llvm-svn: 298638
2017-03-23 20:47:39 +01:00
; RUN: -thinlto-distributed-indexes \
; RUN: -r=%t1.bc,g, \
[ThinLTO] Enable importing of aliases as copy of aliasee Summary: This implements a missing feature to allow importing of aliases, which was previously disabled because alias cannot be available_externally. We instead import an alias as a copy of its aliasee. Some additional work was required in the IndexBitcodeWriter for the distributed build case, to ensure that the aliasee has a value id in the distributed index file (i.e. even when it is not being imported directly). This is a performance win in codes that have many aliases, e.g. C++ applications that have many constructor and destructor aliases. Reviewers: pcc Subscribers: mehdi_amini, inglorion, eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D40747 llvm-svn: 320895
2017-12-16 01:18:12 +01:00
; RUN: -r=%t1.bc,analias, \
[ThinLTO] Add support for emitting minimized bitcode for thin link Summary: The cumulative size of the bitcode files for a very large application can be huge, particularly with -g. In a distributed build environment, all of these files must be sent to the remote build node that performs the thin link step, and this can exceed size limits. The thin link actually only needs the summary along with a bitcode symbol table. Until we have a proper bitcode symbol table, simply stripping the debug metadata results in significant size reduction. Add support for an option to additionally emit minimized bitcode modules, just for use in the thin link step, which for now just strips all debug metadata. I plan to add a cc1 option so this can be invoked easily during the compile step. However, care must be taken to ensure that these minimized thin link bitcode files produce the same index as with the original bitcode files, as these original bitcode files will be used in the backends. Specifically: 1) The module hash used for caching is typically produced by hashing the written bitcode, and we want to include the hash that would correspond to the original bitcode file. This is because we want to ensure that changes in the stripped portions affect caching. Added plumbing to emit the same module hash in the minimized thin link bitcode file. 2) The module paths in the index are constructed from the module ID of each thin linked bitcode, and typically is automatically generated from the input file path. This is the path used for finding the modules to import from, and obviously we need this to point to the original bitcode files. Added gold-plugin support to take a suffix replacement during the thin link that is used to override the identifier on the MemoryBufferRef constructed from the loaded thin link bitcode file. The assumption is that the build system can specify that the minimized bitcode file has a name that is similar but uses a different suffix (e.g. out.thinlink.bc instead of out.o). Added various tests to ensure that we get identical index files out of the thin link step. Reviewers: mehdi_amini, pcc Subscribers: Prazek, llvm-commits Differential Revision: https://reviews.llvm.org/D31027 llvm-svn: 298638
2017-03-23 20:47:39 +01:00
; RUN: -r=%t1.bc,f,px \
[ThinLTO] Enable importing of aliases as copy of aliasee Summary: This implements a missing feature to allow importing of aliases, which was previously disabled because alias cannot be available_externally. We instead import an alias as a copy of its aliasee. Some additional work was required in the IndexBitcodeWriter for the distributed build case, to ensure that the aliasee has a value id in the distributed index file (i.e. even when it is not being imported directly). This is a performance win in codes that have many aliases, e.g. C++ applications that have many constructor and destructor aliases. Reviewers: pcc Subscribers: mehdi_amini, inglorion, eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D40747 llvm-svn: 320895
2017-12-16 01:18:12 +01:00
; RUN: -r=%t2.bc,g,px \
; RUN: -r=%t2.bc,analias,px \
; RUN: -r=%t2.bc,aliasee,px
[ThinLTO] Add support for emitting minimized bitcode for thin link Summary: The cumulative size of the bitcode files for a very large application can be huge, particularly with -g. In a distributed build environment, all of these files must be sent to the remote build node that performs the thin link step, and this can exceed size limits. The thin link actually only needs the summary along with a bitcode symbol table. Until we have a proper bitcode symbol table, simply stripping the debug metadata results in significant size reduction. Add support for an option to additionally emit minimized bitcode modules, just for use in the thin link step, which for now just strips all debug metadata. I plan to add a cc1 option so this can be invoked easily during the compile step. However, care must be taken to ensure that these minimized thin link bitcode files produce the same index as with the original bitcode files, as these original bitcode files will be used in the backends. Specifically: 1) The module hash used for caching is typically produced by hashing the written bitcode, and we want to include the hash that would correspond to the original bitcode file. This is because we want to ensure that changes in the stripped portions affect caching. Added plumbing to emit the same module hash in the minimized thin link bitcode file. 2) The module paths in the index are constructed from the module ID of each thin linked bitcode, and typically is automatically generated from the input file path. This is the path used for finding the modules to import from, and obviously we need this to point to the original bitcode files. Added gold-plugin support to take a suffix replacement during the thin link that is used to override the identifier on the MemoryBufferRef constructed from the loaded thin link bitcode file. The assumption is that the build system can specify that the minimized bitcode file has a name that is similar but uses a different suffix (e.g. out.thinlink.bc instead of out.o). Added various tests to ensure that we get identical index files out of the thin link step. Reviewers: mehdi_amini, pcc Subscribers: Prazek, llvm-commits Differential Revision: https://reviews.llvm.org/D31027 llvm-svn: 298638
2017-03-23 20:47:39 +01:00
; RUN: diff %t1.bc.thinlto.bc.orig %t1.bc.thinlto.bc
; RUN: diff %t2.bc.thinlto.bc.orig %t2.bc.thinlto.bc
; Make sure importing occurs as expected
; RUN: cp %t1.bc.sv %t1.bc
; RUN: cp %t2.bc.sv %t2.bc
Add -enable-import-metadata to test r320895 modified a test so that it needs -enable-import-metadata which is false by default for NDEBUG. llvm-svn: 320899
2017-12-16 02:00:48 +01:00
; RUN: opt -function-import -import-all-index -enable-import-metadata -summary-file %t1.bc.thinlto.bc %t1.bc -o %t1.out
[ThinLTO] Enable importing of aliases as copy of aliasee Summary: This implements a missing feature to allow importing of aliases, which was previously disabled because alias cannot be available_externally. We instead import an alias as a copy of its aliasee. Some additional work was required in the IndexBitcodeWriter for the distributed build case, to ensure that the aliasee has a value id in the distributed index file (i.e. even when it is not being imported directly). This is a performance win in codes that have many aliases, e.g. C++ applications that have many constructor and destructor aliases. Reviewers: pcc Subscribers: mehdi_amini, inglorion, eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D40747 llvm-svn: 320895
2017-12-16 01:18:12 +01:00
; RUN: opt -function-import -import-all-index -summary-file %t2.bc.thinlto.bc %t2.bc -o %t2.out
; RUN: llvm-dis -o - %t1.out | FileCheck %s --check-prefix=IMPORT
; RUN: llvm-dis -o - %t2.out | FileCheck %s --check-prefix=EXPORT
[ThinLTO] Add support for emitting minimized bitcode for thin link Summary: The cumulative size of the bitcode files for a very large application can be huge, particularly with -g. In a distributed build environment, all of these files must be sent to the remote build node that performs the thin link step, and this can exceed size limits. The thin link actually only needs the summary along with a bitcode symbol table. Until we have a proper bitcode symbol table, simply stripping the debug metadata results in significant size reduction. Add support for an option to additionally emit minimized bitcode modules, just for use in the thin link step, which for now just strips all debug metadata. I plan to add a cc1 option so this can be invoked easily during the compile step. However, care must be taken to ensure that these minimized thin link bitcode files produce the same index as with the original bitcode files, as these original bitcode files will be used in the backends. Specifically: 1) The module hash used for caching is typically produced by hashing the written bitcode, and we want to include the hash that would correspond to the original bitcode file. This is because we want to ensure that changes in the stripped portions affect caching. Added plumbing to emit the same module hash in the minimized thin link bitcode file. 2) The module paths in the index are constructed from the module ID of each thin linked bitcode, and typically is automatically generated from the input file path. This is the path used for finding the modules to import from, and obviously we need this to point to the original bitcode files. Added gold-plugin support to take a suffix replacement during the thin link that is used to override the identifier on the MemoryBufferRef constructed from the loaded thin link bitcode file. The assumption is that the build system can specify that the minimized bitcode file has a name that is similar but uses a different suffix (e.g. out.thinlink.bc instead of out.o). Added various tests to ensure that we get identical index files out of the thin link step. Reviewers: mehdi_amini, pcc Subscribers: Prazek, llvm-commits Differential Revision: https://reviews.llvm.org/D31027 llvm-svn: 298638
2017-03-23 20:47:39 +01:00
[ThinLTO] Enable importing of aliases as copy of aliasee Summary: This implements a missing feature to allow importing of aliases, which was previously disabled because alias cannot be available_externally. We instead import an alias as a copy of its aliasee. Some additional work was required in the IndexBitcodeWriter for the distributed build case, to ensure that the aliasee has a value id in the distributed index file (i.e. even when it is not being imported directly). This is a performance win in codes that have many aliases, e.g. C++ applications that have many constructor and destructor aliases. Reviewers: pcc Subscribers: mehdi_amini, inglorion, eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D40747 llvm-svn: 320895
2017-12-16 01:18:12 +01:00
; IMPORT: define available_externally i32 @g() !thinlto_src_module
; IMPORT: define available_externally void @analias() !thinlto_src_module
; EXPORT: @G.llvm.
[ThinLTO] Handle distributed backend case when doing renaming Summary: The recent change I made to consult the summary when deciding whether to rename (to handle inline asm) in r285513 broke the distributed build case. In a distributed backend we will only have a portion of the combined index, specifically for imported modules we only have the summaries for any imported definitions. When renaming on import we were asserting because no summary entry was found for a local reference being linked in (def wasn't imported). We only need to consult the summary for a renaming decision for the exporting module. For imports, we would have prevented importing any references to NoRename values already. Reviewers: mehdi_amini Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D26250 llvm-svn: 285871
2016-11-03 02:07:16 +01:00
[LTO] Make processing of combined module more consistent Summary: 1. Use stream 0 only for combined module. Previously if combined module was not processes ThinLTO used the stream for own output. However small changes in input, could trigger combined module and shuffle outputs making life of llvm::LTO harder. 2. Always process combined module and write output to stream 0. Processing empty combined module is cheap and allows llvm::LTO users to avoid implementing processing which is already done in llvm::LTO. Subscribers: mehdi_amini, inglorion, eraman, hiraditya Differential Revision: https://reviews.llvm.org/D41267 llvm-svn: 320905
2017-12-16 03:10:00 +01:00
target triple = "x86_64-unknown-linux-gnu"
[LTO] Reject modules without datalayout. Also, udpate the ~60 failing tests in the tree which did not contain a valid datalayout. This fixes PR31123. lld will be updated in a following patch, immediately after this is committed. Differential Revision: https://reviews.llvm.org/D27082 llvm-svn: 289719
2016-12-14 22:57:04 +01:00
target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
[ThinLTO] Handle distributed backend case when doing renaming Summary: The recent change I made to consult the summary when deciding whether to rename (to handle inline asm) in r285513 broke the distributed build case. In a distributed backend we will only have a portion of the combined index, specifically for imported modules we only have the summaries for any imported definitions. When renaming on import we were asserting because no summary entry was found for a local reference being linked in (def wasn't imported). We only need to consult the summary for a renaming decision for the exporting module. For imports, we would have prevented importing any references to NoRename values already. Reviewers: mehdi_amini Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D26250 llvm-svn: 285871
2016-11-03 02:07:16 +01:00
declare i32 @g(...)
[ThinLTO] Enable importing of aliases as copy of aliasee Summary: This implements a missing feature to allow importing of aliases, which was previously disabled because alias cannot be available_externally. We instead import an alias as a copy of its aliasee. Some additional work was required in the IndexBitcodeWriter for the distributed build case, to ensure that the aliasee has a value id in the distributed index file (i.e. even when it is not being imported directly). This is a performance win in codes that have many aliases, e.g. C++ applications that have many constructor and destructor aliases. Reviewers: pcc Subscribers: mehdi_amini, inglorion, eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D40747 llvm-svn: 320895
2017-12-16 01:18:12 +01:00
declare void @analias(...)
[ThinLTO] Handle distributed backend case when doing renaming Summary: The recent change I made to consult the summary when deciding whether to rename (to handle inline asm) in r285513 broke the distributed build case. In a distributed backend we will only have a portion of the combined index, specifically for imported modules we only have the summaries for any imported definitions. When renaming on import we were asserting because no summary entry was found for a local reference being linked in (def wasn't imported). We only need to consult the summary for a renaming decision for the exporting module. For imports, we would have prevented importing any references to NoRename values already. Reviewers: mehdi_amini Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D26250 llvm-svn: 285871
2016-11-03 02:07:16 +01:00
define void @f() {
entry:
call i32 (...) @g()
[ThinLTO] Enable importing of aliases as copy of aliasee Summary: This implements a missing feature to allow importing of aliases, which was previously disabled because alias cannot be available_externally. We instead import an alias as a copy of its aliasee. Some additional work was required in the IndexBitcodeWriter for the distributed build case, to ensure that the aliasee has a value id in the distributed index file (i.e. even when it is not being imported directly). This is a performance win in codes that have many aliases, e.g. C++ applications that have many constructor and destructor aliases. Reviewers: pcc Subscribers: mehdi_amini, inglorion, eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D40747 llvm-svn: 320895
2017-12-16 01:18:12 +01:00
call void (...) @analias()
[ThinLTO] Handle distributed backend case when doing renaming Summary: The recent change I made to consult the summary when deciding whether to rename (to handle inline asm) in r285513 broke the distributed build case. In a distributed backend we will only have a portion of the combined index, specifically for imported modules we only have the summaries for any imported definitions. When renaming on import we were asserting because no summary entry was found for a local reference being linked in (def wasn't imported). We only need to consult the summary for a renaming decision for the exporting module. For imports, we would have prevented importing any references to NoRename values already. Reviewers: mehdi_amini Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D26250 llvm-svn: 285871
2016-11-03 02:07:16 +01:00
ret void
}
[ThinLTO] Add support for emitting minimized bitcode for thin link Summary: The cumulative size of the bitcode files for a very large application can be huge, particularly with -g. In a distributed build environment, all of these files must be sent to the remote build node that performs the thin link step, and this can exceed size limits. The thin link actually only needs the summary along with a bitcode symbol table. Until we have a proper bitcode symbol table, simply stripping the debug metadata results in significant size reduction. Add support for an option to additionally emit minimized bitcode modules, just for use in the thin link step, which for now just strips all debug metadata. I plan to add a cc1 option so this can be invoked easily during the compile step. However, care must be taken to ensure that these minimized thin link bitcode files produce the same index as with the original bitcode files, as these original bitcode files will be used in the backends. Specifically: 1) The module hash used for caching is typically produced by hashing the written bitcode, and we want to include the hash that would correspond to the original bitcode file. This is because we want to ensure that changes in the stripped portions affect caching. Added plumbing to emit the same module hash in the minimized thin link bitcode file. 2) The module paths in the index are constructed from the module ID of each thin linked bitcode, and typically is automatically generated from the input file path. This is the path used for finding the modules to import from, and obviously we need this to point to the original bitcode files. Added gold-plugin support to take a suffix replacement during the thin link that is used to override the identifier on the MemoryBufferRef constructed from the loaded thin link bitcode file. The assumption is that the build system can specify that the minimized bitcode file has a name that is similar but uses a different suffix (e.g. out.thinlink.bc instead of out.o). Added various tests to ensure that we get identical index files out of the thin link step. Reviewers: mehdi_amini, pcc Subscribers: Prazek, llvm-commits Differential Revision: https://reviews.llvm.org/D31027 llvm-svn: 298638
2017-03-23 20:47:39 +01:00
ThinLTO Minimized Bitcode File Size Reduction Summary: Currently the ThinLTO minimized bitcode file only strip the debug info, but there is still a lot of information in the minimized bit code file that will be not used for thin linker. In this patch, most of the extra information is striped to reduce the minimized bitcode file. Now only ModuleVersion, ModuleInfo, ModuleGlobalValueSummary, ModuleHash, Symtab and Strtab are left. Now the minimized bitcode file size is reduced to 15%-30% of the debug info stripped bitcode file size. Reviewers: danielcdh, tejohnson, pcc Reviewed By: pcc Subscribers: mehdi_amini, aprantl, inglorion, eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D35334 llvm-svn: 308760
2017-07-21 19:25:20 +02:00
; THINLINKBITCODE-NOT: IDENTIFICATION_BLOCK_ID
; THINLINKBITCODE-NOT: BLOCKINFO_BLOCK
; THINLINKBITCODE-NOT: TYPE_BLOCK_ID
; THINLINKBITCODE-NOT: VSTOFFSET
; THINLINKBITCODE-NOT: CONSTANTS_BLOCK
; THINLINKBITCODE-NOT: METADATA_KIND_BLOCK
; THINLINKBITCODE-NOT: METADATA_BLOCK
; THINLINKBITCODE-NOT: OPERAND_BUNDLE_TAGS_BLOCK
; THINLINKBITCODE-NOT: UnknownBlock26
; THINLINKBITCODE-NOT: FUNCTION_BLOCK
; THINLINKBITCODE-NOT: VALUE_SYMTAB
; THINLINKBITCODE: MODULE_BLOCK
; THINLINKBITCODE: VERSION
; THINLINKBITCODE: SOURCE_FILENAME
; THINLINKBITCODE: GLOBALVAL_SUMMARY_BLOCK
; THINLINKBITCODE: HASH
; THINLINKBITCODE: SYMTAB_BLOCK
; THINLINKBITCODE: STRTAB_BLOCK
[ThinLTO] Add support for emitting minimized bitcode for thin link Summary: The cumulative size of the bitcode files for a very large application can be huge, particularly with -g. In a distributed build environment, all of these files must be sent to the remote build node that performs the thin link step, and this can exceed size limits. The thin link actually only needs the summary along with a bitcode symbol table. Until we have a proper bitcode symbol table, simply stripping the debug metadata results in significant size reduction. Add support for an option to additionally emit minimized bitcode modules, just for use in the thin link step, which for now just strips all debug metadata. I plan to add a cc1 option so this can be invoked easily during the compile step. However, care must be taken to ensure that these minimized thin link bitcode files produce the same index as with the original bitcode files, as these original bitcode files will be used in the backends. Specifically: 1) The module hash used for caching is typically produced by hashing the written bitcode, and we want to include the hash that would correspond to the original bitcode file. This is because we want to ensure that changes in the stripped portions affect caching. Added plumbing to emit the same module hash in the minimized thin link bitcode file. 2) The module paths in the index are constructed from the module ID of each thin linked bitcode, and typically is automatically generated from the input file path. This is the path used for finding the modules to import from, and obviously we need this to point to the original bitcode files. Added gold-plugin support to take a suffix replacement during the thin link that is used to override the identifier on the MemoryBufferRef constructed from the loaded thin link bitcode file. The assumption is that the build system can specify that the minimized bitcode file has a name that is similar but uses a different suffix (e.g. out.thinlink.bc instead of out.o). Added various tests to ensure that we get identical index files out of the thin link step. Reviewers: mehdi_amini, pcc Subscribers: Prazek, llvm-commits Differential Revision: https://reviews.llvm.org/D31027 llvm-svn: 298638
2017-03-23 20:47:39 +01:00
!llvm.dbg.cu = !{}
!1 = !{i32 2, !"Debug Info Version", i32 3}
!llvm.module.flags = !{!1}
Reference in New Issue Copy Permalink