1
0
mirror of https://github.com/RPCS3/llvm-mirror.git synced 2024-10-21 03:53:04 +02:00
Commit Graph

61 Commits

Author SHA1 Message Date
Mehdi Amini
d3e6360b27 ThinLTO: Make aliases explicit in the summary
To be able to work accurately on the reference graph when taking
decision about internalizing, promoting, renaming, etc. We need
to have the alias information explicit.

Differential Revision: http://reviews.llvm.org/D18836

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266517
2016-04-16 06:56:44 +00:00
Mehdi Amini
6968b7bc43 Revert "Make aliases explicit in the summary"
Inadvertently commited...

This reverts commit e618ec93786d99df2ddf280ad2d5e02f5516cecf.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266215
2016-04-13 17:20:07 +00:00
Mehdi Amini
d96d756a2f Make aliases explicit in the summary
Summary:
To be able to work accurately on the reference graph when taking decision
about internalizing, promoting, renaming, etc. We need to have the alias
information explicit.

Reviewers: tejohnson

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D18836

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266214
2016-04-13 17:18:42 +00:00
Mehdi Amini
fcc69cbc7e Add a module Hash in the bitcode and the combined index, implementing a kind of "build-id"
This is intended to be used for ThinLTO incremental build.

Differential Revision: http://reviews.llvm.org/D18213

This is a recommit of r265095 after fixing the Windows issues.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265111
2016-04-01 05:33:11 +00:00
Mehdi Amini
b50dad82a8 Revert "Add support for computing SHA1 in LLVM"
This reverts commit r265096, r265095, and r265094.
Windows build is broken, and the validation does not pass.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265102
2016-04-01 03:03:21 +00:00
Mehdi Amini
e1ba9e8e87 Add missing test for the "Module hash in bitcode" added in r265095
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 265096
2016-04-01 01:37:52 +00:00
Duncan P. N. Exon Smith
a6ffc80357 Support: Implement StreamingMemoryObject::getPointer
The implementation is fairly obvious.  This is preparation for using
some blobs in bitcode.

For clarity (and perhaps future-proofing?), I moved the call to
JumpToBit in BitstreamCursor::readRecord ahead of calling
MemoryObject::getPointer, since JumpToBit can theoretically (a) read
bytes, which (b) invalidates the blob pointer.

This isn't strictly necessary the two memory objects we have:

  - The return of RawMemoryObject::getPointer is valid until the memory
    object is destroyed.

  - StreamingMemoryObject::getPointer is valid until the next chunk is
    read from the stream.  Since the JumpToBit call is only going ahead
    to a word boundary, we'll never load another chunk.

However, reordering makes it clear by inspection that the blob returned
by BitstreamCursor::readRecord will be valid.

I added some tests for StreamingMemoryObject::getPointer and
BitstreamCursor::readRecord.

llvm-svn: 264549
2016-03-27 23:00:59 +00:00
Duncan P. N. Exon Smith
5f89dfaeae Revert "Bitcode: Collect all MDString records into a single blob"
This reverts commit r264409 since it failed to bootstrap:
http://lab.llvm.org:8080/green/job/clang-stage2-configure-Rlto_build/8302/

llvm-svn: 264410
2016-03-25 15:22:27 +00:00
Duncan P. N. Exon Smith
7b1e6b8367 Bitcode: Collect all MDString records into a single blob
Optimize output of MDStrings in bitcode.  This emits them in big blocks
(currently 1024) in a pair of records:
  - BULK_STRING_SIZES: the sizes of the strings in the block, and
  - BULK_STRING_DATA: a single blob, which is the concatenation of all
    the strings.

Inspired by Mehdi's similar patch, http://reviews.llvm.org/D18342, this
should (a) slightly reduce bitcode size, since there is less record
overhead, and (b) greatly improve reading speed, since blobs are super
cheap to deserialize.

I needed to add support for blobs to streaming input to get the test
suite passing.
  - StreamingMemoryObject::getPointer reads ahead and returns the
    address of the blob.
  - To avoid a possible reallocation of StreamingMemoryObject::Bytes,
    BitstreamCursor::readRecord needs to move the call to JumpToEnd
    forward so that getPointer is the last bitstream operation.

llvm-svn: 264409
2016-03-25 14:40:18 +00:00
Teresa Johnson
e1affa2f8c [ThinLTO] Support for reference graph in per-module and combined summary.
Summary:
This patch adds support for including a full reference graph including
call graph edges and other GV references in the summary.

The reference graph edges can be used to make importing decisions
without materializing any source modules, can be used in the plugin
to make file staging decisions for distributed build systems, and is
expected to have other uses.

The call graph edges are recorded in each function summary in the
bitcode via a list of <CalleeValueIds, StaticCount> tuples when no PGO
data exists, or <CalleeValueId, StaticCount, ProfileCount> pairs when
there is PGO, where the ValueId can be mapped to the function GUID via
the ValueSymbolTable. In the function index in memory, the call graph
edges reference the target via the CalleeGUID instead of the
CalleeValueId.

The reference graph edges are recorded in each summary record with a
list of referenced value IDs, which can be mapped to value GUID via the
ValueSymbolTable.

Addtionally, a new summary record type is added to record references
from global variable initializers. A number of bitcode records and data
structures have been renamed to reflect the newly expanded scope of the
summary beyond functions. More cleanup will follow.

Reviewers: joker.eph, davidxl

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D17212

llvm-svn: 263275
2016-03-11 18:52:24 +00:00
Teresa Johnson
c47e95e1b1 Restore "[ThinLTO] Use MD5 hash in function index." with fix
This restores commit r260408, along with a fix for a bot failure.

The bot failure was caused by dereferencing a unique_ptr in the same
call instruction parameter list where it was passed via std::move.
Apparently due to luck this was not exposed when I built the compiler
with clang, only with gcc.

llvm-svn: 260442
2016-02-10 21:55:02 +00:00
Teresa Johnson
02e161e44f Revert "[ThinLTO] Use MD5 hash in function index." due to bot failure
This reverts commit r260408. Bot failure that I need to investigate.

llvm-svn: 260412
2016-02-10 19:11:15 +00:00
Teresa Johnson
b1e839beb3 [ThinLTO] Use MD5 hash in function index.
Summary:
This patch uses the lower 64-bits of the MD5 hash of a function name as
a GUID in the function index, instead of storing function names. Any
local functions are first given a global name by prepending the original
source file name. This is the same naming scheme and GUID used by PGO in
the indexed profile format.

This change has a couple of benefits. The primary benefit is size
reduction in the combined index file, for example 483.xalancbmk's
combined index file was reduced by around 70%. It should also result in
memory savings for the index file in memory, as the in-memory map is
also indexed by the hash instead of the string.

Second, this enables integration with indirect call promotion, since the
indirect call profile targets are recorded using the same global naming
convention and hash. This will enable the function importer to easily
locate function summaries for indirect call profile targets to enable
their import and subsequent promotion.

The original source file name is recorded in the bitcode in a new
module-level record for use in the ThinLTO backend pipeline.

Reviewers: davidxl, joker.eph

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D17028

llvm-svn: 260408
2016-02-10 18:57:54 +00:00
Filipe Cabecinhas
e215a5f192 Error out when faced with value names containing '\0'
Bug found with afl-fuzz.

llvm-svn: 252048
2015-11-04 14:53:36 +00:00
Filipe Cabecinhas
e813424af3 Don't assert if materializing before seeing any function bodies
This assert was reachable from user input. A minimized test case (no
FUNCTION_BLOCK_ID record) is attached.

Bug found with afl-fuzz

llvm-svn: 251910
2015-11-03 13:48:26 +00:00
Filipe Cabecinhas
13277d521a Revert "Don't assert if materializing before seeing any function bodies"
This reverts r251667 since it broke the bots.

llvm-svn: 251671
2015-10-30 00:00:58 +00:00
Filipe Cabecinhas
2ea04f8f0f Don't assert if materializing before seeing any function bodies
This assert was reachable from user input. A minimized test case (no
FUNCTION_BLOCK_ID record) is attached.

Bug found with afl-fuzz

llvm-svn: 251667
2015-10-29 23:37:28 +00:00
Filipe Cabecinhas
58b77fabdc Make sure the CastInst is valid before trying to create it
Bug found with afl-fuzz.

llvm-svn: 249396
2015-10-06 12:37:54 +00:00
Derek Schuff
6203009638 Fix testing for end of stream in bitstream reader.
This fixes a bug found while working on the bitcode reader. In
particular, the method BitstreamReader::AtEndOfStream doesn't always
behave correctly when processing a data streamer. The method
fillCurWord doesn't properly set CurWord/BitsInCurWord if the data
streamer was already at eof, but GetBytes had not yet set the
ObjectSize field of the streaming memory object.

This patch fixes this problem, and provides a test to show that
this problem has been fixed.

Patch by Karl Schimpf.

Differential Revision: http://reviews.llvm.org/D11391

llvm-svn: 243890
2015-08-03 18:01:50 +00:00
Rafael Espindola
43efaa4095 Fix for PR23310: llvm-dis crashes when trying to upgrade an intrinsic.
When trying to upgrade @llvm.x86.sse2.psrl.dq while parsing a module,
BitcodeReader adds the function to its worklist twice, resulting in a
crash when accessing it the second time.

This patch replaces the worklist vector by a map.

Patch by Philip Pfaffe.

llvm-svn: 241281
2015-07-02 16:22:40 +00:00
Rafael Espindola
0eb007e8d1 Improve handling of end of file in the bitcode reader.
Before this patch the bitcode reader would read a module from a file
that contained in order:

* Any number of non MODULE_BLOCK sub blocks.
* One MODULE_BLOCK
* Any number of non MODULE_BLOCK sub blocks.
* 4 '\n' characters to handle OS X's ranlib.

Since we support lazy reading of modules, any information that is relevant
for the module has to be in the MODULE_BLOCK or before it. We don't gain
anything from checking what is after.

This patch then changes the reader to stop once the MODULE_BLOCK has been
successfully parsed.

This avoids the ugly special case for .bc files in an archive and makes it
easier to embed bitcode files.

llvm-svn: 239845
2015-06-16 20:03:39 +00:00
Rafael Espindola
35b7ee8e26 Add a test for padded bitcode files.
llvm-svn: 239829
2015-06-16 16:36:15 +00:00
Filipe Cabecinhas
337bf6fd2c [BitcodeReader] Diagnose type mismatches with aliases
Bug found with AFL fuzz.

llvm-svn: 238895
2015-06-03 01:30:13 +00:00
Filipe Cabecinhas
cc520bbd9a [Bitcode] Minimize the test to not conflict with others
Source for the test:
@bloom = global <3 x i32> <i32 0, i32 1, i32 42>

Plus bit twiddling to set the vector numelts to 0 (in the bc file).

llvm-svn: 238894
2015-06-03 01:30:08 +00:00
Filipe Cabecinhas
0f2c25e1b3 [BitcodeReader] Check vector size before trying to create a VectorType
Bug found with AFL fuzz

llvm-svn: 238891
2015-06-03 00:05:30 +00:00
Filipe Cabecinhas
37a6f20080 [BitcodeReader] Change an assert to a call to a call to Error()
It's reachable from user input.

Bug found with AFL fuzz.

llvm-svn: 238633
2015-05-30 00:17:20 +00:00
Filipe Cabecinhas
829cff9ed4 [BitcodeReader] Change assert to report_fatal_error
It can be triggered by user input.

Bug found with AFL fuzz.

llvm-svn: 238272
2015-05-27 01:05:40 +00:00
Filipe Cabecinhas
41dd91ebe0 [BitstreamReader] Make sure the Array operand type is an encoding
Bug found with AFL fuzz.

llvm-svn: 238269
2015-05-27 00:48:43 +00:00
Filipe Cabecinhas
f74b0c7c40 [BitcodeReader] Make sure abbrev records have at least one operand (record code)
Bug found with AFL fuzz.

llvm-svn: 238265
2015-05-26 23:52:21 +00:00
Filipe Cabecinhas
2da00c394b [BitcodeReader] Sanity check on Comdat ID
Shouldn't be an assert, since user input can trigger it.

Bug found with AFL fuzz.

llvm-svn: 238261
2015-05-26 23:00:56 +00:00
Filipe Cabecinhas
9fd1becc70 Change a reachable unreachable to a fatal error.
Summary:
Also tagged a FIXME comment, and added information about why it breaks.

Bug found using AFL fuzz.

Reviewers: rafael, craig.topper

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D9729

llvm-svn: 237709
2015-05-19 18:18:10 +00:00
Filipe Cabecinhas
0cf74b4130 [BitcodeReader] Error out if we read an invalid function argument type
Bug found with AFL fuzz.

llvm-svn: 237650
2015-05-19 01:21:06 +00:00
Filipe Cabecinhas
ba3c9b8d80 [BitcodeReader] It's a malformed block if CodeLenWidth is too big
Bug found with AFL fuzz.

llvm-svn: 237646
2015-05-19 00:34:17 +00:00
Filipe Cabecinhas
5de3dd3448 [BitcodeReader] Make sure the type of the inserted value matches the type of the aggregate at those indices
Bug found with AFL-fuzz.

llvm-svn: 237628
2015-05-18 22:27:11 +00:00
Filipe Cabecinhas
9afdad64ab Extract the load/store type verification to a separate function.
Summary:
Added isLoadableOrStorableType to PointerType.

We were doing some checks in some places, occasionally assert()ing instead
of telling the caller. With this patch, I'm putting all type checking in
the same place for load/store type instructions, and verifying the same
thing every time.

I also added a check for load/store of a function type.

Applied extracted check to Load, Store, and Cmpxcg.

I don't have exhaustive tests for all of these, but all Error() calls in
TypeCheckLoadStoreInst are being tested (in invalid.test).

Reviewers: dblaikie, rafael

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D9785

llvm-svn: 237619
2015-05-18 21:48:55 +00:00
Filipe Cabecinhas
e8eb389930 [BitcodeReader] Don't allow INSERTVAL/EXTRACTVAL with 0 indices
This would trigger an assertion later.

Bug found with AFL fuzz.

llvm-svn: 237494
2015-05-16 00:33:12 +00:00
Filipe Cabecinhas
841f137646 Don't overflow GCTable
Summary: Bug found with AFL fuzz.

Reviewers: rafael, dexonsmith

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D9361

llvm-svn: 236200
2015-04-30 04:09:41 +00:00
Filipe Cabecinhas
b61d720670 Make sure Op->getType() is a PointerType before we cast<> it.
Bug found with AFL fuzz.

llvm-svn: 236193
2015-04-30 01:13:31 +00:00
Filipe Cabecinhas
c8f68a05d1 Make sure we don't resize(0) when we get a fwdref with Idx == UINT_MAX
Make it an error instead.

Bug found with AFL fuzz.

llvm-svn: 236190
2015-04-30 00:52:42 +00:00
Filipe Cabecinhas
1e65e2471f Check that we have a valid PointerType element type before calling get()
Same as r236073 but for PointerType.

Bug found with AFL fuzz.

llvm-svn: 236079
2015-04-29 02:27:28 +00:00
Filipe Cabecinhas
10fd7e5c6a Turn an assert into report_fatal_error since it's reachable based on user input
Bug found with AFL fuzz.

llvm-svn: 236076
2015-04-29 01:58:31 +00:00
Filipe Cabecinhas
535f776116 Make sure that isValidElementType(Type) before calling {Array,Struct}Type::get(Type)
Bug found with AFL fuzz.

llvm-svn: 236073
2015-04-29 01:27:01 +00:00
Filipe Cabecinhas
febbecdc50 Relax an assert when there's a type mismatch in forward references
Summary:
We don't seem to need to assert here, since this function's callers expect
to get a nullptr on error. This way we don't assert on user input.

Bug found with AFL fuzz.

Reviewers: rafael

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D9308

llvm-svn: 236027
2015-04-28 20:18:47 +00:00
David Blaikie
196bfb60ad [opaque pointer type] bitcode: add explicit callee type to invoke instructions
llvm-svn: 235735
2015-04-24 18:06:06 +00:00
Filipe Cabecinhas
6a560937ff [BitcodeReader] Fix asserts when we read a non-vector type for insert/extract/shuffle
Added some additional checking for vector types + tests.

Bug found with AFL fuzz.

llvm-svn: 235710
2015-04-24 11:30:15 +00:00
Filipe Cabecinhas
53125de6c7 Be more strict about the operand for the array type in BitcodeReader
Summary: Bug found with AFL fuzz.

Reviewers: rafael

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D9016

llvm-svn: 235596
2015-04-23 13:38:21 +00:00
Filipe Cabecinhas
4a7d22d5ac Verify sizes when trying to read a BitcodeAbbrevOp
Summary:
Make sure the abbrev operands are valid and that we can read/skip them
afterwards.

Bug found with AFL fuzz.

Reviewers: rafael

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D9030

llvm-svn: 235595
2015-04-23 13:25:35 +00:00
Filipe Cabecinhas
ae34e5b3f0 Have more strict type checks when creating BinOp nodes in BitcodeReader
Summary: Bug found with AFL.

Reviewers: rafael, bkramer

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D9015

llvm-svn: 235489
2015-04-22 09:06:21 +00:00
David Blaikie
4fc2911327 [opaque pointer type] Explicit pointee type for call instruction
Use an extra bit in the CCInfo to flag the newer version of the
instructiont hat includes the type explicitly.

Tested the newer error cases I added, but didn't add tests for the finer
granularity improvements to existing error paths.

llvm-svn: 235160
2015-04-17 06:40:14 +00:00
Filipe Cabecinhas
2b78cf49b7 Revert "Verify sizes when trying to read a VBR"
This reverts r234984 since it seems to break some bots (most of them
seemed arm*-selfhost).

llvm-svn: 234998
2015-04-15 11:10:17 +00:00