2012-03-15 00:19:53 +01:00
|
|
|
; RUN: opt -inline < %s -S -o - -inline-threshold=10 | FileCheck %s
|
|
|
|
|
|
2013-08-29 00:41:57 +02:00
|
|
|
target datalayout = "p:32:32-p1:64:64-p2:16:16-n16:32:64"
|
Initial commit for the rewrite of the inline cost analysis to operate
on a per-callsite walk of the called function's instructions, in
breadth-first order over the potentially reachable set of basic blocks.
This is a major shift in how inline cost analysis works to improve the
accuracy and rationality of inlining decisions. A brief outline of the
algorithm this moves to:
- Build a simplification mapping based on the callsite arguments to the
function arguments.
- Push the entry block onto a worklist of potentially-live basic blocks.
- Pop the first block off of the *front* of the worklist (for
breadth-first ordering) and walk its instructions using a custom
InstVisitor.
- For each instruction's operands, re-map them based on the
simplification mappings available for the given callsite.
- Compute any simplification possible of the instruction after
re-mapping, and store that back int othe simplification mapping.
- Compute any bonuses, costs, or other impacts of the instruction on the
cost metric.
- When the terminator is reached, replace any conditional value in the
terminator with any simplifications from the mapping we have, and add
any successors which are not proven to be dead from these
simplifications to the worklist.
- Pop the next block off of the front of the worklist, and repeat.
- As soon as the cost of inlining exceeds the threshold for the
callsite, stop analyzing the function in order to bound cost.
The primary goal of this algorithm is to perfectly handle dead code
paths. We do not want any code in trivially dead code paths to impact
inlining decisions. The previous metric was *extremely* flawed here, and
would always subtract the average cost of two successors of
a conditional branch when it was proven to become an unconditional
branch at the callsite. There was no handling of wildly different costs
between the two successors, which would cause inlining when the path
actually taken was too large, and no inlining when the path actually
taken was trivially simple. There was also no handling of the code
*path*, only the immediate successors. These problems vanish completely
now. See the added regression tests for the shiny new features -- we
skip recursive function calls, SROA-killing instructions, and high cost
complex CFG structures when dead at the callsite being analyzed.
Switching to this algorithm required refactoring the inline cost
interface to accept the actual threshold rather than simply returning
a single cost. The resulting interface is pretty bad, and I'm planning
to do lots of interface cleanup after this patch.
Several other refactorings fell out of this, but I've tried to minimize
them for this patch. =/ There is still more cleanup that can be done
here. Please point out anything that you see in review.
I've worked really hard to try to mirror at least the spirit of all of
the previous heuristics in the new model. It's not clear that they are
all correct any more, but I wanted to minimize the change in this single
patch, it's already a bit ridiculous. One heuristic that is *not* yet
mirrored is to allow inlining of functions with a dynamic alloca *if*
the caller has a dynamic alloca. I will add this back, but I think the
most reasonable way requires changes to the inliner itself rather than
just the cost metric, and so I've deferred this for a subsequent patch.
The test case is XFAIL-ed until then.
As mentioned in the review mail, this seems to make Clang run about 1%
to 2% faster in -O0, but makes its binary size grow by just under 4%.
I've looked into the 4% growth, and it can be fixed, but requires
changes to other parts of the inliner.
llvm-svn: 153812
2012-03-31 14:42:41 +02:00
|
|
|
|
2012-03-15 00:19:53 +01:00
|
|
|
define i32 @outer1() {
|
2013-07-14 03:42:54 +02:00
|
|
|
; CHECK-LABEL: @outer1(
|
2012-03-15 00:19:53 +01:00
|
|
|
; CHECK-NOT: call
|
|
|
|
|
; CHECK: ret i32
|
|
|
|
|
|
|
|
|
|
%ptr = alloca i32
|
[opaque pointer type] Add textual IR support for explicit type parameter to getelementptr instruction
One of several parallel first steps to remove the target type of pointers,
replacing them with a single opaque pointer type.
This adds an explicit type parameter to the gep instruction so that when the
first parameter becomes an opaque pointer type, the type to gep through is
still available to the instructions.
* This doesn't modify gep operators, only instructions (operators will be
handled separately)
* Textual IR changes only. Bitcode (including upgrade) and changing the
in-memory representation will be in separate changes.
* geps of vectors are transformed as:
getelementptr <4 x float*> %x, ...
->getelementptr float, <4 x float*> %x, ...
Then, once the opaque pointer type is introduced, this will ultimately look
like:
getelementptr float, <4 x ptr> %x
with the unambiguous interpretation that it is a vector of pointers to float.
* address spaces remain on the pointer, not the type:
getelementptr float addrspace(1)* %x
->getelementptr float, float addrspace(1)* %x
Then, eventually:
getelementptr float, ptr addrspace(1) %x
Importantly, the massive amount of test case churn has been automated by
same crappy python code. I had to manually update a few test cases that
wouldn't fit the script's model (r228970,r229196,r229197,r229198). The
python script just massages stdin and writes the result to stdout, I
then wrapped that in a shell script to handle replacing files, then
using the usual find+xargs to migrate all the files.
update.py:
import fileinput
import sys
import re
ibrep = re.compile(r"(^.*?[^%\w]getelementptr inbounds )(((?:<\d* x )?)(.*?)(| addrspace\(\d\)) *\*(|>)(?:$| *(?:%|@|null|undef|blockaddress|getelementptr|addrspacecast|bitcast|inttoptr|\[\[[a-zA-Z]|\{\{).*$))")
normrep = re.compile( r"(^.*?[^%\w]getelementptr )(((?:<\d* x )?)(.*?)(| addrspace\(\d\)) *\*(|>)(?:$| *(?:%|@|null|undef|blockaddress|getelementptr|addrspacecast|bitcast|inttoptr|\[\[[a-zA-Z]|\{\{).*$))")
def conv(match, line):
if not match:
return line
line = match.groups()[0]
if len(match.groups()[5]) == 0:
line += match.groups()[2]
line += match.groups()[3]
line += ", "
line += match.groups()[1]
line += "\n"
return line
for line in sys.stdin:
if line.find("getelementptr ") == line.find("getelementptr inbounds"):
if line.find("getelementptr inbounds") != line.find("getelementptr inbounds ("):
line = conv(re.match(ibrep, line), line)
elif line.find("getelementptr ") != line.find("getelementptr ("):
line = conv(re.match(normrep, line), line)
sys.stdout.write(line)
apply.sh:
for name in "$@"
do
python3 `dirname "$0"`/update.py < "$name" > "$name.tmp" && mv "$name.tmp" "$name"
rm -f "$name.tmp"
done
The actual commands:
From llvm/src:
find test/ -name *.ll | xargs ./apply.sh
From llvm/src/tools/clang:
find test/ -name *.mm -o -name *.m -o -name *.cpp -o -name *.c | xargs -I '{}' ../../apply.sh "{}"
From llvm/src/tools/polly:
find test/ -name *.ll | xargs ./apply.sh
After that, check-all (with llvm, clang, clang-tools-extra, lld,
compiler-rt, and polly all checked out).
The extra 'rm' in the apply.sh script is due to a few files in clang's test
suite using interesting unicode stuff that my python script was throwing
exceptions on. None of those files needed to be migrated, so it seemed
sufficient to ignore those cases.
Reviewers: rafael, dexonsmith, grosser
Differential Revision: http://reviews.llvm.org/D7636
llvm-svn: 230786
2015-02-27 20:29:02 +01:00
|
|
|
%ptr1 = getelementptr inbounds i32, i32* %ptr, i32 0
|
|
|
|
|
%ptr2 = getelementptr inbounds i32, i32* %ptr, i32 42
|
2012-03-15 00:19:53 +01:00
|
|
|
%result = call i32 @inner1(i32* %ptr1, i32* %ptr2)
|
|
|
|
|
ret i32 %result
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
define i32 @inner1(i32* %begin, i32* %end) {
|
|
|
|
|
%begin.i = ptrtoint i32* %begin to i32
|
|
|
|
|
%end.i = ptrtoint i32* %end to i32
|
|
|
|
|
%distance = sub i32 %end.i, %begin.i
|
|
|
|
|
%icmp = icmp sle i32 %distance, 42
|
|
|
|
|
br i1 %icmp, label %then, label %else
|
|
|
|
|
|
|
|
|
|
then:
|
|
|
|
|
ret i32 3
|
|
|
|
|
|
|
|
|
|
else:
|
2015-02-27 22:17:42 +01:00
|
|
|
%t = load i32, i32* %begin
|
2012-03-15 00:19:53 +01:00
|
|
|
ret i32 %t
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
define i32 @outer2(i32* %ptr) {
|
|
|
|
|
; Test that an inbounds GEP disables this -- it isn't safe in general as
|
2014-01-24 18:20:08 +01:00
|
|
|
; wrapping changes the behavior of lessthan and greaterthan comparisons.
|
2013-07-14 03:42:54 +02:00
|
|
|
; CHECK-LABEL: @outer2(
|
2012-03-15 00:19:53 +01:00
|
|
|
; CHECK: call i32 @inner2
|
|
|
|
|
; CHECK: ret i32
|
|
|
|
|
|
[opaque pointer type] Add textual IR support for explicit type parameter to getelementptr instruction
One of several parallel first steps to remove the target type of pointers,
replacing them with a single opaque pointer type.
This adds an explicit type parameter to the gep instruction so that when the
first parameter becomes an opaque pointer type, the type to gep through is
still available to the instructions.
* This doesn't modify gep operators, only instructions (operators will be
handled separately)
* Textual IR changes only. Bitcode (including upgrade) and changing the
in-memory representation will be in separate changes.
* geps of vectors are transformed as:
getelementptr <4 x float*> %x, ...
->getelementptr float, <4 x float*> %x, ...
Then, once the opaque pointer type is introduced, this will ultimately look
like:
getelementptr float, <4 x ptr> %x
with the unambiguous interpretation that it is a vector of pointers to float.
* address spaces remain on the pointer, not the type:
getelementptr float addrspace(1)* %x
->getelementptr float, float addrspace(1)* %x
Then, eventually:
getelementptr float, ptr addrspace(1) %x
Importantly, the massive amount of test case churn has been automated by
same crappy python code. I had to manually update a few test cases that
wouldn't fit the script's model (r228970,r229196,r229197,r229198). The
python script just massages stdin and writes the result to stdout, I
then wrapped that in a shell script to handle replacing files, then
using the usual find+xargs to migrate all the files.
update.py:
import fileinput
import sys
import re
ibrep = re.compile(r"(^.*?[^%\w]getelementptr inbounds )(((?:<\d* x )?)(.*?)(| addrspace\(\d\)) *\*(|>)(?:$| *(?:%|@|null|undef|blockaddress|getelementptr|addrspacecast|bitcast|inttoptr|\[\[[a-zA-Z]|\{\{).*$))")
normrep = re.compile( r"(^.*?[^%\w]getelementptr )(((?:<\d* x )?)(.*?)(| addrspace\(\d\)) *\*(|>)(?:$| *(?:%|@|null|undef|blockaddress|getelementptr|addrspacecast|bitcast|inttoptr|\[\[[a-zA-Z]|\{\{).*$))")
def conv(match, line):
if not match:
return line
line = match.groups()[0]
if len(match.groups()[5]) == 0:
line += match.groups()[2]
line += match.groups()[3]
line += ", "
line += match.groups()[1]
line += "\n"
return line
for line in sys.stdin:
if line.find("getelementptr ") == line.find("getelementptr inbounds"):
if line.find("getelementptr inbounds") != line.find("getelementptr inbounds ("):
line = conv(re.match(ibrep, line), line)
elif line.find("getelementptr ") != line.find("getelementptr ("):
line = conv(re.match(normrep, line), line)
sys.stdout.write(line)
apply.sh:
for name in "$@"
do
python3 `dirname "$0"`/update.py < "$name" > "$name.tmp" && mv "$name.tmp" "$name"
rm -f "$name.tmp"
done
The actual commands:
From llvm/src:
find test/ -name *.ll | xargs ./apply.sh
From llvm/src/tools/clang:
find test/ -name *.mm -o -name *.m -o -name *.cpp -o -name *.c | xargs -I '{}' ../../apply.sh "{}"
From llvm/src/tools/polly:
find test/ -name *.ll | xargs ./apply.sh
After that, check-all (with llvm, clang, clang-tools-extra, lld,
compiler-rt, and polly all checked out).
The extra 'rm' in the apply.sh script is due to a few files in clang's test
suite using interesting unicode stuff that my python script was throwing
exceptions on. None of those files needed to be migrated, so it seemed
sufficient to ignore those cases.
Reviewers: rafael, dexonsmith, grosser
Differential Revision: http://reviews.llvm.org/D7636
llvm-svn: 230786
2015-02-27 20:29:02 +01:00
|
|
|
%ptr1 = getelementptr i32, i32* %ptr, i32 0
|
|
|
|
|
%ptr2 = getelementptr i32, i32* %ptr, i32 42
|
2012-03-15 00:19:53 +01:00
|
|
|
%result = call i32 @inner2(i32* %ptr1, i32* %ptr2)
|
|
|
|
|
ret i32 %result
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
define i32 @inner2(i32* %begin, i32* %end) {
|
|
|
|
|
%begin.i = ptrtoint i32* %begin to i32
|
|
|
|
|
%end.i = ptrtoint i32* %end to i32
|
|
|
|
|
%distance = sub i32 %end.i, %begin.i
|
|
|
|
|
%icmp = icmp sle i32 %distance, 42
|
|
|
|
|
br i1 %icmp, label %then, label %else
|
|
|
|
|
|
|
|
|
|
then:
|
|
|
|
|
ret i32 3
|
|
|
|
|
|
|
|
|
|
else:
|
2015-02-27 22:17:42 +01:00
|
|
|
%t = load i32, i32* %begin
|
2012-03-15 00:19:53 +01:00
|
|
|
ret i32 %t
|
|
|
|
|
}
|
2013-08-29 00:41:57 +02:00
|
|
|
|
|
|
|
|
; The inttoptrs are free since it is a smaller integer to a larger
|
|
|
|
|
; pointer size
|
|
|
|
|
define i32 @inttoptr_free_cost(i32 %a, i32 %b, i32 %c) {
|
|
|
|
|
%p1 = inttoptr i32 %a to i32 addrspace(1)*
|
|
|
|
|
%p2 = inttoptr i32 %b to i32 addrspace(1)*
|
|
|
|
|
%p3 = inttoptr i32 %c to i32 addrspace(1)*
|
2015-02-27 22:17:42 +01:00
|
|
|
%t1 = load i32, i32 addrspace(1)* %p1
|
|
|
|
|
%t2 = load i32, i32 addrspace(1)* %p2
|
|
|
|
|
%t3 = load i32, i32 addrspace(1)* %p3
|
2013-08-29 00:41:57 +02:00
|
|
|
%s = add i32 %t1, %t2
|
|
|
|
|
%s1 = add i32 %s, %t3
|
|
|
|
|
ret i32 %s1
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
define i32 @inttoptr_free_cost_user(i32 %begin, i32 %end) {
|
|
|
|
|
; CHECK-LABEL: @inttoptr_free_cost_user(
|
|
|
|
|
; CHECK-NOT: call
|
|
|
|
|
%x = call i32 @inttoptr_free_cost(i32 %begin, i32 %end, i32 9)
|
|
|
|
|
ret i32 %x
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
; The inttoptrs have a cost since it is a larger integer to a smaller
|
|
|
|
|
; pointer size
|
|
|
|
|
define i32 @inttoptr_cost_smaller_ptr(i32 %a, i32 %b, i32 %c) {
|
|
|
|
|
%p1 = inttoptr i32 %a to i32 addrspace(2)*
|
|
|
|
|
%p2 = inttoptr i32 %b to i32 addrspace(2)*
|
|
|
|
|
%p3 = inttoptr i32 %c to i32 addrspace(2)*
|
2015-02-27 22:17:42 +01:00
|
|
|
%t1 = load i32, i32 addrspace(2)* %p1
|
|
|
|
|
%t2 = load i32, i32 addrspace(2)* %p2
|
|
|
|
|
%t3 = load i32, i32 addrspace(2)* %p3
|
2013-08-29 00:41:57 +02:00
|
|
|
%s = add i32 %t1, %t2
|
|
|
|
|
%s1 = add i32 %s, %t3
|
|
|
|
|
ret i32 %s1
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
define i32 @inttoptr_cost_smaller_ptr_user(i32 %begin, i32 %end) {
|
|
|
|
|
; CHECK-LABEL: @inttoptr_cost_smaller_ptr_user(
|
|
|
|
|
; CHECK: call
|
|
|
|
|
%x = call i32 @inttoptr_cost_smaller_ptr(i32 %begin, i32 %end, i32 9)
|
|
|
|
|
ret i32 %x
|
|
|
|
|
}
|
|
|
|
|
|
