RPCS3/llvm-mirror
1
0
Fork 0
mirror of https://github.com/RPCS3/llvm-mirror.git synced 2024-10-31 07:52:55 +01:00
Code Issues Projects Releases Wiki Activity
67f2cc8ebf
llvm-mirror/test/CodeGen/X86/block-placement.ll

1124 lines
30 KiB
LLVM
Raw Normal View History

Fix tail merging to assign the (more) correct BasicBlock when splitting. This makes it possible to write unit tests that are less susceptible to minor code motion, particularly copy placement. block-placement.ll covers this case with -pre-RA-sched=source which will soon be default. One incorrectly named block is already fixed, but without this fix, enabling new coalescing and scheduling would cause more failures. llvm-svn: 184680
2013-06-24 03:55:01 +02:00
; RUN: llc -mtriple=i686-linux -pre-RA-sched=source < %s | FileCheck %s
Add a very basic test for MachineBlockPlacement. This is essentially the canonical example I used when developing it, and is one of the primary motivating real-world use cases for __builtin_expect (when burried under a macro). I'm working on more test cases here, but I'm trying to make sure both that the pass is doing the right thing with the test cases and that they aren't too brittle to changes elsewhere in the code generation pipeline. Feedback and/or suggestions on how to test this are very welcome. Especially feedback on whether testing the block comments is a good strategy; I couldn't find any good examples to steal from but all the other ideas I had were a lot uglier or more fragile. llvm-svn: 142644
2011-10-21 10:01:56 +02:00
declare void @error(i32 %i, i32 %a, i32 %b)
Add loop aligning to MachineBlockPlacement based on review discussion so it's a bit more plausible to use this instead of CodePlacementOpt. The code for this was shamelessly stolen from CodePlacementOpt, and then trimmed down a bit. There doesn't seem to be much utility in returning true/false from this pass as we may or may not have rewritten all of the blocks. Also, the statistic of counting how many loops were aligned doesn't seem terribly important so I removed it. If folks would like it to be included, I'm happy to add it back. This was probably the most egregious of the missing features, and now I'm going to start gathering some performance numbers and looking at specific loop structures that have different layout between the two. Test is updated to include both basic loop alignment and nested loop alignment. llvm-svn: 142645
2011-10-21 10:57:37 +02:00
define i32 @test_ifchains(i32 %i, i32* %a, i32 %b) {
Add a very basic test for MachineBlockPlacement. This is essentially the canonical example I used when developing it, and is one of the primary motivating real-world use cases for __builtin_expect (when burried under a macro). I'm working on more test cases here, but I'm trying to make sure both that the pass is doing the right thing with the test cases and that they aren't too brittle to changes elsewhere in the code generation pipeline. Feedback and/or suggestions on how to test this are very welcome. Especially feedback on whether testing the block comments is a good strategy; I couldn't find any good examples to steal from but all the other ideas I had were a lot uglier or more fragile. llvm-svn: 142644
2011-10-21 10:01:56 +02:00
; Test a chain of ifs, where the block guarded by the if is error handling code
; that is not expected to run.
Convert CodeGen/*/*.ll tests to use the new CHECK-LABEL for easier debugging. No functionality change and all tests pass after conversion. This was done with the following sed invocation to catch label lines demarking function boundaries: sed -i '' "s/^;\( *\)\([A-Z0-9_]*\):\( *\)test\([A-Za-z0-9_-]*\):\( *\)$/;\1\2-LABEL:\3test\4:\5/g" test/CodeGen/*/*.ll which was written conservatively to avoid false positives rather than false negatives. I scanned through all the changes and everything looks correct. llvm-svn: 186258
2013-07-13 22:38:47 +02:00
; CHECK-LABEL: test_ifchains:
Add a very basic test for MachineBlockPlacement. This is essentially the canonical example I used when developing it, and is one of the primary motivating real-world use cases for __builtin_expect (when burried under a macro). I'm working on more test cases here, but I'm trying to make sure both that the pass is doing the right thing with the test cases and that they aren't too brittle to changes elsewhere in the code generation pipeline. Feedback and/or suggestions on how to test this are very welcome. Especially feedback on whether testing the block comments is a good strategy; I couldn't find any good examples to steal from but all the other ideas I had were a lot uglier or more fragile. llvm-svn: 142644
2011-10-21 10:01:56 +02:00
; CHECK: %entry
Add a much more conservative strategy for aligning branch targets. Previously, MBP essentially aligned every branch target it could. This bloats code quite a bit, especially non-looping code which has no real reason to prefer aligned branch targets so heavily. As Andy said in review, it's still a bit odd to do this without a real cost model, but this at least has much more plausible heuristics. Fixes PR13265. llvm-svn: 161409
2012-08-07 11:45:24 +02:00
; CHECK-NOT: .align
Add a very basic test for MachineBlockPlacement. This is essentially the canonical example I used when developing it, and is one of the primary motivating real-world use cases for __builtin_expect (when burried under a macro). I'm working on more test cases here, but I'm trying to make sure both that the pass is doing the right thing with the test cases and that they aren't too brittle to changes elsewhere in the code generation pipeline. Feedback and/or suggestions on how to test this are very welcome. Especially feedback on whether testing the block comments is a good strategy; I couldn't find any good examples to steal from but all the other ideas I had were a lot uglier or more fragile. llvm-svn: 142644
2011-10-21 10:01:56 +02:00
; CHECK: %else1
Add a much more conservative strategy for aligning branch targets. Previously, MBP essentially aligned every branch target it could. This bloats code quite a bit, especially non-looping code which has no real reason to prefer aligned branch targets so heavily. As Andy said in review, it's still a bit odd to do this without a real cost model, but this at least has much more plausible heuristics. Fixes PR13265. llvm-svn: 161409
2012-08-07 11:45:24 +02:00
; CHECK-NOT: .align
Add a very basic test for MachineBlockPlacement. This is essentially the canonical example I used when developing it, and is one of the primary motivating real-world use cases for __builtin_expect (when burried under a macro). I'm working on more test cases here, but I'm trying to make sure both that the pass is doing the right thing with the test cases and that they aren't too brittle to changes elsewhere in the code generation pipeline. Feedback and/or suggestions on how to test this are very welcome. Especially feedback on whether testing the block comments is a good strategy; I couldn't find any good examples to steal from but all the other ideas I had were a lot uglier or more fragile. llvm-svn: 142644
2011-10-21 10:01:56 +02:00
; CHECK: %else2
Add a much more conservative strategy for aligning branch targets. Previously, MBP essentially aligned every branch target it could. This bloats code quite a bit, especially non-looping code which has no real reason to prefer aligned branch targets so heavily. As Andy said in review, it's still a bit odd to do this without a real cost model, but this at least has much more plausible heuristics. Fixes PR13265. llvm-svn: 161409
2012-08-07 11:45:24 +02:00
; CHECK-NOT: .align
Add a very basic test for MachineBlockPlacement. This is essentially the canonical example I used when developing it, and is one of the primary motivating real-world use cases for __builtin_expect (when burried under a macro). I'm working on more test cases here, but I'm trying to make sure both that the pass is doing the right thing with the test cases and that they aren't too brittle to changes elsewhere in the code generation pipeline. Feedback and/or suggestions on how to test this are very welcome. Especially feedback on whether testing the block comments is a good strategy; I couldn't find any good examples to steal from but all the other ideas I had were a lot uglier or more fragile. llvm-svn: 142644
2011-10-21 10:01:56 +02:00
; CHECK: %else3
Add a much more conservative strategy for aligning branch targets. Previously, MBP essentially aligned every branch target it could. This bloats code quite a bit, especially non-looping code which has no real reason to prefer aligned branch targets so heavily. As Andy said in review, it's still a bit odd to do this without a real cost model, but this at least has much more plausible heuristics. Fixes PR13265. llvm-svn: 161409
2012-08-07 11:45:24 +02:00
; CHECK-NOT: .align
Add a very basic test for MachineBlockPlacement. This is essentially the canonical example I used when developing it, and is one of the primary motivating real-world use cases for __builtin_expect (when burried under a macro). I'm working on more test cases here, but I'm trying to make sure both that the pass is doing the right thing with the test cases and that they aren't too brittle to changes elsewhere in the code generation pipeline. Feedback and/or suggestions on how to test this are very welcome. Especially feedback on whether testing the block comments is a good strategy; I couldn't find any good examples to steal from but all the other ideas I had were a lot uglier or more fragile. llvm-svn: 142644
2011-10-21 10:01:56 +02:00
; CHECK: %else4
Add a much more conservative strategy for aligning branch targets. Previously, MBP essentially aligned every branch target it could. This bloats code quite a bit, especially non-looping code which has no real reason to prefer aligned branch targets so heavily. As Andy said in review, it's still a bit odd to do this without a real cost model, but this at least has much more plausible heuristics. Fixes PR13265. llvm-svn: 161409
2012-08-07 11:45:24 +02:00
; CHECK-NOT: .align
Add a very basic test for MachineBlockPlacement. This is essentially the canonical example I used when developing it, and is one of the primary motivating real-world use cases for __builtin_expect (when burried under a macro). I'm working on more test cases here, but I'm trying to make sure both that the pass is doing the right thing with the test cases and that they aren't too brittle to changes elsewhere in the code generation pipeline. Feedback and/or suggestions on how to test this are very welcome. Especially feedback on whether testing the block comments is a good strategy; I couldn't find any good examples to steal from but all the other ideas I had were a lot uglier or more fragile. llvm-svn: 142644
2011-10-21 10:01:56 +02:00
; CHECK: %exit
; CHECK: %then1
; CHECK: %then2
; CHECK: %then3
; CHECK: %then4
; CHECK: %then5
entry:
%gep1 = getelementptr i32* %a, i32 1
%val1 = load i32* %gep1
%cond1 = icmp ugt i32 %val1, 1
br i1 %cond1, label %then1, label %else1, !prof !0
then1:
call void @error(i32 %i, i32 1, i32 %b)
br label %else1
else1:
%gep2 = getelementptr i32* %a, i32 2
%val2 = load i32* %gep2
%cond2 = icmp ugt i32 %val2, 2
br i1 %cond2, label %then2, label %else2, !prof !0
then2:
call void @error(i32 %i, i32 1, i32 %b)
br label %else2
else2:
%gep3 = getelementptr i32* %a, i32 3
%val3 = load i32* %gep3
%cond3 = icmp ugt i32 %val3, 3
br i1 %cond3, label %then3, label %else3, !prof !0
then3:
call void @error(i32 %i, i32 1, i32 %b)
br label %else3
else3:
%gep4 = getelementptr i32* %a, i32 4
%val4 = load i32* %gep4
%cond4 = icmp ugt i32 %val4, 4
br i1 %cond4, label %then4, label %else4, !prof !0
then4:
call void @error(i32 %i, i32 1, i32 %b)
br label %else4
else4:
%gep5 = getelementptr i32* %a, i32 3
%val5 = load i32* %gep5
%cond5 = icmp ugt i32 %val5, 3
br i1 %cond5, label %then5, label %exit, !prof !0
then5:
call void @error(i32 %i, i32 1, i32 %b)
br label %exit
exit:
ret i32 %b
}
Rewrite #3 of machine block placement. This is based somewhat on the second algorithm, but only loosely. It is more heavily based on the last discussion I had with Andy. It continues to walk from the inner-most loop outward, but there is a key difference. With this algorithm we ensure that as we visit each loop, the entire loop is merged into a single chain. At the end, the entire function is treated as a "loop", and merged into a single chain. This chain forms the desired sequence of blocks within the function. Switching to a single algorithm removes my biggest problem with the previous approaches -- they had different behavior depending on which system triggered the layout. Now there is exactly one algorithm and one basis for the decision making. The other key difference is how the chain is formed. This is based heavily on the idea Andy mentioned of keeping a worklist of blocks that are viable layout successors based on the CFG. Having this set allows us to consistently select the best layout successor for each block. It is expensive though. The code here remains very rough. There is a lot that needs to be done to clean up the code, and to make the runtime cost of this pass much lower. Very much WIP, but this was a giant chunk of code and I'd rather folks see it sooner than later. Everything remains behind a flag of course. I've added a couple of tests to exercise the issues that this iteration was motivated by: loop structure preservation. I've also fixed one test that was exhibiting the broken behavior of the previous version. llvm-svn: 144495
2011-11-13 12:20:44 +01:00
define i32 @test_loop_cold_blocks(i32 %i, i32* %a) {
; Check that we sink cold loop blocks after the hot loop body.
Convert CodeGen/*/*.ll tests to use the new CHECK-LABEL for easier debugging. No functionality change and all tests pass after conversion. This was done with the following sed invocation to catch label lines demarking function boundaries: sed -i '' "s/^;\( *\)\([A-Z0-9_]*\):\( *\)test\([A-Za-z0-9_-]*\):\( *\)$/;\1\2-LABEL:\3test\4:\5/g" test/CodeGen/*/*.ll which was written conservatively to avoid false positives rather than false negatives. I scanned through all the changes and everything looks correct. llvm-svn: 186258
2013-07-13 22:38:47 +02:00
; CHECK-LABEL: test_loop_cold_blocks:
Rewrite #3 of machine block placement. This is based somewhat on the second algorithm, but only loosely. It is more heavily based on the last discussion I had with Andy. It continues to walk from the inner-most loop outward, but there is a key difference. With this algorithm we ensure that as we visit each loop, the entire loop is merged into a single chain. At the end, the entire function is treated as a "loop", and merged into a single chain. This chain forms the desired sequence of blocks within the function. Switching to a single algorithm removes my biggest problem with the previous approaches -- they had different behavior depending on which system triggered the layout. Now there is exactly one algorithm and one basis for the decision making. The other key difference is how the chain is formed. This is based heavily on the idea Andy mentioned of keeping a worklist of blocks that are viable layout successors based on the CFG. Having this set allows us to consistently select the best layout successor for each block. It is expensive though. The code here remains very rough. There is a lot that needs to be done to clean up the code, and to make the runtime cost of this pass much lower. Very much WIP, but this was a giant chunk of code and I'd rather folks see it sooner than later. Everything remains behind a flag of course. I've added a couple of tests to exercise the issues that this iteration was motivated by: loop structure preservation. I've also fixed one test that was exhibiting the broken behavior of the previous version. llvm-svn: 144495
2011-11-13 12:20:44 +01:00
; CHECK: %entry
Add a much more conservative strategy for aligning branch targets. Previously, MBP essentially aligned every branch target it could. This bloats code quite a bit, especially non-looping code which has no real reason to prefer aligned branch targets so heavily. As Andy said in review, it's still a bit odd to do this without a real cost model, but this at least has much more plausible heuristics. Fixes PR13265. llvm-svn: 161409
2012-08-07 11:45:24 +02:00
; CHECK-NOT: .align
Rewrite how machine block placement handles loop rotation. This is a complex change that resulted from a great deal of experimentation with several different benchmarks. The one which proved the most useful is included as a test case, but I don't know that it captures all of the relevant changes, as I didn't have specific regression tests for each, they were more the result of reasoning about what the old algorithm would possibly do wrong. I'm also failing at the moment to craft more targeted regression tests for these changes, if anyone has ideas, it would be welcome. The first big thing broken with the old algorithm is the idea that we can take a basic block which has a loop-exiting successor and a looping successor and use the looping successor as the layout top in order to get that particular block to be the bottom of the loop after layout. This happens to work in many cases, but not in all. The second big thing broken was that we didn't try to select the exit which fell into the nearest enclosing loop (to which we exit at all). As a consequence, even if the rotation worked perfectly, it would result in one of two bad layouts. Either the bottom of the loop would get fallthrough, skipping across a nearer enclosing loop and thereby making it discontiguous, or it would be forced to take an explicit jump over the nearest enclosing loop to earch its successor. The point of the rotation is to get fallthrough, so we need it to fallthrough to the nearest loop it can. The fix to the first issue is to actually layout the loop from the loop header, and then rotate the loop such that the correct exiting edge can be a fallthrough edge. This is actually much easier than I anticipated because we can handle all the hard parts of finding a viable rotation before we do the layout. We just store that, and then rotate after layout is finished. No inner loops get split across the post-rotation backedge because we check for them when selecting the rotation. That fix exposed a latent problem with our exitting block selection -- we should allow the backedge to point into the middle of some inner-loop chain as there is no real penalty to it, the whole point is that it *won't* be a fallthrough edge. This may have blocked the rotation at all in some cases, I have no idea and no test case as I've never seen it in practice, it was just noticed by inspection. Finally, all of these fixes, and studying the loops they produce, highlighted another problem: in rotating loops like this, we sometimes fail to align the destination of these backwards jumping edges. Fix this by actually walking the backwards edges rather than relying on loopinfo. This fixes regressions on heapsort if block placement is enabled as well as lots of other cases where the previous logic would introduce an abundance of unnecessary branches into the execution. llvm-svn: 154783
2012-04-16 03:12:56 +02:00
; CHECK: %unlikely1
Add a much more conservative strategy for aligning branch targets. Previously, MBP essentially aligned every branch target it could. This bloats code quite a bit, especially non-looping code which has no real reason to prefer aligned branch targets so heavily. As Andy said in review, it's still a bit odd to do this without a real cost model, but this at least has much more plausible heuristics. Fixes PR13265. llvm-svn: 161409
2012-08-07 11:45:24 +02:00
; CHECK-NOT: .align
Rewrite how machine block placement handles loop rotation. This is a complex change that resulted from a great deal of experimentation with several different benchmarks. The one which proved the most useful is included as a test case, but I don't know that it captures all of the relevant changes, as I didn't have specific regression tests for each, they were more the result of reasoning about what the old algorithm would possibly do wrong. I'm also failing at the moment to craft more targeted regression tests for these changes, if anyone has ideas, it would be welcome. The first big thing broken with the old algorithm is the idea that we can take a basic block which has a loop-exiting successor and a looping successor and use the looping successor as the layout top in order to get that particular block to be the bottom of the loop after layout. This happens to work in many cases, but not in all. The second big thing broken was that we didn't try to select the exit which fell into the nearest enclosing loop (to which we exit at all). As a consequence, even if the rotation worked perfectly, it would result in one of two bad layouts. Either the bottom of the loop would get fallthrough, skipping across a nearer enclosing loop and thereby making it discontiguous, or it would be forced to take an explicit jump over the nearest enclosing loop to earch its successor. The point of the rotation is to get fallthrough, so we need it to fallthrough to the nearest loop it can. The fix to the first issue is to actually layout the loop from the loop header, and then rotate the loop such that the correct exiting edge can be a fallthrough edge. This is actually much easier than I anticipated because we can handle all the hard parts of finding a viable rotation before we do the layout. We just store that, and then rotate after layout is finished. No inner loops get split across the post-rotation backedge because we check for them when selecting the rotation. That fix exposed a latent problem with our exitting block selection -- we should allow the backedge to point into the middle of some inner-loop chain as there is no real penalty to it, the whole point is that it *won't* be a fallthrough edge. This may have blocked the rotation at all in some cases, I have no idea and no test case as I've never seen it in practice, it was just noticed by inspection. Finally, all of these fixes, and studying the loops they produce, highlighted another problem: in rotating loops like this, we sometimes fail to align the destination of these backwards jumping edges. Fix this by actually walking the backwards edges rather than relying on loopinfo. This fixes regressions on heapsort if block placement is enabled as well as lots of other cases where the previous logic would introduce an abundance of unnecessary branches into the execution. llvm-svn: 154783
2012-04-16 03:12:56 +02:00
; CHECK: %unlikely2
Add a much more conservative strategy for aligning branch targets. Previously, MBP essentially aligned every branch target it could. This bloats code quite a bit, especially non-looping code which has no real reason to prefer aligned branch targets so heavily. As Andy said in review, it's still a bit odd to do this without a real cost model, but this at least has much more plausible heuristics. Fixes PR13265. llvm-svn: 161409
2012-08-07 11:45:24 +02:00
; CHECK: .align
Rewrite #3 of machine block placement. This is based somewhat on the second algorithm, but only loosely. It is more heavily based on the last discussion I had with Andy. It continues to walk from the inner-most loop outward, but there is a key difference. With this algorithm we ensure that as we visit each loop, the entire loop is merged into a single chain. At the end, the entire function is treated as a "loop", and merged into a single chain. This chain forms the desired sequence of blocks within the function. Switching to a single algorithm removes my biggest problem with the previous approaches -- they had different behavior depending on which system triggered the layout. Now there is exactly one algorithm and one basis for the decision making. The other key difference is how the chain is formed. This is based heavily on the idea Andy mentioned of keeping a worklist of blocks that are viable layout successors based on the CFG. Having this set allows us to consistently select the best layout successor for each block. It is expensive though. The code here remains very rough. There is a lot that needs to be done to clean up the code, and to make the runtime cost of this pass much lower. Very much WIP, but this was a giant chunk of code and I'd rather folks see it sooner than later. Everything remains behind a flag of course. I've added a couple of tests to exercise the issues that this iteration was motivated by: loop structure preservation. I've also fixed one test that was exhibiting the broken behavior of the previous version. llvm-svn: 144495
2011-11-13 12:20:44 +01:00
; CHECK: %body1
; CHECK: %body2
; CHECK: %body3
; CHECK: %exit
entry:
br label %body1
body1:
%iv = phi i32 [ 0, %entry ], [ %next, %body3 ]
%base = phi i32 [ 0, %entry ], [ %sum, %body3 ]
%unlikelycond1 = icmp slt i32 %base, 42
br i1 %unlikelycond1, label %unlikely1, label %body2, !prof !0
unlikely1:
call void @error(i32 %i, i32 1, i32 %base)
br label %body2
body2:
%unlikelycond2 = icmp sgt i32 %base, 21
br i1 %unlikelycond2, label %unlikely2, label %body3, !prof !0
unlikely2:
call void @error(i32 %i, i32 2, i32 %base)
br label %body3
body3:
%arrayidx = getelementptr inbounds i32* %a, i32 %iv
%0 = load i32* %arrayidx
%sum = add nsw i32 %0, %base
%next = add i32 %iv, 1
%exitcond = icmp eq i32 %next, %i
br i1 %exitcond, label %exit, label %body1
exit:
ret i32 %sum
}
Add a very basic test for MachineBlockPlacement. This is essentially the canonical example I used when developing it, and is one of the primary motivating real-world use cases for __builtin_expect (when burried under a macro). I'm working on more test cases here, but I'm trying to make sure both that the pass is doing the right thing with the test cases and that they aren't too brittle to changes elsewhere in the code generation pipeline. Feedback and/or suggestions on how to test this are very welcome. Especially feedback on whether testing the block comments is a good strategy; I couldn't find any good examples to steal from but all the other ideas I had were a lot uglier or more fragile. llvm-svn: 142644
2011-10-21 10:01:56 +02:00
!0 = metadata !{metadata !"branch_weights", i32 4, i32 64}
Add loop aligning to MachineBlockPlacement based on review discussion so it's a bit more plausible to use this instead of CodePlacementOpt. The code for this was shamelessly stolen from CodePlacementOpt, and then trimmed down a bit. There doesn't seem to be much utility in returning true/false from this pass as we may or may not have rewritten all of the blocks. Also, the statistic of counting how many loops were aligned doesn't seem terribly important so I removed it. If folks would like it to be included, I'm happy to add it back. This was probably the most egregious of the missing features, and now I'm going to start gathering some performance numbers and looking at specific loop structures that have different layout between the two. Test is updated to include both basic loop alignment and nested loop alignment. llvm-svn: 142645
2011-10-21 10:57:37 +02:00
Rewrite #3 of machine block placement. This is based somewhat on the second algorithm, but only loosely. It is more heavily based on the last discussion I had with Andy. It continues to walk from the inner-most loop outward, but there is a key difference. With this algorithm we ensure that as we visit each loop, the entire loop is merged into a single chain. At the end, the entire function is treated as a "loop", and merged into a single chain. This chain forms the desired sequence of blocks within the function. Switching to a single algorithm removes my biggest problem with the previous approaches -- they had different behavior depending on which system triggered the layout. Now there is exactly one algorithm and one basis for the decision making. The other key difference is how the chain is formed. This is based heavily on the idea Andy mentioned of keeping a worklist of blocks that are viable layout successors based on the CFG. Having this set allows us to consistently select the best layout successor for each block. It is expensive though. The code here remains very rough. There is a lot that needs to be done to clean up the code, and to make the runtime cost of this pass much lower. Very much WIP, but this was a giant chunk of code and I'd rather folks see it sooner than later. Everything remains behind a flag of course. I've added a couple of tests to exercise the issues that this iteration was motivated by: loop structure preservation. I've also fixed one test that was exhibiting the broken behavior of the previous version. llvm-svn: 144495
2011-11-13 12:20:44 +01:00
define i32 @test_loop_early_exits(i32 %i, i32* %a) {
; Check that we sink early exit blocks out of loop bodies.
Convert CodeGen/*/*.ll tests to use the new CHECK-LABEL for easier debugging. No functionality change and all tests pass after conversion. This was done with the following sed invocation to catch label lines demarking function boundaries: sed -i '' "s/^;\( *\)\([A-Z0-9_]*\):\( *\)test\([A-Za-z0-9_-]*\):\( *\)$/;\1\2-LABEL:\3test\4:\5/g" test/CodeGen/*/*.ll which was written conservatively to avoid false positives rather than false negatives. I scanned through all the changes and everything looks correct. llvm-svn: 186258
2013-07-13 22:38:47 +02:00
; CHECK-LABEL: test_loop_early_exits:
Rewrite #3 of machine block placement. This is based somewhat on the second algorithm, but only loosely. It is more heavily based on the last discussion I had with Andy. It continues to walk from the inner-most loop outward, but there is a key difference. With this algorithm we ensure that as we visit each loop, the entire loop is merged into a single chain. At the end, the entire function is treated as a "loop", and merged into a single chain. This chain forms the desired sequence of blocks within the function. Switching to a single algorithm removes my biggest problem with the previous approaches -- they had different behavior depending on which system triggered the layout. Now there is exactly one algorithm and one basis for the decision making. The other key difference is how the chain is formed. This is based heavily on the idea Andy mentioned of keeping a worklist of blocks that are viable layout successors based on the CFG. Having this set allows us to consistently select the best layout successor for each block. It is expensive though. The code here remains very rough. There is a lot that needs to be done to clean up the code, and to make the runtime cost of this pass much lower. Very much WIP, but this was a giant chunk of code and I'd rather folks see it sooner than later. Everything remains behind a flag of course. I've added a couple of tests to exercise the issues that this iteration was motivated by: loop structure preservation. I've also fixed one test that was exhibiting the broken behavior of the previous version. llvm-svn: 144495
2011-11-13 12:20:44 +01:00
; CHECK: %entry
Tweak the loop rotation logic to check whether the loop is naturally laid out in a form with a fallthrough into the header and a fallthrough out of the bottom. In that case, leave the loop alone because any rotation will introduce unnecessary branches. If either side looks like it will require an explicit branch, then the rotation won't add any, do it to ensure the branch occurs outside of the loop (if possible) and maximize the benefit of the fallthrough in the bottom. llvm-svn: 154806
2012-04-16 11:31:23 +02:00
; CHECK: %body1
Rewrite #3 of machine block placement. This is based somewhat on the second algorithm, but only loosely. It is more heavily based on the last discussion I had with Andy. It continues to walk from the inner-most loop outward, but there is a key difference. With this algorithm we ensure that as we visit each loop, the entire loop is merged into a single chain. At the end, the entire function is treated as a "loop", and merged into a single chain. This chain forms the desired sequence of blocks within the function. Switching to a single algorithm removes my biggest problem with the previous approaches -- they had different behavior depending on which system triggered the layout. Now there is exactly one algorithm and one basis for the decision making. The other key difference is how the chain is formed. This is based heavily on the idea Andy mentioned of keeping a worklist of blocks that are viable layout successors based on the CFG. Having this set allows us to consistently select the best layout successor for each block. It is expensive though. The code here remains very rough. There is a lot that needs to be done to clean up the code, and to make the runtime cost of this pass much lower. Very much WIP, but this was a giant chunk of code and I'd rather folks see it sooner than later. Everything remains behind a flag of course. I've added a couple of tests to exercise the issues that this iteration was motivated by: loop structure preservation. I've also fixed one test that was exhibiting the broken behavior of the previous version. llvm-svn: 144495
2011-11-13 12:20:44 +01:00
; CHECK: %body2
; CHECK: %body3
; CHECK: %body4
Tweak the loop rotation logic to check whether the loop is naturally laid out in a form with a fallthrough into the header and a fallthrough out of the bottom. In that case, leave the loop alone because any rotation will introduce unnecessary branches. If either side looks like it will require an explicit branch, then the rotation won't add any, do it to ensure the branch occurs outside of the loop (if possible) and maximize the benefit of the fallthrough in the bottom. llvm-svn: 154806
2012-04-16 11:31:23 +02:00
; CHECK: %exit
Rewrite #3 of machine block placement. This is based somewhat on the second algorithm, but only loosely. It is more heavily based on the last discussion I had with Andy. It continues to walk from the inner-most loop outward, but there is a key difference. With this algorithm we ensure that as we visit each loop, the entire loop is merged into a single chain. At the end, the entire function is treated as a "loop", and merged into a single chain. This chain forms the desired sequence of blocks within the function. Switching to a single algorithm removes my biggest problem with the previous approaches -- they had different behavior depending on which system triggered the layout. Now there is exactly one algorithm and one basis for the decision making. The other key difference is how the chain is formed. This is based heavily on the idea Andy mentioned of keeping a worklist of blocks that are viable layout successors based on the CFG. Having this set allows us to consistently select the best layout successor for each block. It is expensive though. The code here remains very rough. There is a lot that needs to be done to clean up the code, and to make the runtime cost of this pass much lower. Very much WIP, but this was a giant chunk of code and I'd rather folks see it sooner than later. Everything remains behind a flag of course. I've added a couple of tests to exercise the issues that this iteration was motivated by: loop structure preservation. I've also fixed one test that was exhibiting the broken behavior of the previous version. llvm-svn: 144495
2011-11-13 12:20:44 +01:00
; CHECK: %bail1
; CHECK: %bail2
; CHECK: %bail3
entry:
br label %body1
body1:
%iv = phi i32 [ 0, %entry ], [ %next, %body4 ]
%base = phi i32 [ 0, %entry ], [ %sum, %body4 ]
%bailcond1 = icmp eq i32 %base, 42
br i1 %bailcond1, label %bail1, label %body2
bail1:
ret i32 -1
body2:
%bailcond2 = icmp eq i32 %base, 43
br i1 %bailcond2, label %bail2, label %body3
bail2:
ret i32 -2
body3:
%bailcond3 = icmp eq i32 %base, 44
br i1 %bailcond3, label %bail3, label %body4
bail3:
ret i32 -3
body4:
%arrayidx = getelementptr inbounds i32* %a, i32 %iv
%0 = load i32* %arrayidx
%sum = add nsw i32 %0, %base
%next = add i32 %iv, 1
%exitcond = icmp eq i32 %next, %i
br i1 %exitcond, label %exit, label %body1
exit:
ret i32 %sum
}
Introduce a loop block rotation optimization to the new block placement pass. This is designed to achieve one of the important optimizations that the old code placement pass did, but more simply. This is a somewhat rough and *very* conservative version of the transform. We could get a lot fancier here if there are profitable cases to do so. In particular, this only looks for a single pattern, it insists that the loop backedge being rotated away is the last backedge in the chain, and it doesn't provide any means of doing better in-loop placement due to the rotation. However, it appears that it will handle the important loops I am finding in the LLVM test suite. llvm-svn: 145158
2011-11-27 01:38:03 +01:00
define i32 @test_loop_rotate(i32 %i, i32* %a) {
; Check that we rotate conditional exits from the loop to the bottom of the
; loop, eliminating unconditional branches to the top.
Convert CodeGen/*/*.ll tests to use the new CHECK-LABEL for easier debugging. No functionality change and all tests pass after conversion. This was done with the following sed invocation to catch label lines demarking function boundaries: sed -i '' "s/^;\( *\)\([A-Z0-9_]*\):\( *\)test\([A-Za-z0-9_-]*\):\( *\)$/;\1\2-LABEL:\3test\4:\5/g" test/CodeGen/*/*.ll which was written conservatively to avoid false positives rather than false negatives. I scanned through all the changes and everything looks correct. llvm-svn: 186258
2013-07-13 22:38:47 +02:00
; CHECK-LABEL: test_loop_rotate:
Introduce a loop block rotation optimization to the new block placement pass. This is designed to achieve one of the important optimizations that the old code placement pass did, but more simply. This is a somewhat rough and *very* conservative version of the transform. We could get a lot fancier here if there are profitable cases to do so. In particular, this only looks for a single pattern, it insists that the loop backedge being rotated away is the last backedge in the chain, and it doesn't provide any means of doing better in-loop placement due to the rotation. However, it appears that it will handle the important loops I am finding in the LLVM test suite. llvm-svn: 145158
2011-11-27 01:38:03 +01:00
; CHECK: %entry
; CHECK: %body1
; CHECK: %body0
; CHECK: %exit
entry:
br label %body0
body0:
%iv = phi i32 [ 0, %entry ], [ %next, %body1 ]
%base = phi i32 [ 0, %entry ], [ %sum, %body1 ]
%next = add i32 %iv, 1
%exitcond = icmp eq i32 %next, %i
br i1 %exitcond, label %exit, label %body1
body1:
%arrayidx = getelementptr inbounds i32* %a, i32 %iv
%0 = load i32* %arrayidx
%sum = add nsw i32 %0, %base
%bailcond1 = icmp eq i32 %sum, 42
br label %body0
exit:
ret i32 %base
}
Tweak the loop rotation logic to check whether the loop is naturally laid out in a form with a fallthrough into the header and a fallthrough out of the bottom. In that case, leave the loop alone because any rotation will introduce unnecessary branches. If either side looks like it will require an explicit branch, then the rotation won't add any, do it to ensure the branch occurs outside of the loop (if possible) and maximize the benefit of the fallthrough in the bottom. llvm-svn: 154806
2012-04-16 11:31:23 +02:00
define i32 @test_no_loop_rotate(i32 %i, i32* %a) {
; Check that we don't try to rotate a loop which is already laid out with
; fallthrough opportunities into the top and out of the bottom.
Convert CodeGen/*/*.ll tests to use the new CHECK-LABEL for easier debugging. No functionality change and all tests pass after conversion. This was done with the following sed invocation to catch label lines demarking function boundaries: sed -i '' "s/^;\( *\)\([A-Z0-9_]*\):\( *\)test\([A-Za-z0-9_-]*\):\( *\)$/;\1\2-LABEL:\3test\4:\5/g" test/CodeGen/*/*.ll which was written conservatively to avoid false positives rather than false negatives. I scanned through all the changes and everything looks correct. llvm-svn: 186258
2013-07-13 22:38:47 +02:00
; CHECK-LABEL: test_no_loop_rotate:
Tweak the loop rotation logic to check whether the loop is naturally laid out in a form with a fallthrough into the header and a fallthrough out of the bottom. In that case, leave the loop alone because any rotation will introduce unnecessary branches. If either side looks like it will require an explicit branch, then the rotation won't add any, do it to ensure the branch occurs outside of the loop (if possible) and maximize the benefit of the fallthrough in the bottom. llvm-svn: 154806
2012-04-16 11:31:23 +02:00
; CHECK: %entry
; CHECK: %body0
; CHECK: %body1
; CHECK: %exit
entry:
br label %body0
body0:
%iv = phi i32 [ 0, %entry ], [ %next, %body1 ]
%base = phi i32 [ 0, %entry ], [ %sum, %body1 ]
%arrayidx = getelementptr inbounds i32* %a, i32 %iv
%0 = load i32* %arrayidx
%sum = add nsw i32 %0, %base
%bailcond1 = icmp eq i32 %sum, 42
br i1 %bailcond1, label %exit, label %body1
body1:
%next = add i32 %iv, 1
%exitcond = icmp eq i32 %next, %i
br i1 %exitcond, label %exit, label %body0
exit:
ret i32 %base
}
Rework a bit of the implementation of loop block rotation to not rely so heavily on AnalyzeBranch. That routine doesn't behave as we want given that rotation occurs mid-way through re-ordering the function. Instead merely check that there are not unanalyzable branching constructs present, and then reason about the CFG via successor lists. This actually simplifies my mental model for all of this as well. The concrete result is that we now will rotate more loop chains. I've added a test case from Olden highlighting the effect. There is still a bit more to do here though in order to regain all of the performance in Olden. llvm-svn: 145179
2011-11-27 10:22:53 +01:00
define void @test_loop_rotate_reversed_blocks() {
; This test case (greatly reduced from an Olden bencmark) ensures that the loop
; rotate implementation doesn't assume that loops are laid out in a particular
; order. The first loop will get split into two basic blocks, with the loop
; header coming after the loop latch.
;
; CHECK: test_loop_rotate_reversed_blocks
; CHECK: %entry
; Look for a jump into the middle of the loop, and no branches mid-way.
; CHECK: jmp
; CHECK: %loop1
; CHECK-NOT: j{{\w*}} .LBB{{.*}}
; CHECK: %loop1
; CHECK: je
entry:
%cond1 = load volatile i1* undef
br i1 %cond1, label %loop2.preheader, label %loop1
loop1:
call i32 @f()
%cond2 = load volatile i1* undef
br i1 %cond2, label %loop2.preheader, label %loop1
loop2.preheader:
call i32 @f()
%cond3 = load volatile i1* undef
br i1 %cond3, label %exit, label %loop2
loop2:
call i32 @f()
%cond4 = load volatile i1* undef
br i1 %cond4, label %exit, label %loop2
exit:
ret void
}
Add loop aligning to MachineBlockPlacement based on review discussion so it's a bit more plausible to use this instead of CodePlacementOpt. The code for this was shamelessly stolen from CodePlacementOpt, and then trimmed down a bit. There doesn't seem to be much utility in returning true/false from this pass as we may or may not have rewritten all of the blocks. Also, the statistic of counting how many loops were aligned doesn't seem terribly important so I removed it. If folks would like it to be included, I'm happy to add it back. This was probably the most egregious of the missing features, and now I'm going to start gathering some performance numbers and looking at specific loop structures that have different layout between the two. Test is updated to include both basic loop alignment and nested loop alignment. llvm-svn: 142645
2011-10-21 10:57:37 +02:00
define i32 @test_loop_align(i32 %i, i32* %a) {
; Check that we provide basic loop body alignment with the block placement
; pass.
Convert CodeGen/*/*.ll tests to use the new CHECK-LABEL for easier debugging. No functionality change and all tests pass after conversion. This was done with the following sed invocation to catch label lines demarking function boundaries: sed -i '' "s/^;\( *\)\([A-Z0-9_]*\):\( *\)test\([A-Za-z0-9_-]*\):\( *\)$/;\1\2-LABEL:\3test\4:\5/g" test/CodeGen/*/*.ll which was written conservatively to avoid false positives rather than false negatives. I scanned through all the changes and everything looks correct. llvm-svn: 186258
2013-07-13 22:38:47 +02:00
; CHECK-LABEL: test_loop_align:
Add loop aligning to MachineBlockPlacement based on review discussion so it's a bit more plausible to use this instead of CodePlacementOpt. The code for this was shamelessly stolen from CodePlacementOpt, and then trimmed down a bit. There doesn't seem to be much utility in returning true/false from this pass as we may or may not have rewritten all of the blocks. Also, the statistic of counting how many loops were aligned doesn't seem terribly important so I removed it. If folks would like it to be included, I'm happy to add it back. This was probably the most egregious of the missing features, and now I'm going to start gathering some performance numbers and looking at specific loop structures that have different layout between the two. Test is updated to include both basic loop alignment and nested loop alignment. llvm-svn: 142645
2011-10-21 10:57:37 +02:00
; CHECK: %entry
Don't hard code the desired alignment for loops -- it isn't 16-bytes on all x86 systems. Sorry for the breakage. llvm-svn: 142656
2011-10-21 18:41:39 +02:00
; CHECK: .align [[ALIGN:[0-9]+]],
Add loop aligning to MachineBlockPlacement based on review discussion so it's a bit more plausible to use this instead of CodePlacementOpt. The code for this was shamelessly stolen from CodePlacementOpt, and then trimmed down a bit. There doesn't seem to be much utility in returning true/false from this pass as we may or may not have rewritten all of the blocks. Also, the statistic of counting how many loops were aligned doesn't seem terribly important so I removed it. If folks would like it to be included, I'm happy to add it back. This was probably the most egregious of the missing features, and now I'm going to start gathering some performance numbers and looking at specific loop structures that have different layout between the two. Test is updated to include both basic loop alignment and nested loop alignment. llvm-svn: 142645
2011-10-21 10:57:37 +02:00
; CHECK-NEXT: %body
; CHECK: %exit
entry:
br label %body
body:
%iv = phi i32 [ 0, %entry ], [ %next, %body ]
%base = phi i32 [ 0, %entry ], [ %sum, %body ]
%arrayidx = getelementptr inbounds i32* %a, i32 %iv
%0 = load i32* %arrayidx
%sum = add nsw i32 %0, %base
%next = add i32 %iv, 1
%exitcond = icmp eq i32 %next, %i
br i1 %exitcond, label %exit, label %body
exit:
ret i32 %sum
}
define i32 @test_nested_loop_align(i32 %i, i32* %a, i32* %b) {
; Check that we provide nested loop body alignment.
Convert CodeGen/*/*.ll tests to use the new CHECK-LABEL for easier debugging. No functionality change and all tests pass after conversion. This was done with the following sed invocation to catch label lines demarking function boundaries: sed -i '' "s/^;\( *\)\([A-Z0-9_]*\):\( *\)test\([A-Za-z0-9_-]*\):\( *\)$/;\1\2-LABEL:\3test\4:\5/g" test/CodeGen/*/*.ll which was written conservatively to avoid false positives rather than false negatives. I scanned through all the changes and everything looks correct. llvm-svn: 186258
2013-07-13 22:38:47 +02:00
; CHECK-LABEL: test_nested_loop_align:
Add loop aligning to MachineBlockPlacement based on review discussion so it's a bit more plausible to use this instead of CodePlacementOpt. The code for this was shamelessly stolen from CodePlacementOpt, and then trimmed down a bit. There doesn't seem to be much utility in returning true/false from this pass as we may or may not have rewritten all of the blocks. Also, the statistic of counting how many loops were aligned doesn't seem terribly important so I removed it. If folks would like it to be included, I'm happy to add it back. This was probably the most egregious of the missing features, and now I'm going to start gathering some performance numbers and looking at specific loop structures that have different layout between the two. Test is updated to include both basic loop alignment and nested loop alignment. llvm-svn: 142645
2011-10-21 10:57:37 +02:00
; CHECK: %entry
Don't hard code the desired alignment for loops -- it isn't 16-bytes on all x86 systems. Sorry for the breakage. llvm-svn: 142656
2011-10-21 18:41:39 +02:00
; CHECK: .align [[ALIGN]],
Rewrite #3 of machine block placement. This is based somewhat on the second algorithm, but only loosely. It is more heavily based on the last discussion I had with Andy. It continues to walk from the inner-most loop outward, but there is a key difference. With this algorithm we ensure that as we visit each loop, the entire loop is merged into a single chain. At the end, the entire function is treated as a "loop", and merged into a single chain. This chain forms the desired sequence of blocks within the function. Switching to a single algorithm removes my biggest problem with the previous approaches -- they had different behavior depending on which system triggered the layout. Now there is exactly one algorithm and one basis for the decision making. The other key difference is how the chain is formed. This is based heavily on the idea Andy mentioned of keeping a worklist of blocks that are viable layout successors based on the CFG. Having this set allows us to consistently select the best layout successor for each block. It is expensive though. The code here remains very rough. There is a lot that needs to be done to clean up the code, and to make the runtime cost of this pass much lower. Very much WIP, but this was a giant chunk of code and I'd rather folks see it sooner than later. Everything remains behind a flag of course. I've added a couple of tests to exercise the issues that this iteration was motivated by: loop structure preservation. I've also fixed one test that was exhibiting the broken behavior of the previous version. llvm-svn: 144495
2011-11-13 12:20:44 +01:00
; CHECK-NEXT: %loop.body.1
Don't hard code the desired alignment for loops -- it isn't 16-bytes on all x86 systems. Sorry for the breakage. llvm-svn: 142656
2011-10-21 18:41:39 +02:00
; CHECK: .align [[ALIGN]],
Add loop aligning to MachineBlockPlacement based on review discussion so it's a bit more plausible to use this instead of CodePlacementOpt. The code for this was shamelessly stolen from CodePlacementOpt, and then trimmed down a bit. There doesn't seem to be much utility in returning true/false from this pass as we may or may not have rewritten all of the blocks. Also, the statistic of counting how many loops were aligned doesn't seem terribly important so I removed it. If folks would like it to be included, I'm happy to add it back. This was probably the most egregious of the missing features, and now I'm going to start gathering some performance numbers and looking at specific loop structures that have different layout between the two. Test is updated to include both basic loop alignment and nested loop alignment. llvm-svn: 142645
2011-10-21 10:57:37 +02:00
; CHECK-NEXT: %inner.loop.body
; CHECK-NOT: .align
; CHECK: %exit
entry:
br label %loop.body.1
loop.body.1:
%iv = phi i32 [ 0, %entry ], [ %next, %loop.body.2 ]
%arrayidx = getelementptr inbounds i32* %a, i32 %iv
%bidx = load i32* %arrayidx
br label %inner.loop.body
inner.loop.body:
%inner.iv = phi i32 [ 0, %loop.body.1 ], [ %inner.next, %inner.loop.body ]
%base = phi i32 [ 0, %loop.body.1 ], [ %sum, %inner.loop.body ]
%scaled_idx = mul i32 %bidx, %iv
%inner.arrayidx = getelementptr inbounds i32* %b, i32 %scaled_idx
%0 = load i32* %inner.arrayidx
%sum = add nsw i32 %0, %base
%inner.next = add i32 %iv, 1
%inner.exitcond = icmp eq i32 %inner.next, %i
br i1 %inner.exitcond, label %loop.body.2, label %inner.loop.body
loop.body.2:
%next = add i32 %iv, 1
%exitcond = icmp eq i32 %next, %i
br i1 %exitcond, label %exit, label %loop.body.1
exit:
ret i32 %sum
}
Teach machine block placement to cope with unnatural loops. These don't get loop info structures associated with them, and so we need some way to make forward progress selecting and placing basic blocks. The technique used here is pretty brutal -- it just scans the list of blocks looking for the first unplaced candidate. It keeps placing blocks like this until the CFG becomes tractable. The cost is somewhat unfortunate, it requires allocating a vector of all basic block pointers eagerly. I have some ideas about how to simplify and optimize this, but I'm trying to get the logic correct first. Thanks to Benjamin Kramer for the reduced test case out of GCC. Sadly there are other bugs that GCC is tickling that I'm reducing and working on now. llvm-svn: 144516
2011-11-14 01:00:35 +01:00
define void @unnatural_cfg1() {
; Test that we can handle a loop with an inner unnatural loop at the end of
; a function. This is a gross CFG reduced out of the single source GCC.
; CHECK: unnatural_cfg1
; CHECK: %entry
; CHECK: %loop.body1
; CHECK: %loop.body2
Rather than trying to use the loop block sequence *or* the function block sequence when recovering from unanalyzable control flow constructs, *always* use the function sequence. I'm not sure why I ever went down the path of trying to use the loop sequence, it is fundamentally not the correct sequence to use. We're trying to preserve the incoming layout in the cases of unreasonable control flow, and that is only encoded at the function level. We already have a filter to select *exactly* the sub-set of blocks within the function that we're trying to form into a chain. The resulting code layout is also significantly better because of this. In several places we were ending up with completely unreasonable control flow constructs due to the ordering chosen by the loop structure for its internal storage. This change removes a completely wasteful vector of basic blocks, saving memory allocation in the common case even though it costs us CPU in the fairly rare case of unnatural loops. Finally, it fixes the latest crasher reduced out of GCC's single source. Thanks again to Benjamin Kramer for the reduction, my bugpoint skills failed at it. llvm-svn: 144627
2011-11-15 07:26:43 +01:00
; CHECK: %loop.body3
Teach machine block placement to cope with unnatural loops. These don't get loop info structures associated with them, and so we need some way to make forward progress selecting and placing basic blocks. The technique used here is pretty brutal -- it just scans the list of blocks looking for the first unplaced candidate. It keeps placing blocks like this until the CFG becomes tractable. The cost is somewhat unfortunate, it requires allocating a vector of all basic block pointers eagerly. I have some ideas about how to simplify and optimize this, but I'm trying to get the logic correct first. Thanks to Benjamin Kramer for the reduced test case out of GCC. Sadly there are other bugs that GCC is tickling that I'm reducing and working on now. llvm-svn: 144516
2011-11-14 01:00:35 +01:00
entry:
br label %loop.header
loop.header:
br label %loop.body1
loop.body1:
br i1 undef, label %loop.body3, label %loop.body2
loop.body2:
%ptr = load i32** undef, align 4
br label %loop.body3
loop.body3:
%myptr = phi i32* [ %ptr2, %loop.body5 ], [ %ptr, %loop.body2 ], [ undef, %loop.body1 ]
%bcmyptr = bitcast i32* %myptr to i32*
%val = load i32* %bcmyptr, align 4
%comp = icmp eq i32 %val, 48
br i1 %comp, label %loop.body4, label %loop.body5
loop.body4:
br i1 undef, label %loop.header, label %loop.body5
loop.body5:
%ptr2 = load i32** undef, align 4
br label %loop.body3
}
Fix an overflow bug in MachineBranchProbabilityInfo. This pass relied on the sum of the edge weights not overflowing uint32, and crashed when they did. This is generally safe as BranchProbabilityInfo tries to provide this guarantee. However, the CFG can get modified during codegen in a way that grows the *sum* of the edge weights. This doesn't seem unreasonable (imagine just adding more blocks all with the default weight of 16), but it is hard to come up with a case that actually triggers 32-bit overflow. Fortuately, the single-source GCC build is good at this. The solution isn't very pretty, but its no worse than the previous code. We're already summing all of the edge weights on each query, we can sum them, check for an overflow, compute a scale, and sum them again. I've included a *greatly* reduced test case out of the GCC source that triggers it. It's a pretty lame test, as it clearly is just barely triggering the overflow. I'd like to have something that is much more definitive, but I don't understand the fundamental pattern that triggers an explosion in the edge weight sums. The buggy code is duplicated within this file. I'll colapse them into a single implementation in a subsequent commit. llvm-svn: 144526
2011-11-14 09:50:16 +01:00
Rather than trying to use the loop block sequence *or* the function block sequence when recovering from unanalyzable control flow constructs, *always* use the function sequence. I'm not sure why I ever went down the path of trying to use the loop sequence, it is fundamentally not the correct sequence to use. We're trying to preserve the incoming layout in the cases of unreasonable control flow, and that is only encoded at the function level. We already have a filter to select *exactly* the sub-set of blocks within the function that we're trying to form into a chain. The resulting code layout is also significantly better because of this. In several places we were ending up with completely unreasonable control flow constructs due to the ordering chosen by the loop structure for its internal storage. This change removes a completely wasteful vector of basic blocks, saving memory allocation in the common case even though it costs us CPU in the fairly rare case of unnatural loops. Finally, it fixes the latest crasher reduced out of GCC's single source. Thanks again to Benjamin Kramer for the reduction, my bugpoint skills failed at it. llvm-svn: 144627
2011-11-15 07:26:43 +01:00
define void @unnatural_cfg2() {
; Test that we can handle a loop with a nested natural loop *and* an unnatural
; loop. This was reduced from a crash on block placement when run over
; single-source GCC.
; CHECK: unnatural_cfg2
; CHECK: %entry
; CHECK: %loop.body1
; CHECK: %loop.body2
; CHECK: %loop.body3
; CHECK: %loop.inner1.begin
; The end block is folded with %loop.body3...
; CHECK-NOT: %loop.inner1.end
; CHECK: %loop.body4
; CHECK: %loop.inner2.begin
; The loop.inner2.end block is folded
Rewrite how machine block placement handles loop rotation. This is a complex change that resulted from a great deal of experimentation with several different benchmarks. The one which proved the most useful is included as a test case, but I don't know that it captures all of the relevant changes, as I didn't have specific regression tests for each, they were more the result of reasoning about what the old algorithm would possibly do wrong. I'm also failing at the moment to craft more targeted regression tests for these changes, if anyone has ideas, it would be welcome. The first big thing broken with the old algorithm is the idea that we can take a basic block which has a loop-exiting successor and a looping successor and use the looping successor as the layout top in order to get that particular block to be the bottom of the loop after layout. This happens to work in many cases, but not in all. The second big thing broken was that we didn't try to select the exit which fell into the nearest enclosing loop (to which we exit at all). As a consequence, even if the rotation worked perfectly, it would result in one of two bad layouts. Either the bottom of the loop would get fallthrough, skipping across a nearer enclosing loop and thereby making it discontiguous, or it would be forced to take an explicit jump over the nearest enclosing loop to earch its successor. The point of the rotation is to get fallthrough, so we need it to fallthrough to the nearest loop it can. The fix to the first issue is to actually layout the loop from the loop header, and then rotate the loop such that the correct exiting edge can be a fallthrough edge. This is actually much easier than I anticipated because we can handle all the hard parts of finding a viable rotation before we do the layout. We just store that, and then rotate after layout is finished. No inner loops get split across the post-rotation backedge because we check for them when selecting the rotation. That fix exposed a latent problem with our exitting block selection -- we should allow the backedge to point into the middle of some inner-loop chain as there is no real penalty to it, the whole point is that it *won't* be a fallthrough edge. This may have blocked the rotation at all in some cases, I have no idea and no test case as I've never seen it in practice, it was just noticed by inspection. Finally, all of these fixes, and studying the loops they produce, highlighted another problem: in rotating loops like this, we sometimes fail to align the destination of these backwards jumping edges. Fix this by actually walking the backwards edges rather than relying on loopinfo. This fixes regressions on heapsort if block placement is enabled as well as lots of other cases where the previous logic would introduce an abundance of unnecessary branches into the execution. llvm-svn: 154783
2012-04-16 03:12:56 +02:00
; CHECK: %loop.header
Rather than trying to use the loop block sequence *or* the function block sequence when recovering from unanalyzable control flow constructs, *always* use the function sequence. I'm not sure why I ever went down the path of trying to use the loop sequence, it is fundamentally not the correct sequence to use. We're trying to preserve the incoming layout in the cases of unreasonable control flow, and that is only encoded at the function level. We already have a filter to select *exactly* the sub-set of blocks within the function that we're trying to form into a chain. The resulting code layout is also significantly better because of this. In several places we were ending up with completely unreasonable control flow constructs due to the ordering chosen by the loop structure for its internal storage. This change removes a completely wasteful vector of basic blocks, saving memory allocation in the common case even though it costs us CPU in the fairly rare case of unnatural loops. Finally, it fixes the latest crasher reduced out of GCC's single source. Thanks again to Benjamin Kramer for the reduction, my bugpoint skills failed at it. llvm-svn: 144627
2011-11-15 07:26:43 +01:00
; CHECK: %bail
entry:
br label %loop.header
loop.header:
%comp0 = icmp eq i32* undef, null
br i1 %comp0, label %bail, label %loop.body1
loop.body1:
%val0 = load i32** undef, align 4
br i1 undef, label %loop.body2, label %loop.inner1.begin
loop.body2:
br i1 undef, label %loop.body4, label %loop.body3
loop.body3:
%ptr1 = getelementptr inbounds i32* %val0, i32 0
%castptr1 = bitcast i32* %ptr1 to i32**
%val1 = load i32** %castptr1, align 4
br label %loop.inner1.begin
loop.inner1.begin:
%valphi = phi i32* [ %val2, %loop.inner1.end ], [ %val1, %loop.body3 ], [ %val0, %loop.body1 ]
%castval = bitcast i32* %valphi to i32*
%comp1 = icmp eq i32 undef, 48
br i1 %comp1, label %loop.inner1.end, label %loop.body4
loop.inner1.end:
%ptr2 = getelementptr inbounds i32* %valphi, i32 0
%castptr2 = bitcast i32* %ptr2 to i32**
%val2 = load i32** %castptr2, align 4
br label %loop.inner1.begin
loop.body4.dead:
br label %loop.body4
loop.body4:
%comp2 = icmp ult i32 undef, 3
br i1 %comp2, label %loop.inner2.begin, label %loop.end
loop.inner2.begin:
br i1 false, label %loop.end, label %loop.inner2.end
loop.inner2.end:
%comp3 = icmp eq i32 undef, 1769472
br i1 %comp3, label %loop.end, label %loop.inner2.begin
loop.end:
br label %loop.header
bail:
unreachable
}
Fix an overflow bug in MachineBranchProbabilityInfo. This pass relied on the sum of the edge weights not overflowing uint32, and crashed when they did. This is generally safe as BranchProbabilityInfo tries to provide this guarantee. However, the CFG can get modified during codegen in a way that grows the *sum* of the edge weights. This doesn't seem unreasonable (imagine just adding more blocks all with the default weight of 16), but it is hard to come up with a case that actually triggers 32-bit overflow. Fortuately, the single-source GCC build is good at this. The solution isn't very pretty, but its no worse than the previous code. We're already summing all of the edge weights on each query, we can sum them, check for an overflow, compute a scale, and sum them again. I've included a *greatly* reduced test case out of the GCC source that triggers it. It's a pretty lame test, as it clearly is just barely triggering the overflow. I'd like to have something that is much more definitive, but I don't understand the fundamental pattern that triggers an explosion in the edge weight sums. The buggy code is duplicated within this file. I'll colapse them into a single implementation in a subsequent commit. llvm-svn: 144526
2011-11-14 09:50:16 +01:00
define i32 @problematic_switch() {
; This function's CFG caused overlow in the machine branch probability
; calculation, triggering asserts. Make sure we don't crash on it.
; CHECK: problematic_switch
entry:
switch i32 undef, label %exit [
i32 879, label %bogus
i32 877, label %step
i32 876, label %step
i32 875, label %step
i32 874, label %step
i32 873, label %step
i32 872, label %step
i32 868, label %step
i32 867, label %step
i32 866, label %step
i32 861, label %step
i32 860, label %step
i32 856, label %step
i32 855, label %step
i32 854, label %step
i32 831, label %step
i32 830, label %step
i32 829, label %step
i32 828, label %step
i32 815, label %step
i32 814, label %step
i32 811, label %step
i32 806, label %step
i32 805, label %step
i32 804, label %step
i32 803, label %step
i32 802, label %step
i32 801, label %step
i32 800, label %step
i32 799, label %step
i32 798, label %step
i32 797, label %step
i32 796, label %step
i32 795, label %step
]
bogus:
unreachable
step:
br label %exit
exit:
%merge = phi i32 [ 3, %step ], [ 6, %entry ]
ret i32 %merge
}
Move the handling of unanalyzable branches out of the loop-driven chain formation phase and into the initial walk of the basic blocks. We essentially pre-merge all blocks where unanalyzable fallthrough exists, as we won't be able to update the terminators effectively after any reorderings. This is quite a bit more principled as there may be CFGs where the second half of the unanalyzable pair has some analyzable predecessor that gets placed first. Then it may get placed next, implicitly breaking the unanalyzable branch even though we never even looked at the part that isn't analyzable. I've included a test case that triggers this (thanks Benjamin yet again!), and I'm hoping to synthesize some more general ones as I dig into related issues. Also, to make this new scheme work we have to be able to handle branches into the middle of a chain, so add this check. We always fallback on the incoming ordering. Finally, this starts to really underscore a known limitation of the current implementation -- we don't consider broken predecessors when merging successors. This can caused major missed opportunities, and is something I'm planning on looking at next (modulo more bug reports). llvm-svn: 144994
2011-11-19 11:26:02 +01:00
define void @fpcmp_unanalyzable_branch(i1 %cond) {
Add some comments to the latest test case I added here to document what is actually being tested. Also add some FileCheck goodness to much more carefully ensure that the result is the desired result. Before this test would only have failed through an assert failure if the underlying fix were reverted. Also, add some weight metadata and a comment explaining exactly what is going on to a trick section of the test case. Originally, we were getting very unlucky and trying to form a block chain that isn't actually profitable. I'm working on a fix to avoid forming these unprofitable chains, and that would also have masked any failure from this test case. The easy solution is to add some metadata that makes it *really* profitable to form the bad chain here. llvm-svn: 145006
2011-11-20 10:30:40 +01:00
; This function's CFG contains an unanalyzable branch that is likely to be
; split due to having a different high-probability predecessor.
; CHECK: fpcmp_unanalyzable_branch
; CHECK: %entry
; CHECK: %exit
; CHECK-NOT: %if.then
; CHECK-NOT: %if.end
; CHECK-NOT: jne
; CHECK-NOT: jnp
; CHECK: jne
; CHECK-NEXT: jnp
; CHECK-NEXT: %if.then
Move the handling of unanalyzable branches out of the loop-driven chain formation phase and into the initial walk of the basic blocks. We essentially pre-merge all blocks where unanalyzable fallthrough exists, as we won't be able to update the terminators effectively after any reorderings. This is quite a bit more principled as there may be CFGs where the second half of the unanalyzable pair has some analyzable predecessor that gets placed first. Then it may get placed next, implicitly breaking the unanalyzable branch even though we never even looked at the part that isn't analyzable. I've included a test case that triggers this (thanks Benjamin yet again!), and I'm hoping to synthesize some more general ones as I dig into related issues. Also, to make this new scheme work we have to be able to handle branches into the middle of a chain, so add this check. We always fallback on the incoming ordering. Finally, this starts to really underscore a known limitation of the current implementation -- we don't consider broken predecessors when merging successors. This can caused major missed opportunities, and is something I'm planning on looking at next (modulo more bug reports). llvm-svn: 144994
2011-11-19 11:26:02 +01:00
entry:
Add some comments to the latest test case I added here to document what is actually being tested. Also add some FileCheck goodness to much more carefully ensure that the result is the desired result. Before this test would only have failed through an assert failure if the underlying fix were reverted. Also, add some weight metadata and a comment explaining exactly what is going on to a trick section of the test case. Originally, we were getting very unlucky and trying to form a block chain that isn't actually profitable. I'm working on a fix to avoid forming these unprofitable chains, and that would also have masked any failure from this test case. The easy solution is to add some metadata that makes it *really* profitable to form the bad chain here. llvm-svn: 145006
2011-11-20 10:30:40 +01:00
; Note that this branch must be strongly biased toward
; 'entry.if.then_crit_edge' to ensure that we would try to form a chain for
; 'entry' -> 'entry.if.then_crit_edge' -> 'if.then'. It is the last edge in that
; chain which would violate the unanalyzable branch in 'exit', but we won't even
; try this trick unless 'if.then' is believed to almost always be reached from
; 'entry.if.then_crit_edge'.
br i1 %cond, label %entry.if.then_crit_edge, label %lor.lhs.false, !prof !1
Move the handling of unanalyzable branches out of the loop-driven chain formation phase and into the initial walk of the basic blocks. We essentially pre-merge all blocks where unanalyzable fallthrough exists, as we won't be able to update the terminators effectively after any reorderings. This is quite a bit more principled as there may be CFGs where the second half of the unanalyzable pair has some analyzable predecessor that gets placed first. Then it may get placed next, implicitly breaking the unanalyzable branch even though we never even looked at the part that isn't analyzable. I've included a test case that triggers this (thanks Benjamin yet again!), and I'm hoping to synthesize some more general ones as I dig into related issues. Also, to make this new scheme work we have to be able to handle branches into the middle of a chain, so add this check. We always fallback on the incoming ordering. Finally, this starts to really underscore a known limitation of the current implementation -- we don't consider broken predecessors when merging successors. This can caused major missed opportunities, and is something I'm planning on looking at next (modulo more bug reports). llvm-svn: 144994
2011-11-19 11:26:02 +01:00
entry.if.then_crit_edge:
TBAA: remove !tbaa from testing cases if not used. This will make it easier to turn on struct-path aware TBAA since the metadata format will change. llvm-svn: 180796
2013-04-30 19:52:57 +02:00
%.pre14 = load i8* undef, align 1
Move the handling of unanalyzable branches out of the loop-driven chain formation phase and into the initial walk of the basic blocks. We essentially pre-merge all blocks where unanalyzable fallthrough exists, as we won't be able to update the terminators effectively after any reorderings. This is quite a bit more principled as there may be CFGs where the second half of the unanalyzable pair has some analyzable predecessor that gets placed first. Then it may get placed next, implicitly breaking the unanalyzable branch even though we never even looked at the part that isn't analyzable. I've included a test case that triggers this (thanks Benjamin yet again!), and I'm hoping to synthesize some more general ones as I dig into related issues. Also, to make this new scheme work we have to be able to handle branches into the middle of a chain, so add this check. We always fallback on the incoming ordering. Finally, this starts to really underscore a known limitation of the current implementation -- we don't consider broken predecessors when merging successors. This can caused major missed opportunities, and is something I'm planning on looking at next (modulo more bug reports). llvm-svn: 144994
2011-11-19 11:26:02 +01:00
br label %if.then
lor.lhs.false:
br i1 undef, label %if.end, label %exit
exit:
%cmp.i = fcmp une double 0.000000e+00, undef
br i1 %cmp.i, label %if.then, label %if.end
if.then:
%0 = phi i8 [ %.pre14, %entry.if.then_crit_edge ], [ undef, %exit ]
%1 = and i8 %0, 1
TBAA: remove !tbaa from testing cases if not used. This will make it easier to turn on struct-path aware TBAA since the metadata format will change. llvm-svn: 180796
2013-04-30 19:52:57 +02:00
store i8 %1, i8* undef, align 4
Move the handling of unanalyzable branches out of the loop-driven chain formation phase and into the initial walk of the basic blocks. We essentially pre-merge all blocks where unanalyzable fallthrough exists, as we won't be able to update the terminators effectively after any reorderings. This is quite a bit more principled as there may be CFGs where the second half of the unanalyzable pair has some analyzable predecessor that gets placed first. Then it may get placed next, implicitly breaking the unanalyzable branch even though we never even looked at the part that isn't analyzable. I've included a test case that triggers this (thanks Benjamin yet again!), and I'm hoping to synthesize some more general ones as I dig into related issues. Also, to make this new scheme work we have to be able to handle branches into the middle of a chain, so add this check. We always fallback on the incoming ordering. Finally, this starts to really underscore a known limitation of the current implementation -- we don't consider broken predecessors when merging successors. This can caused major missed opportunities, and is something I'm planning on looking at next (modulo more bug reports). llvm-svn: 144994
2011-11-19 11:26:02 +01:00
br label %if.end
if.end:
ret void
}
Add some comments to the latest test case I added here to document what is actually being tested. Also add some FileCheck goodness to much more carefully ensure that the result is the desired result. Before this test would only have failed through an assert failure if the underlying fix were reverted. Also, add some weight metadata and a comment explaining exactly what is going on to a trick section of the test case. Originally, we were getting very unlucky and trying to form a block chain that isn't actually profitable. I'm working on a fix to avoid forming these unprofitable chains, and that would also have masked any failure from this test case. The easy solution is to add some metadata that makes it *really* profitable to form the bad chain here. llvm-svn: 145006
2011-11-20 10:30:40 +01:00
!1 = metadata !{metadata !"branch_weights", i32 1000, i32 1}
The logic for breaking the CFG in the presence of hot successors didn't properly account for the *global* probability of the edge being taken. This manifested as a very large number of unconditional branches to blocks being merged against the CFG even though they weren't particularly hot within the CFG. The fix is to check whether the edge being merged is both locally hot relative to other successors for the source block, and globally hot compared to other (unmerged) predecessors of the destination block. This introduces a new crasher on GCC single-source, but it's currently behind a flag, and Ben has offered to work on the reduction. =] llvm-svn: 145010
2011-11-20 12:22:06 +01:00
declare i32 @f()
declare i32 @g()
declare i32 @h(i32 %x)
define i32 @test_global_cfg_break_profitability() {
; Check that our metrics for the profitability of a CFG break are global rather
; than local. A successor may be very hot, but if the current block isn't, it
; doesn't matter. Within this test the 'then' block is slightly warmer than the
; 'else' block, but not nearly enough to merit merging it with the exit block
; even though the probability of 'then' branching to the 'exit' block is very
; high.
; CHECK: test_global_cfg_break_profitability
test/CodeGen/X86/block-placement.ll: Relax expressions for Win32. llvm-svn: 145011
2011-11-20 13:49:45 +01:00
; CHECK: calll {{_?}}f
; CHECK: calll {{_?}}g
; CHECK: calll {{_?}}h
The logic for breaking the CFG in the presence of hot successors didn't properly account for the *global* probability of the edge being taken. This manifested as a very large number of unconditional branches to blocks being merged against the CFG even though they weren't particularly hot within the CFG. The fix is to check whether the edge being merged is both locally hot relative to other successors for the source block, and globally hot compared to other (unmerged) predecessors of the destination block. This introduces a new crasher on GCC single-source, but it's currently behind a flag, and Ben has offered to work on the reduction. =] llvm-svn: 145010
2011-11-20 12:22:06 +01:00
; CHECK: ret
entry:
br i1 undef, label %then, label %else, !prof !2
then:
%then.result = call i32 @f()
br label %exit
else:
%else.result = call i32 @g()
br label %exit
exit:
%result = phi i32 [ %then.result, %then ], [ %else.result, %else ]
%result2 = call i32 @h(i32 %result)
ret i32 %result
}
!2 = metadata !{metadata !"branch_weights", i32 3, i32 1}
Fix a devilish miscompile exposed by block placement. The updateTerminator code didn't correctly handle EH terminators in one very specific case. AnalyzeBranch would find no terminator instruction, and so the fallback in updateTerminator is to assume fallthrough. This is correct, but the destination of the fallthrough was assumed to be the first successor. This is *almost always* true, but in certain cases the loop transformations will cause the landing pad to be the first successor! Instead of this brittle logic, actually look through the successors for a non-landing-pad accessor, and to assert if more than one is found. This will hopefully fix some (if not all) of the self host miscompiles with block placement. Thanks to Benjamin Kramer for reporting, Nick Lewycky for an initial stab at a reduction, and Duncan for endless advice on EH (which I know nothing about) as well as reviewing the actual fix. llvm-svn: 145062
2011-11-22 14:13:16 +01:00
declare i32 @__gxx_personality_v0(...)
define void @test_eh_lpad_successor() {
; Some times the landing pad ends up as the first successor of an invoke block.
; When this happens, a strange result used to fall out of updateTerminators: we
; didn't correctly locate the fallthrough successor, assuming blindly that the
; first one was the fallthrough successor. As a result, we would add an
; erroneous jump to the landing pad thinking *that* was the default successor.
; CHECK: test_eh_lpad_successor
; CHECK: %entry
; CHECK-NOT: jmp
; CHECK: %loop
entry:
invoke i32 @f() to label %preheader unwind label %lpad
preheader:
br label %loop
lpad:
%lpad.val = landingpad { i8*, i32 } personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*)
cleanup
resume { i8*, i32 } %lpad.val
loop:
br label %loop
}
Fix a crash in block placement due to an inner loop that happened to be reversed in the function's original ordering, and we happened to encounter it while handling an outer unnatural CFG structure. Thanks to the test case reduced from GCC's source by Benjamin Kramer. This may also fix a crasher in gzip that Duncan reduced for me, but I haven't yet gotten to testing that one. llvm-svn: 145094
2011-11-23 04:03:21 +01:00
Handle the case of a no-return invoke correctly. It actually still has successors, they just are all landing pad successors. We handle this the same way as no successors. Comments attached for the next person to wade through here and another lovely test case courtesy of Benjamin Kramer's bugpoint reduction. llvm-svn: 145098
2011-11-23 09:23:54 +01:00
declare void @fake_throw() noreturn
define void @test_eh_throw() {
; For blocks containing a 'throw' (or similar functionality), we have
; a no-return invoke. In this case, only EH successors will exist, and
; fallthrough simply won't occur. Make sure we don't crash trying to update
; terminators for such constructs.
;
; CHECK: test_eh_throw
; CHECK: %entry
; CHECK: %cleanup
entry:
invoke void @fake_throw() to label %continue unwind label %cleanup
continue:
unreachable
cleanup:
%0 = landingpad { i8*, i32 } personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*)
cleanup
unreachable
}
Fix a crash in block placement due to an inner loop that happened to be reversed in the function's original ordering, and we happened to encounter it while handling an outer unnatural CFG structure. Thanks to the test case reduced from GCC's source by Benjamin Kramer. This may also fix a crasher in gzip that Duncan reduced for me, but I haven't yet gotten to testing that one. llvm-svn: 145094
2011-11-23 04:03:21 +01:00
define void @test_unnatural_cfg_backwards_inner_loop() {
; Test that when we encounter an unnatural CFG structure after having formed
; a chain for an inner loop which happened to be laid out backwards we don't
; attempt to merge onto the wrong end of the inner loop just because we find it
; first. This was reduced from a crasher in GCC's single source.
;
; CHECK: test_unnatural_cfg_backwards_inner_loop
; CHECK: %entry
MachineCSE: Update the heuristics for isProfitableToCSE. If the result of a common subexpression is used at all uses of the candidate expression, CSE should not increase the live range of the common subexpression. rdar://11393714 and rdar://11819721 llvm-svn: 161396
2012-08-07 08:16:46 +02:00
; CHECK: [[BODY:# BB#[0-9]+]]:
Fix a crash in block placement due to an inner loop that happened to be reversed in the function's original ordering, and we happened to encounter it while handling an outer unnatural CFG structure. Thanks to the test case reduced from GCC's source by Benjamin Kramer. This may also fix a crasher in gzip that Duncan reduced for me, but I haven't yet gotten to testing that one. llvm-svn: 145094
2011-11-23 04:03:21 +01:00
; CHECK: %loop2b
Rework a bit of the implementation of loop block rotation to not rely so heavily on AnalyzeBranch. That routine doesn't behave as we want given that rotation occurs mid-way through re-ordering the function. Instead merely check that there are not unanalyzable branching constructs present, and then reason about the CFG via successor lists. This actually simplifies my mental model for all of this as well. The concrete result is that we now will rotate more loop chains. I've added a test case from Olden highlighting the effect. There is still a bit more to do here though in order to regain all of the performance in Olden. llvm-svn: 145179
2011-11-27 10:22:53 +01:00
; CHECK: %loop1
Fix a crash in block placement due to an inner loop that happened to be reversed in the function's original ordering, and we happened to encounter it while handling an outer unnatural CFG structure. Thanks to the test case reduced from GCC's source by Benjamin Kramer. This may also fix a crasher in gzip that Duncan reduced for me, but I haven't yet gotten to testing that one. llvm-svn: 145094
2011-11-23 04:03:21 +01:00
; CHECK: %loop2a
entry:
br i1 undef, label %loop2a, label %body
body:
br label %loop2a
loop1:
%next.load = load i32** undef
br i1 %comp.a, label %loop2a, label %loop2b
loop2a:
%var = phi i32* [ null, %entry ], [ null, %body ], [ %next.phi, %loop1 ]
%next.var = phi i32* [ null, %entry ], [ undef, %body ], [ %next.load, %loop1 ]
%comp.a = icmp eq i32* %var, null
br label %loop3
loop2b:
%gep = getelementptr inbounds i32* %var.phi, i32 0
%next.ptr = bitcast i32* %gep to i32**
store i32* %next.phi, i32** %next.ptr
br label %loop3
loop3:
%var.phi = phi i32* [ %next.phi, %loop2b ], [ %var, %loop2a ]
%next.phi = phi i32* [ %next.load, %loop2b ], [ %next.var, %loop2a ]
br label %loop1
}
Relax an invariant that block placement was trying to assert a bit further. This invariant just wasn't going to work in the face of unanalyzable branches; we need to be resillient to the phenomenon of chains poking into a loop and poking out of a loop. In fact, we already were, we just needed to not assert on it. This was found during a bootstrap with block placement turned on. llvm-svn: 145100
2011-11-23 11:35:36 +01:00
define void @unanalyzable_branch_to_loop_header() {
; Ensure that we can handle unanalyzable branches into loop headers. We
; pre-form chains for unanalyzable branches, and will find the tail end of that
; at the start of the loop. This function uses floating point comparison
; fallthrough because that happens to always produce unanalyzable branches on
; x86.
;
; CHECK: unanalyzable_branch_to_loop_header
; CHECK: %entry
; CHECK: %loop
; CHECK: %exit
entry:
%cmp = fcmp une double 0.000000e+00, undef
br i1 %cmp, label %loop, label %exit
loop:
%cond = icmp eq i8 undef, 42
br i1 %cond, label %exit, label %loop
exit:
ret void
}
When adding blocks to the list of those which no longer have any CFG conflicts, we should only be adding the first block of the chain to the list, lest we try to merge into the middle of that chain. Most of the places we were doing this we already happened to be looking at the first block, but there is no reason to assume that, and in some cases it was clearly wrong. I've added a couple of tests here. One already worked, but I like having an explicit test for it. The other is reduced from a test case Duncan reduced for me and used to crash. Now it is handled correctly. llvm-svn: 145119
2011-11-24 09:46:04 +01:00
define void @unanalyzable_branch_to_best_succ(i1 %cond) {
; Ensure that we can handle unanalyzable branches where the destination block
; gets selected as the optimal sucessor to merge.
;
; CHECK: unanalyzable_branch_to_best_succ
; CHECK: %entry
; CHECK: %foo
; CHECK: %bar
; CHECK: %exit
entry:
; Bias this branch toward bar to ensure we form that chain.
br i1 %cond, label %bar, label %foo, !prof !1
foo:
%cmp = fcmp une double 0.000000e+00, undef
br i1 %cmp, label %bar, label %exit
bar:
call i32 @f()
br label %exit
exit:
ret void
}
define void @unanalyzable_branch_to_free_block(float %x) {
; Ensure that we can handle unanalyzable branches where the destination block
; gets selected as the best free block in the CFG.
;
; CHECK: unanalyzable_branch_to_free_block
; CHECK: %entry
; CHECK: %a
; CHECK: %b
; CHECK: %c
; CHECK: %exit
entry:
br i1 undef, label %a, label %b
a:
call i32 @f()
br label %c
b:
%cmp = fcmp une float %x, undef
br i1 %cmp, label %c, label %exit
c:
call i32 @g()
br label %exit
exit:
ret void
}
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
define void @many_unanalyzable_branches() {
; Ensure that we don't crash as we're building up many unanalyzable branches,
; blocks, and loops.
;
; CHECK: many_unanalyzable_branches
; CHECK: %entry
; CHECK: %exit
entry:
br label %0
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val0 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp0 = fcmp une float %val0, undef
br i1 %cmp0, label %1, label %0
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val1 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp1 = fcmp une float %val1, undef
br i1 %cmp1, label %2, label %1
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val2 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp2 = fcmp une float %val2, undef
br i1 %cmp2, label %3, label %2
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val3 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp3 = fcmp une float %val3, undef
br i1 %cmp3, label %4, label %3
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val4 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp4 = fcmp une float %val4, undef
br i1 %cmp4, label %5, label %4
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val5 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp5 = fcmp une float %val5, undef
br i1 %cmp5, label %6, label %5
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val6 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp6 = fcmp une float %val6, undef
br i1 %cmp6, label %7, label %6
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val7 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp7 = fcmp une float %val7, undef
br i1 %cmp7, label %8, label %7
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val8 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp8 = fcmp une float %val8, undef
br i1 %cmp8, label %9, label %8
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val9 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp9 = fcmp une float %val9, undef
br i1 %cmp9, label %10, label %9
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val10 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp10 = fcmp une float %val10, undef
br i1 %cmp10, label %11, label %10
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val11 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp11 = fcmp une float %val11, undef
br i1 %cmp11, label %12, label %11
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val12 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp12 = fcmp une float %val12, undef
br i1 %cmp12, label %13, label %12
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val13 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp13 = fcmp une float %val13, undef
br i1 %cmp13, label %14, label %13
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val14 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp14 = fcmp une float %val14, undef
br i1 %cmp14, label %15, label %14
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val15 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp15 = fcmp une float %val15, undef
br i1 %cmp15, label %16, label %15
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val16 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp16 = fcmp une float %val16, undef
br i1 %cmp16, label %17, label %16
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val17 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp17 = fcmp une float %val17, undef
br i1 %cmp17, label %18, label %17
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val18 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp18 = fcmp une float %val18, undef
br i1 %cmp18, label %19, label %18
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val19 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp19 = fcmp une float %val19, undef
br i1 %cmp19, label %20, label %19
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val20 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp20 = fcmp une float %val20, undef
br i1 %cmp20, label %21, label %20
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val21 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp21 = fcmp une float %val21, undef
br i1 %cmp21, label %22, label %21
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val22 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp22 = fcmp une float %val22, undef
br i1 %cmp22, label %23, label %22
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val23 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp23 = fcmp une float %val23, undef
br i1 %cmp23, label %24, label %23
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val24 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp24 = fcmp une float %val24, undef
br i1 %cmp24, label %25, label %24
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val25 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp25 = fcmp une float %val25, undef
br i1 %cmp25, label %26, label %25
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val26 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp26 = fcmp une float %val26, undef
br i1 %cmp26, label %27, label %26
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val27 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp27 = fcmp une float %val27, undef
br i1 %cmp27, label %28, label %27
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val28 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp28 = fcmp une float %val28, undef
br i1 %cmp28, label %29, label %28
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val29 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp29 = fcmp une float %val29, undef
br i1 %cmp29, label %30, label %29
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val30 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp30 = fcmp une float %val30, undef
br i1 %cmp30, label %31, label %30
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val31 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp31 = fcmp une float %val31, undef
br i1 %cmp31, label %32, label %31
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val32 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp32 = fcmp une float %val32, undef
br i1 %cmp32, label %33, label %32
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val33 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp33 = fcmp une float %val33, undef
br i1 %cmp33, label %34, label %33
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val34 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp34 = fcmp une float %val34, undef
br i1 %cmp34, label %35, label %34
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val35 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp35 = fcmp une float %val35, undef
br i1 %cmp35, label %36, label %35
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val36 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp36 = fcmp une float %val36, undef
br i1 %cmp36, label %37, label %36
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val37 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp37 = fcmp une float %val37, undef
br i1 %cmp37, label %38, label %37
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val38 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp38 = fcmp une float %val38, undef
br i1 %cmp38, label %39, label %38
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val39 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp39 = fcmp une float %val39, undef
br i1 %cmp39, label %40, label %39
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val40 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp40 = fcmp une float %val40, undef
br i1 %cmp40, label %41, label %40
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val41 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp41 = fcmp une float %val41, undef
br i1 %cmp41, label %42, label %41
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val42 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp42 = fcmp une float %val42, undef
br i1 %cmp42, label %43, label %42
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val43 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp43 = fcmp une float %val43, undef
br i1 %cmp43, label %44, label %43
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val44 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp44 = fcmp une float %val44, undef
br i1 %cmp44, label %45, label %44
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val45 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp45 = fcmp une float %val45, undef
br i1 %cmp45, label %46, label %45
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val46 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp46 = fcmp une float %val46, undef
br i1 %cmp46, label %47, label %46
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val47 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp47 = fcmp une float %val47, undef
br i1 %cmp47, label %48, label %47
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val48 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp48 = fcmp une float %val48, undef
br i1 %cmp48, label %49, label %48
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val49 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp49 = fcmp une float %val49, undef
br i1 %cmp49, label %50, label %49
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val50 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp50 = fcmp une float %val50, undef
br i1 %cmp50, label %51, label %50
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val51 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp51 = fcmp une float %val51, undef
br i1 %cmp51, label %52, label %51
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val52 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp52 = fcmp une float %val52, undef
br i1 %cmp52, label %53, label %52
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val53 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp53 = fcmp une float %val53, undef
br i1 %cmp53, label %54, label %53
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val54 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp54 = fcmp une float %val54, undef
br i1 %cmp54, label %55, label %54
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val55 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp55 = fcmp une float %val55, undef
br i1 %cmp55, label %56, label %55
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val56 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp56 = fcmp une float %val56, undef
br i1 %cmp56, label %57, label %56
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val57 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp57 = fcmp une float %val57, undef
br i1 %cmp57, label %58, label %57
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val58 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp58 = fcmp une float %val58, undef
br i1 %cmp58, label %59, label %58
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val59 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp59 = fcmp une float %val59, undef
br i1 %cmp59, label %60, label %59
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val60 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp60 = fcmp une float %val60, undef
br i1 %cmp60, label %61, label %60
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val61 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp61 = fcmp une float %val61, undef
br i1 %cmp61, label %62, label %61
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val62 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp62 = fcmp une float %val62, undef
br i1 %cmp62, label %63, label %62
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val63 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp63 = fcmp une float %val63, undef
br i1 %cmp63, label %64, label %63
Upgrade syntax of tests using volatile instructions to use 'load volatile' instead of 'volatile load', which is archaic. llvm-svn: 145171
2011-11-27 07:54:59 +01:00
%val64 = load volatile float* undef
Fix a silly use-after-free issue. A much earlier version of this code need lots of fanciness around retaining a reference to a Chain's slot in the BlockToChain map, but that's all gone now. We can just go directly to allocating the new chain (which will update the mapping for us) and using it. Somewhat gross mechanically generated test case replicates the issue Duncan spotted when actually testing this out. llvm-svn: 145120
2011-11-24 12:23:15 +01:00
%cmp64 = fcmp une float %val64, undef
br i1 %cmp64, label %65, label %64
br label %exit
exit:
ret void
}
Rewrite how machine block placement handles loop rotation. This is a complex change that resulted from a great deal of experimentation with several different benchmarks. The one which proved the most useful is included as a test case, but I don't know that it captures all of the relevant changes, as I didn't have specific regression tests for each, they were more the result of reasoning about what the old algorithm would possibly do wrong. I'm also failing at the moment to craft more targeted regression tests for these changes, if anyone has ideas, it would be welcome. The first big thing broken with the old algorithm is the idea that we can take a basic block which has a loop-exiting successor and a looping successor and use the looping successor as the layout top in order to get that particular block to be the bottom of the loop after layout. This happens to work in many cases, but not in all. The second big thing broken was that we didn't try to select the exit which fell into the nearest enclosing loop (to which we exit at all). As a consequence, even if the rotation worked perfectly, it would result in one of two bad layouts. Either the bottom of the loop would get fallthrough, skipping across a nearer enclosing loop and thereby making it discontiguous, or it would be forced to take an explicit jump over the nearest enclosing loop to earch its successor. The point of the rotation is to get fallthrough, so we need it to fallthrough to the nearest loop it can. The fix to the first issue is to actually layout the loop from the loop header, and then rotate the loop such that the correct exiting edge can be a fallthrough edge. This is actually much easier than I anticipated because we can handle all the hard parts of finding a viable rotation before we do the layout. We just store that, and then rotate after layout is finished. No inner loops get split across the post-rotation backedge because we check for them when selecting the rotation. That fix exposed a latent problem with our exitting block selection -- we should allow the backedge to point into the middle of some inner-loop chain as there is no real penalty to it, the whole point is that it *won't* be a fallthrough edge. This may have blocked the rotation at all in some cases, I have no idea and no test case as I've never seen it in practice, it was just noticed by inspection. Finally, all of these fixes, and studying the loops they produce, highlighted another problem: in rotating loops like this, we sometimes fail to align the destination of these backwards jumping edges. Fix this by actually walking the backwards edges rather than relying on loopinfo. This fixes regressions on heapsort if block placement is enabled as well as lots of other cases where the previous logic would introduce an abundance of unnecessary branches into the execution. llvm-svn: 154783
2012-04-16 03:12:56 +02:00
define void @benchmark_heapsort(i32 %n, double* nocapture %ra) {
; This test case comes from the heapsort benchmark, and exemplifies several
; important aspects to block placement in the presence of loops:
; 1) Loop rotation needs to *ensure* that the desired exiting edge can be
; a fallthrough.
; 2) The exiting edge from the loop which is rotated to be laid out at the
; bottom of the loop needs to be exiting into the nearest enclosing loop (to
; which there is an exit). Otherwise, we force that enclosing loop into
; strange layouts that are siginificantly less efficient, often times maing
; it discontiguous.
;
; CHECK: @benchmark_heapsort
; CHECK: %entry
; First rotated loop top.
; CHECK: .align
Add a somewhat hacky heuristic to do something different from whole-loop rotation. When there is a loop backedge which is an unconditional branch, we will end up with a branch somewhere no matter what. Try placing this backedge in a fallthrough position above the loop header as that will definitely remove at least one branch from the loop iteration, where whole loop rotation may not. I haven't seen any benchmarks where this is important but loop-blocks.ll tests for it, and so this will be covered when I flip the default. llvm-svn: 154812
2012-04-16 15:33:36 +02:00
; CHECK: %while.end
; CHECK: %for.cond
; CHECK: %if.then
; CHECK: %if.else
Rewrite how machine block placement handles loop rotation. This is a complex change that resulted from a great deal of experimentation with several different benchmarks. The one which proved the most useful is included as a test case, but I don't know that it captures all of the relevant changes, as I didn't have specific regression tests for each, they were more the result of reasoning about what the old algorithm would possibly do wrong. I'm also failing at the moment to craft more targeted regression tests for these changes, if anyone has ideas, it would be welcome. The first big thing broken with the old algorithm is the idea that we can take a basic block which has a loop-exiting successor and a looping successor and use the looping successor as the layout top in order to get that particular block to be the bottom of the loop after layout. This happens to work in many cases, but not in all. The second big thing broken was that we didn't try to select the exit which fell into the nearest enclosing loop (to which we exit at all). As a consequence, even if the rotation worked perfectly, it would result in one of two bad layouts. Either the bottom of the loop would get fallthrough, skipping across a nearer enclosing loop and thereby making it discontiguous, or it would be forced to take an explicit jump over the nearest enclosing loop to earch its successor. The point of the rotation is to get fallthrough, so we need it to fallthrough to the nearest loop it can. The fix to the first issue is to actually layout the loop from the loop header, and then rotate the loop such that the correct exiting edge can be a fallthrough edge. This is actually much easier than I anticipated because we can handle all the hard parts of finding a viable rotation before we do the layout. We just store that, and then rotate after layout is finished. No inner loops get split across the post-rotation backedge because we check for them when selecting the rotation. That fix exposed a latent problem with our exitting block selection -- we should allow the backedge to point into the middle of some inner-loop chain as there is no real penalty to it, the whole point is that it *won't* be a fallthrough edge. This may have blocked the rotation at all in some cases, I have no idea and no test case as I've never seen it in practice, it was just noticed by inspection. Finally, all of these fixes, and studying the loops they produce, highlighted another problem: in rotating loops like this, we sometimes fail to align the destination of these backwards jumping edges. Fix this by actually walking the backwards edges rather than relying on loopinfo. This fixes regressions on heapsort if block placement is enabled as well as lots of other cases where the previous logic would introduce an abundance of unnecessary branches into the execution. llvm-svn: 154783
2012-04-16 03:12:56 +02:00
; CHECK: %if.end10
; Second rotated loop top
; CHECK: .align
Add a somewhat hacky heuristic to do something different from whole-loop rotation. When there is a loop backedge which is an unconditional branch, we will end up with a branch somewhere no matter what. Try placing this backedge in a fallthrough position above the loop header as that will definitely remove at least one branch from the loop iteration, where whole loop rotation may not. I haven't seen any benchmarks where this is important but loop-blocks.ll tests for it, and so this will be covered when I flip the default. llvm-svn: 154812
2012-04-16 15:33:36 +02:00
; CHECK: %if.then24
Rewrite how machine block placement handles loop rotation. This is a complex change that resulted from a great deal of experimentation with several different benchmarks. The one which proved the most useful is included as a test case, but I don't know that it captures all of the relevant changes, as I didn't have specific regression tests for each, they were more the result of reasoning about what the old algorithm would possibly do wrong. I'm also failing at the moment to craft more targeted regression tests for these changes, if anyone has ideas, it would be welcome. The first big thing broken with the old algorithm is the idea that we can take a basic block which has a loop-exiting successor and a looping successor and use the looping successor as the layout top in order to get that particular block to be the bottom of the loop after layout. This happens to work in many cases, but not in all. The second big thing broken was that we didn't try to select the exit which fell into the nearest enclosing loop (to which we exit at all). As a consequence, even if the rotation worked perfectly, it would result in one of two bad layouts. Either the bottom of the loop would get fallthrough, skipping across a nearer enclosing loop and thereby making it discontiguous, or it would be forced to take an explicit jump over the nearest enclosing loop to earch its successor. The point of the rotation is to get fallthrough, so we need it to fallthrough to the nearest loop it can. The fix to the first issue is to actually layout the loop from the loop header, and then rotate the loop such that the correct exiting edge can be a fallthrough edge. This is actually much easier than I anticipated because we can handle all the hard parts of finding a viable rotation before we do the layout. We just store that, and then rotate after layout is finished. No inner loops get split across the post-rotation backedge because we check for them when selecting the rotation. That fix exposed a latent problem with our exitting block selection -- we should allow the backedge to point into the middle of some inner-loop chain as there is no real penalty to it, the whole point is that it *won't* be a fallthrough edge. This may have blocked the rotation at all in some cases, I have no idea and no test case as I've never seen it in practice, it was just noticed by inspection. Finally, all of these fixes, and studying the loops they produce, highlighted another problem: in rotating loops like this, we sometimes fail to align the destination of these backwards jumping edges. Fix this by actually walking the backwards edges rather than relying on loopinfo. This fixes regressions on heapsort if block placement is enabled as well as lots of other cases where the previous logic would introduce an abundance of unnecessary branches into the execution. llvm-svn: 154783
2012-04-16 03:12:56 +02:00
; CHECK: %while.cond.outer
; Third rotated loop top
; CHECK: .align
; CHECK: %while.cond
; CHECK: %while.body
; CHECK: %land.lhs.true
; CHECK: %if.then19
Fix tail merging to assign the (more) correct BasicBlock when splitting. This makes it possible to write unit tests that are less susceptible to minor code motion, particularly copy placement. block-placement.ll covers this case with -pre-RA-sched=source which will soon be default. One incorrectly named block is already fixed, but without this fix, enabling new coalescing and scheduling would cause more failures. llvm-svn: 184680
2013-06-24 03:55:01 +02:00
; CHECK: %if.end20
Rewrite how machine block placement handles loop rotation. This is a complex change that resulted from a great deal of experimentation with several different benchmarks. The one which proved the most useful is included as a test case, but I don't know that it captures all of the relevant changes, as I didn't have specific regression tests for each, they were more the result of reasoning about what the old algorithm would possibly do wrong. I'm also failing at the moment to craft more targeted regression tests for these changes, if anyone has ideas, it would be welcome. The first big thing broken with the old algorithm is the idea that we can take a basic block which has a loop-exiting successor and a looping successor and use the looping successor as the layout top in order to get that particular block to be the bottom of the loop after layout. This happens to work in many cases, but not in all. The second big thing broken was that we didn't try to select the exit which fell into the nearest enclosing loop (to which we exit at all). As a consequence, even if the rotation worked perfectly, it would result in one of two bad layouts. Either the bottom of the loop would get fallthrough, skipping across a nearer enclosing loop and thereby making it discontiguous, or it would be forced to take an explicit jump over the nearest enclosing loop to earch its successor. The point of the rotation is to get fallthrough, so we need it to fallthrough to the nearest loop it can. The fix to the first issue is to actually layout the loop from the loop header, and then rotate the loop such that the correct exiting edge can be a fallthrough edge. This is actually much easier than I anticipated because we can handle all the hard parts of finding a viable rotation before we do the layout. We just store that, and then rotate after layout is finished. No inner loops get split across the post-rotation backedge because we check for them when selecting the rotation. That fix exposed a latent problem with our exitting block selection -- we should allow the backedge to point into the middle of some inner-loop chain as there is no real penalty to it, the whole point is that it *won't* be a fallthrough edge. This may have blocked the rotation at all in some cases, I have no idea and no test case as I've never seen it in practice, it was just noticed by inspection. Finally, all of these fixes, and studying the loops they produce, highlighted another problem: in rotating loops like this, we sometimes fail to align the destination of these backwards jumping edges. Fix this by actually walking the backwards edges rather than relying on loopinfo. This fixes regressions on heapsort if block placement is enabled as well as lots of other cases where the previous logic would introduce an abundance of unnecessary branches into the execution. llvm-svn: 154783
2012-04-16 03:12:56 +02:00
; CHECK: %if.then8
; CHECK: ret
entry:
%shr = ashr i32 %n, 1
%add = add nsw i32 %shr, 1
%arrayidx3 = getelementptr inbounds double* %ra, i64 1
br label %for.cond
for.cond:
%ir.0 = phi i32 [ %n, %entry ], [ %ir.1, %while.end ]
%l.0 = phi i32 [ %add, %entry ], [ %l.1, %while.end ]
%cmp = icmp sgt i32 %l.0, 1
br i1 %cmp, label %if.then, label %if.else
if.then:
%dec = add nsw i32 %l.0, -1
%idxprom = sext i32 %dec to i64
%arrayidx = getelementptr inbounds double* %ra, i64 %idxprom
%0 = load double* %arrayidx, align 8
br label %if.end10
if.else:
%idxprom1 = sext i32 %ir.0 to i64
%arrayidx2 = getelementptr inbounds double* %ra, i64 %idxprom1
%1 = load double* %arrayidx2, align 8
%2 = load double* %arrayidx3, align 8
store double %2, double* %arrayidx2, align 8
%dec6 = add nsw i32 %ir.0, -1
%cmp7 = icmp eq i32 %dec6, 1
br i1 %cmp7, label %if.then8, label %if.end10
if.then8:
store double %1, double* %arrayidx3, align 8
ret void
if.end10:
%ir.1 = phi i32 [ %ir.0, %if.then ], [ %dec6, %if.else ]
%l.1 = phi i32 [ %dec, %if.then ], [ %l.0, %if.else ]
%rra.0 = phi double [ %0, %if.then ], [ %1, %if.else ]
%add31 = add nsw i32 %ir.1, 1
br label %while.cond.outer
while.cond.outer:
%j.0.ph.in = phi i32 [ %l.1, %if.end10 ], [ %j.1, %if.then24 ]
%j.0.ph = shl i32 %j.0.ph.in, 1
br label %while.cond
while.cond:
%j.0 = phi i32 [ %add31, %if.end20 ], [ %j.0.ph, %while.cond.outer ]
%cmp11 = icmp sgt i32 %j.0, %ir.1
br i1 %cmp11, label %while.end, label %while.body
while.body:
%cmp12 = icmp slt i32 %j.0, %ir.1
br i1 %cmp12, label %land.lhs.true, label %if.end20
land.lhs.true:
%idxprom13 = sext i32 %j.0 to i64
%arrayidx14 = getelementptr inbounds double* %ra, i64 %idxprom13
%3 = load double* %arrayidx14, align 8
%add15 = add nsw i32 %j.0, 1
%idxprom16 = sext i32 %add15 to i64
%arrayidx17 = getelementptr inbounds double* %ra, i64 %idxprom16
%4 = load double* %arrayidx17, align 8
%cmp18 = fcmp olt double %3, %4
br i1 %cmp18, label %if.then19, label %if.end20
if.then19:
br label %if.end20
if.end20:
%j.1 = phi i32 [ %add15, %if.then19 ], [ %j.0, %land.lhs.true ], [ %j.0, %while.body ]
%idxprom21 = sext i32 %j.1 to i64
%arrayidx22 = getelementptr inbounds double* %ra, i64 %idxprom21
%5 = load double* %arrayidx22, align 8
%cmp23 = fcmp olt double %rra.0, %5
br i1 %cmp23, label %if.then24, label %while.cond
if.then24:
%idxprom27 = sext i32 %j.0.ph.in to i64
%arrayidx28 = getelementptr inbounds double* %ra, i64 %idxprom27
store double %5, double* %arrayidx28, align 8
br label %while.cond.outer
while.end:
%idxprom33 = sext i32 %j.0.ph.in to i64
%arrayidx34 = getelementptr inbounds double* %ra, i64 %idxprom33
store double %rra.0, double* %arrayidx34, align 8
br label %for.cond
}
Add a new function attribute 'cold' to functions. Other than recognizing the attribute, the patch does little else. It changes the branch probability analyzer so that edges into blocks postdominated by a cold function are given low weight. Added analysis and code generation tests. Added documentation for the new attribute. llvm-svn: 182638
2013-05-24 14:26:52 +02:00
declare void @cold_function() cold
define i32 @test_cold_calls(i32* %a) {
; Test that edges to blocks post-dominated by cold calls are
; marked as not expected to be taken. They should be laid out
; at the bottom.
Convert CodeGen/*/*.ll tests to use the new CHECK-LABEL for easier debugging. No functionality change and all tests pass after conversion. This was done with the following sed invocation to catch label lines demarking function boundaries: sed -i '' "s/^;\( *\)\([A-Z0-9_]*\):\( *\)test\([A-Za-z0-9_-]*\):\( *\)$/;\1\2-LABEL:\3test\4:\5/g" test/CodeGen/*/*.ll which was written conservatively to avoid false positives rather than false negatives. I scanned through all the changes and everything looks correct. llvm-svn: 186258
2013-07-13 22:38:47 +02:00
; CHECK-LABEL: test_cold_calls:
Add a new function attribute 'cold' to functions. Other than recognizing the attribute, the patch does little else. It changes the branch probability analyzer so that edges into blocks postdominated by a cold function are given low weight. Added analysis and code generation tests. Added documentation for the new attribute. llvm-svn: 182638
2013-05-24 14:26:52 +02:00
; CHECK: %entry
; CHECK: %else
; CHECK: %exit
; CHECK: %then
entry:
%gep1 = getelementptr i32* %a, i32 1
%val1 = load i32* %gep1
%cond1 = icmp ugt i32 %val1, 1
br i1 %cond1, label %then, label %else
then:
call void @cold_function()
br label %exit
else:
%gep2 = getelementptr i32* %a, i32 2
%val2 = load i32* %gep2
br label %exit
exit:
%ret = phi i32 [ %val1, %then ], [ %val2, %else ]
ret i32 %ret
}
Reference in New Issue Copy Permalink