TailRecursionElimination.cpp revision 32b1e87f111d79f3f5b0e86c8f138c02bef8aa18
12240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner//===- TailRecursionElimination.cpp - Eliminate Tail Calls ----------------===// 2fd93908ae8b9684fe71c239e3c6cfe13ff6a2663Misha Brukman// 3b576c94c15af9a440f69d9d03c2afead7971118cJohn Criswell// The LLVM Compiler Infrastructure 4b576c94c15af9a440f69d9d03c2afead7971118cJohn Criswell// 5b576c94c15af9a440f69d9d03c2afead7971118cJohn Criswell// This file was developed by the LLVM research group and is distributed under 6b576c94c15af9a440f69d9d03c2afead7971118cJohn Criswell// the University of Illinois Open Source License. See LICENSE.TXT for details. 7fd93908ae8b9684fe71c239e3c6cfe13ff6a2663Misha Brukman// 8b576c94c15af9a440f69d9d03c2afead7971118cJohn Criswell//===----------------------------------------------------------------------===// 92240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner// 107152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// This file transforms calls of the current function (self recursion) followed 117152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// by a return instruction with a branch to the entry of the function, creating 127152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// a loop. This pass also implements the following extensions to the basic 137152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// algorithm: 142240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner// 157152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// 1. Trivial instructions between the call and return do not prevent the 167152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// transformation from taking place, though currently the analysis cannot 177152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// support moving any really useful instructions (only dead ones). 18543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner// 2. This pass transforms functions that are prevented from being tail 19543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner// recursive by an associative expression to use an accumulator variable, 20543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner// thus compiling the typical naive factorial or 'fib' implementation into 21543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner// efficient code. 22d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner// 3. TRE is performed if the function returns void, if the return 23d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner// returns the result returned by the call, or if the function returns a 24d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner// run-time constant on all exits from the function. It is possible, though 25d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner// unlikely, that the return returns something else (like constant 0), and 26d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner// can still be TRE'd. It can be TRE'd if ALL OTHER return instructions in 27d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner// the function return the exact same value. 287f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner// 4. If it can prove that callees do not access theier caller stack frame, 297f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner// they are marked as eligible for tail call elimination (by the code 307f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner// generator). 312240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner// 327152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// There are several improvements that could be made: 337152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// 347152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// 1. If the function has any alloca instructions, these instructions will be 357152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// moved out of the entry block of the function, causing them to be 367152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// evaluated each time through the tail recursion. Safely keeping allocas 377152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// in the entry block requires analysis to proves that the tail-called 387152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// function does not read or write the stack object. 392240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner// 2. Tail recursion is only performed if the call immediately preceeds the 407152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// return instruction. It's possible that there could be a jump between 417152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// the call and the return. 42d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner// 3. There can be intervening operations between the call and the return that 437152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// prevent the TRE from occurring. For example, there could be GEP's and 447152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// stores to memory that will not be read or written by the call. This 457152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// requires some substantial analysis (such as with DSA) to prove safe to 467152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// move ahead of the call, but doing so could allow many more TREs to be 477152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner// performed, for example in TreeAdd/TreeAlloc from the treeadd benchmark. 487f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner// 4. The algorithm we use to detect if callees access their caller stack 497f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner// frames is very primitive. 502240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner// 512240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner//===----------------------------------------------------------------------===// 522240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner 533fc6ef1bb96d9a3194cef667a2d3cbc94e3fb189Chris Lattner#include "llvm/Transforms/Scalar.h" 54ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner#include "llvm/Constants.h" 552240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner#include "llvm/DerivedTypes.h" 562240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner#include "llvm/Function.h" 572240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner#include "llvm/Instructions.h" 582240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner#include "llvm/Pass.h" 59543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner#include "llvm/Support/CFG.h" 60551ccae044b0ff658fe629dd67edd5ffe75d10e8Reid Spencer#include "llvm/ADT/Statistic.h" 61f8485c643412dbff46fe87ea2867445169a5c28eChris Lattnerusing namespace llvm; 62d0fde30ce850b78371fd1386338350591f9ff494Brian Gaeke 632240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattnernamespace { 642240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner Statistic<> NumEliminated("tailcallelim", "Number of tail calls removed"); 65543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner Statistic<> NumAccumAdded("tailcallelim","Number of accumulators introduced"); 662240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner 672240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner struct TailCallElim : public FunctionPass { 682240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner virtual bool runOnFunction(Function &F); 697152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner 707152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner private: 717152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner bool ProcessReturningBlock(ReturnInst *RI, BasicBlock *&OldEntry, 72ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner bool &TailCallsAreMarkedTail, 73ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner std::vector<PHINode*> &ArgumentPHIs, 74ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner bool CannotTailCallElimCallsMarkedTail); 757152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner bool CanMoveAboveCall(Instruction *I, CallInst *CI); 76543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner Value *CanTransformAccumulatorRecursion(Instruction *I, CallInst *CI); 772240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner }; 787f8897f22e88271cfa114998a4d6088e7c8e8e11Chris Lattner RegisterPass<TailCallElim> X("tailcallelim", "Tail Call Elimination"); 792240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner} 802240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner 81d0fde30ce850b78371fd1386338350591f9ff494Brian Gaeke// Public interface to the TailCallElimination pass 82f8485c643412dbff46fe87ea2867445169a5c28eChris LattnerFunctionPass *llvm::createTailCallEliminationPass() { 83f8485c643412dbff46fe87ea2867445169a5c28eChris Lattner return new TailCallElim(); 84f8485c643412dbff46fe87ea2867445169a5c28eChris Lattner} 853fc6ef1bb96d9a3194cef667a2d3cbc94e3fb189Chris Lattner 862240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner 877f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner/// AllocaMightEscapeToCalls - Return true if this alloca may be accessed by 887f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner/// callees of this function. We only do very simple analysis right now, this 897f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner/// could be expanded in the future to use mod/ref information for particular 907f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner/// call sites if desired. 917f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattnerstatic bool AllocaMightEscapeToCalls(AllocaInst *AI) { 927f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner // FIXME: do simple 'address taken' analysis. 937f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner return true; 947f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner} 957f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner 967f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner/// FunctionContainsAllocas - Scan the specified basic block for alloca 977f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner/// instructions. If it contains any that might be accessed by calls, return 987f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner/// true. 99ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattnerstatic bool CheckForEscapingAllocas(BasicBlock *BB, 100ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner bool &CannotTCETailMarkedCall) { 101ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner bool RetVal = false; 1027f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner for (BasicBlock::iterator I = BB->begin(), E = BB->end(); I != E; ++I) 103ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner if (AllocaInst *AI = dyn_cast<AllocaInst>(I)) { 104ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner RetVal |= AllocaMightEscapeToCalls(AI); 105ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner 106ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner // If this alloca is in the body of the function, or if it is a variable 107ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner // sized allocation, we cannot tail call eliminate calls marked 'tail' 108ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner // with this mechanism. 109ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner if (BB != &BB->getParent()->front() || 110ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner !isa<ConstantInt>(AI->getArraySize())) 111ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner CannotTCETailMarkedCall = true; 112ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner } 113ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner return RetVal; 1147f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner} 1157f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner 1162240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattnerbool TailCallElim::runOnFunction(Function &F) { 1172240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner // If this function is a varargs function, we won't be able to PHI the args 1182240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner // right, so don't even try to convert it... 1192240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner if (F.getFunctionType()->isVarArg()) return false; 1202240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner 1212240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner BasicBlock *OldEntry = 0; 122ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner bool TailCallsAreMarkedTail = false; 1232240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner std::vector<PHINode*> ArgumentPHIs; 1242240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner bool MadeChange = false; 1252240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner 1267f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner bool FunctionContainsEscapingAllocas = false; 1277f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner 128ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner // CannotTCETailMarkedCall - If true, we cannot perform TCE on tail calls 129ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner // marked with the 'tail' attribute, because doing so would cause the stack 130ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner // size to increase (real TCE would deallocate variable sized allocas, TCE 131ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner // doesn't). 132ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner bool CannotTCETailMarkedCall = false; 133ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner 1347f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner // Loop over the function, looking for any returning blocks, and keeping track 1357f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner // of whether this function has any non-trivially used allocas. 1367f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner for (Function::iterator BB = F.begin(), E = F.end(); BB != E; ++BB) { 137ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner if (FunctionContainsEscapingAllocas && CannotTCETailMarkedCall) 138ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner break; 13900b16889ab461b7ecef1c91ade101186b7f1fce2Jeff Cohen 1404c0e4cdc40bbf94766242294fe386f4eb2d32d2eChris Lattner FunctionContainsEscapingAllocas |= 141ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner CheckForEscapingAllocas(BB, CannotTCETailMarkedCall); 1427f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner } 14332b1e87f111d79f3f5b0e86c8f138c02bef8aa18Chris Lattner 14432b1e87f111d79f3f5b0e86c8f138c02bef8aa18Chris Lattner /// FIXME: The code generator produces really bad code when an 'escaping 14532b1e87f111d79f3f5b0e86c8f138c02bef8aa18Chris Lattner /// alloca' is changed from being a static alloca to being a dynamic alloca. 14632b1e87f111d79f3f5b0e86c8f138c02bef8aa18Chris Lattner /// Until this is resolved, disable this transformation if that would ever 14732b1e87f111d79f3f5b0e86c8f138c02bef8aa18Chris Lattner /// happen. This bug is PR962. 14832b1e87f111d79f3f5b0e86c8f138c02bef8aa18Chris Lattner if (FunctionContainsEscapingAllocas) 14932b1e87f111d79f3f5b0e86c8f138c02bef8aa18Chris Lattner return false; 15032b1e87f111d79f3f5b0e86c8f138c02bef8aa18Chris Lattner 151fd93908ae8b9684fe71c239e3c6cfe13ff6a2663Misha Brukman 152ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner // Second pass, change any tail calls to loops. 153ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner for (Function::iterator BB = F.begin(), E = F.end(); BB != E; ++BB) 154ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner if (ReturnInst *Ret = dyn_cast<ReturnInst>(BB->getTerminator())) 155ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner MadeChange |= ProcessReturningBlock(Ret, OldEntry, TailCallsAreMarkedTail, 156ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner ArgumentPHIs,CannotTCETailMarkedCall); 157ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner 158cf2f89251d45bc9783917874adb9029f71d50068Chris Lattner // If we eliminated any tail recursions, it's possible that we inserted some 159cf2f89251d45bc9783917874adb9029f71d50068Chris Lattner // silly PHI nodes which just merge an initial value (the incoming operand) 160cf2f89251d45bc9783917874adb9029f71d50068Chris Lattner // with themselves. Check to see if we did and clean up our mess if so. This 161cf2f89251d45bc9783917874adb9029f71d50068Chris Lattner // occurs when a function passes an argument straight through to its tail 162cf2f89251d45bc9783917874adb9029f71d50068Chris Lattner // call. 163cf2f89251d45bc9783917874adb9029f71d50068Chris Lattner if (!ArgumentPHIs.empty()) { 164cf2f89251d45bc9783917874adb9029f71d50068Chris Lattner unsigned NumIncoming = ArgumentPHIs[0]->getNumIncomingValues(); 165cf2f89251d45bc9783917874adb9029f71d50068Chris Lattner for (unsigned i = 0, e = ArgumentPHIs.size(); i != e; ++i) { 166cf2f89251d45bc9783917874adb9029f71d50068Chris Lattner PHINode *PN = ArgumentPHIs[i]; 167cf2f89251d45bc9783917874adb9029f71d50068Chris Lattner 168cf2f89251d45bc9783917874adb9029f71d50068Chris Lattner // If the PHI Node is a dynamic constant, replace it with the value it is. 169ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner if (Value *PNV = PN->hasConstantValue()) { 170ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner PN->replaceAllUsesWith(PNV); 171ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner PN->eraseFromParent(); 172cf2f89251d45bc9783917874adb9029f71d50068Chris Lattner } 173cf2f89251d45bc9783917874adb9029f71d50068Chris Lattner } 174cf2f89251d45bc9783917874adb9029f71d50068Chris Lattner } 175cf2f89251d45bc9783917874adb9029f71d50068Chris Lattner 1767f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner // Finally, if this function contains no non-escaping allocas, mark all calls 1777f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner // in the function as eligible for tail calls (there is no stack memory for 1787f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner // them to access). 1797f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner if (!FunctionContainsEscapingAllocas) 1807f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner for (Function::iterator BB = F.begin(), E = F.end(); BB != E; ++BB) 1817f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner for (BasicBlock::iterator I = BB->begin(), E = BB->end(); I != E; ++I) 1827f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner if (CallInst *CI = dyn_cast<CallInst>(I)) 1837f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner CI->setTailCall(); 1847f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner 1852240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner return MadeChange; 1862240d2b3f7b9f5835868c83ce78d125d1b65212bChris Lattner} 1877152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner 1887152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner 189543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner/// CanMoveAboveCall - Return true if it is safe to move the specified 190543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner/// instruction from after the call to before the call, assuming that all 191543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner/// instructions between the call and this instruction are movable. 192543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner/// 1937152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattnerbool TailCallElim::CanMoveAboveCall(Instruction *I, CallInst *CI) { 1947152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // FIXME: We can move load/store/call/free instructions above the call if the 1957152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // call does not mod/ref the memory location being processed. 1967152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner if (I->mayWriteToMemory() || isa<LoadInst>(I)) 1977152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner return false; 1987152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner 1997152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // Otherwise, if this is a side-effect free instruction, check to make sure 2007152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // that it does not use the return value of the call. If it doesn't use the 2017152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // return value of the call, it must only use things that are defined before 2027152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // the call, or movable instructions between the call and the instruction 2037152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // itself. 2047152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner for (unsigned i = 0, e = I->getNumOperands(); i != e; ++i) 2057152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner if (I->getOperand(i) == CI) 2067152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner return false; 2077152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner return true; 2087152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner} 2097152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner 210d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner// isDynamicConstant - Return true if the specified value is the same when the 211d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner// return would exit as it was when the initial iteration of the recursive 212d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner// function was executed. 213d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner// 214d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner// We currently handle static constants and arguments that are not modified as 215d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner// part of the recursion. 216d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner// 217d64152a70842b2f4186aa912938e69ca09c1434cChris Lattnerstatic bool isDynamicConstant(Value *V, CallInst *CI) { 218d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner if (isa<Constant>(V)) return true; // Static constants are always dyn consts 219d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner 220d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner // Check to see if this is an immutable argument, if so, the value 221d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner // will be available to initialize the accumulator. 222d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner if (Argument *Arg = dyn_cast<Argument>(V)) { 223d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner // Figure out which argument number this is... 224d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner unsigned ArgNo = 0; 225d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner Function *F = CI->getParent()->getParent(); 226e4d5c441e04bdc00ccf1804744af670655123b07Chris Lattner for (Function::arg_iterator AI = F->arg_begin(); &*AI != Arg; ++AI) 227d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner ++ArgNo; 228fd93908ae8b9684fe71c239e3c6cfe13ff6a2663Misha Brukman 229d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner // If we are passing this argument into call as the corresponding 230d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner // argument operand, then the argument is dynamically constant. 231d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner // Otherwise, we cannot transform this function safely. 232d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner if (CI->getOperand(ArgNo+1) == Arg) 233d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner return true; 234d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner } 235d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner // Not a constant or immutable argument, we can't safely transform. 236d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner return false; 237d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner} 238d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner 239d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner// getCommonReturnValue - Check to see if the function containing the specified 240d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner// return instruction and tail call consistently returns the same 241d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner// runtime-constant value at all exit points. If so, return the returned value. 242d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner// 243d64152a70842b2f4186aa912938e69ca09c1434cChris Lattnerstatic Value *getCommonReturnValue(ReturnInst *TheRI, CallInst *CI) { 244d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner Function *F = TheRI->getParent()->getParent(); 245d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner Value *ReturnedValue = 0; 246d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner 247d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner for (Function::iterator BBI = F->begin(), E = F->end(); BBI != E; ++BBI) 248d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner if (ReturnInst *RI = dyn_cast<ReturnInst>(BBI->getTerminator())) 249d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner if (RI != TheRI) { 250d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner Value *RetOp = RI->getOperand(0); 251d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner 252d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner // We can only perform this transformation if the value returned is 253d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner // evaluatable at the start of the initial invocation of the function, 254d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner // instead of at the end of the evaluation. 255d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner // 256d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner if (!isDynamicConstant(RetOp, CI)) 257d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner return 0; 258d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner 259d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner if (ReturnedValue && RetOp != ReturnedValue) 260d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner return 0; // Cannot transform if differing values are returned. 261d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner ReturnedValue = RetOp; 262d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner } 263d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner return ReturnedValue; 264d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner} 2657152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner 266543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner/// CanTransformAccumulatorRecursion - If the specified instruction can be 267543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner/// transformed using accumulator recursion elimination, return the constant 268543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner/// which is the start of the accumulator value. Otherwise return null. 269543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner/// 270543d622ef7505910c1cdc09ada0ab797c3437590Chris LattnerValue *TailCallElim::CanTransformAccumulatorRecursion(Instruction *I, 271543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner CallInst *CI) { 272543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner if (!I->isAssociative()) return 0; 273543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner assert(I->getNumOperands() == 2 && 274543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner "Associative operations should have 2 args!"); 275543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner 276543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // Exactly one operand should be the result of the call instruction... 277543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner if (I->getOperand(0) == CI && I->getOperand(1) == CI || 278543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner I->getOperand(0) != CI && I->getOperand(1) != CI) 279543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner return 0; 280543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner 281543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // The only user of this instruction we allow is a single return instruction. 282543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner if (!I->hasOneUse() || !isa<ReturnInst>(I->use_back())) 283543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner return 0; 284543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner 285543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // Ok, now we have to check all of the other return instructions in this 286543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // function. If they return non-constants or differing values, then we cannot 287543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // transform the function safely. 288d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner return getCommonReturnValue(cast<ReturnInst>(I->use_back()), CI); 289543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner} 290543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner 2917152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattnerbool TailCallElim::ProcessReturningBlock(ReturnInst *Ret, BasicBlock *&OldEntry, 292ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner bool &TailCallsAreMarkedTail, 293ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner std::vector<PHINode*> &ArgumentPHIs, 294ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner bool CannotTailCallElimCallsMarkedTail) { 2957152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner BasicBlock *BB = Ret->getParent(); 2967152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner Function *F = BB->getParent(); 2977152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner 2987152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner if (&BB->front() == Ret) // Make sure there is something before the ret... 2997152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner return false; 3007152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner 3017152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // Scan backwards from the return, checking to see if there is a tail call in 3027152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // this block. If so, set CI to it. 3037152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner CallInst *CI; 3047152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner BasicBlock::iterator BBI = Ret; 3057152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner while (1) { 3067152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner CI = dyn_cast<CallInst>(BBI); 3077152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner if (CI && CI->getCalledFunction() == F) 3087152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner break; 3097152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner 3107152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner if (BBI == BB->begin()) 3117152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner return false; // Didn't find a potential tail call. 3127152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner --BBI; 3137152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner } 3147152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner 315ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner // If this call is marked as a tail call, and if there are dynamic allocas in 316ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner // the function, we cannot perform this optimization. 317ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner if (CI->isTailCall() && CannotTailCallElimCallsMarkedTail) 318ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner return false; 319ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner 320543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // If we are introducing accumulator recursion to eliminate associative 321543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // operations after the call instruction, this variable contains the initial 322543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // value for the accumulator. If this value is set, we actually perform 323543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // accumulator recursion elimination instead of simple tail recursion 324543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // elimination. 325543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner Value *AccumulatorRecursionEliminationInitVal = 0; 326543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner Instruction *AccumulatorRecursionInstr = 0; 327543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner 3287152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // Ok, we found a potential tail call. We can currently only transform the 3297152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // tail call if all of the instructions between the call and the return are 3307152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // movable to above the call itself, leaving the call next to the return. 3317152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // Check that this is the case now. 3327152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner for (BBI = CI, ++BBI; &*BBI != Ret; ++BBI) 333543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner if (!CanMoveAboveCall(BBI, CI)) { 334543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // If we can't move the instruction above the call, it might be because it 335543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // is an associative operation that could be tranformed using accumulator 336543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // recursion elimination. Check to see if this is the case, and if so, 337543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // remember the initial accumulator value for later. 338543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner if ((AccumulatorRecursionEliminationInitVal = 339543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner CanTransformAccumulatorRecursion(BBI, CI))) { 340543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // Yes, this is accumulator recursion. Remember which instruction 341543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // accumulates. 342543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner AccumulatorRecursionInstr = BBI; 343543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner } else { 344543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner return false; // Otherwise, we cannot eliminate the tail recursion! 345543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner } 346543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner } 3477152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner 3487152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // We can only transform call/return pairs that either ignore the return value 349d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner // of the call and return void, ignore the value of the call and return a 350d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner // constant, return the value returned by the tail call, or that are being 351d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner // accumulator recursion variable eliminated. 352543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner if (Ret->getNumOperands() != 0 && Ret->getReturnValue() != CI && 3533b5f45042b17ad52815e3dd6c0c1df99f196dd04Chris Lattner !isa<UndefValue>(Ret->getReturnValue()) && 354d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner AccumulatorRecursionEliminationInitVal == 0 && 355d64152a70842b2f4186aa912938e69ca09c1434cChris Lattner !getCommonReturnValue(Ret, CI)) 3567152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner return false; 3577152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner 3587152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // OK! We can transform this tail call. If this is the first one found, 3597152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // create the new entry block, allowing us to branch back to the old entry. 3607152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner if (OldEntry == 0) { 3617152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner OldEntry = &F->getEntryBlock(); 3627152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner std::string OldName = OldEntry->getName(); OldEntry->setName("tailrecurse"); 363c24a076c6a22a932e4fc2b745fd748fe7ee3cb15Chris Lattner BasicBlock *NewEntry = new BasicBlock(OldName, F, OldEntry); 3647152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner new BranchInst(OldEntry, NewEntry); 365fd93908ae8b9684fe71c239e3c6cfe13ff6a2663Misha Brukman 366ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner // If this tail call is marked 'tail' and if there are any allocas in the 367ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner // entry block, move them up to the new entry block. 368ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner TailCallsAreMarkedTail = CI->isTailCall(); 369ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner if (TailCallsAreMarkedTail) 370ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner // Move all fixed sized allocas from OldEntry to NewEntry. 371ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner for (BasicBlock::iterator OEBI = OldEntry->begin(), E = OldEntry->end(), 372ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner NEBI = NewEntry->begin(); OEBI != E; ) 373ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner if (AllocaInst *AI = dyn_cast<AllocaInst>(OEBI++)) 374ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner if (isa<ConstantInt>(AI->getArraySize())) 3754bc5f8071a28b6fc4f4c2207dd03a5f747d0d84bChris Lattner AI->moveBefore(NEBI); 376ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner 3777152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // Now that we have created a new block, which jumps to the entry 3787152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // block, insert a PHI node for each argument of the function. 3797152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // For now, we initialize each PHI to only have the real arguments 3807152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // which are passed in. 3817152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner Instruction *InsertPos = OldEntry->begin(); 3827f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner for (Function::arg_iterator I = F->arg_begin(), E = F->arg_end(); 3837f78f218fdb3515a61b995ac64f909bfd8ee84a7Chris Lattner I != E; ++I) { 3847152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner PHINode *PN = new PHINode(I->getType(), I->getName()+".tr", InsertPos); 3857152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner I->replaceAllUsesWith(PN); // Everyone use the PHI node now! 3867152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner PN->addIncoming(I, NewEntry); 3877152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner ArgumentPHIs.push_back(PN); 3887152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner } 3897152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner } 390fd93908ae8b9684fe71c239e3c6cfe13ff6a2663Misha Brukman 391ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner // If this function has self recursive calls in the tail position where some 392ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner // are marked tail and some are not, only transform one flavor or another. We 393ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner // have to choose whether we move allocas in the entry block to the new entry 394ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner // block or not, so we can't make a good choice for both. NOTE: We could do 395ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner // slightly better here in the case that the function has no entry block 396ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner // allocas. 397ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner if (TailCallsAreMarkedTail && !CI->isTailCall()) 398ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner return false; 399ce869ee05bfcb9d4750a3d0919a8260a727841c3Chris Lattner 4007152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // Ok, now that we know we have a pseudo-entry block WITH all of the 4017152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // required PHI nodes, add entries into the PHI node for the actual 4027152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // parameters passed into the tail-recursive call. 4037152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner for (unsigned i = 0, e = CI->getNumOperands()-1; i != e; ++i) 4047152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner ArgumentPHIs[i]->addIncoming(CI->getOperand(i+1), BB); 405fd93908ae8b9684fe71c239e3c6cfe13ff6a2663Misha Brukman 406543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // If we are introducing an accumulator variable to eliminate the recursion, 407543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // do so now. Note that we _know_ that no subsequent tail recursion 408543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // eliminations will happen on this function because of the way the 409543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // accumulator recursion predicate is set up. 410543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // 411543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner if (AccumulatorRecursionEliminationInitVal) { 412543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner Instruction *AccRecInstr = AccumulatorRecursionInstr; 413543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // Start by inserting a new PHI node for the accumulator. 414543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner PHINode *AccPN = new PHINode(AccRecInstr->getType(), "accumulator.tr", 415543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner OldEntry->begin()); 416543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner 417543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // Loop over all of the predecessors of the tail recursion block. For the 418543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // real entry into the function we seed the PHI with the initial value, 419543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // computed earlier. For any other existing branches to this block (due to 420543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // other tail recursions eliminated) the accumulator is not modified. 421543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // Because we haven't added the branch in the current block to OldEntry yet, 422543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // it will not show up as a predecessor. 423543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner for (pred_iterator PI = pred_begin(OldEntry), PE = pred_end(OldEntry); 424543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner PI != PE; ++PI) { 425543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner if (*PI == &F->getEntryBlock()) 426543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner AccPN->addIncoming(AccumulatorRecursionEliminationInitVal, *PI); 427543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner else 428543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner AccPN->addIncoming(AccPN, *PI); 429543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner } 430543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner 431543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // Add an incoming argument for the current block, which is computed by our 432543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // associative accumulator instruction. 433543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner AccPN->addIncoming(AccRecInstr, BB); 434543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner 435543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // Next, rewrite the accumulator recursion instruction so that it does not 436543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // use the result of the call anymore, instead, use the PHI node we just 437543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // inserted. 438543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner AccRecInstr->setOperand(AccRecInstr->getOperand(0) != CI, AccPN); 439543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner 440543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // Finally, rewrite any return instructions in the program to return the PHI 441543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // node instead of the "initval" that they do currently. This loop will 442543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner // actually rewrite the return value we are destroying, but that's ok. 443543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner for (Function::iterator BBI = F->begin(), E = F->end(); BBI != E; ++BBI) 444543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner if (ReturnInst *RI = dyn_cast<ReturnInst>(BBI->getTerminator())) 445543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner RI->setOperand(0, AccPN); 446543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner ++NumAccumAdded; 447543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner } 448543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner 4497152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // Now that all of the PHI nodes are in place, remove the call and 4507152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner // ret instructions, replacing them with an unconditional branch. 4517152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner new BranchInst(OldEntry, Ret); 4527152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner BB->getInstList().erase(Ret); // Remove return. 4537152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner BB->getInstList().erase(CI); // Remove call. 454543d622ef7505910c1cdc09ada0ab797c3437590Chris Lattner ++NumEliminated; 4557152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner return true; 4567152da38ef2c1f9f8ee407dc7740dfd43cbc41a6Chris Lattner} 457