Scribd
Upload
2K views136 pages

Dynamic Programming Coding Interviews

Dynamic programming coding interviews

Uploaded by

phulam146
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
2K views136 pages

Dynamic Programming Coding Interviews

Dynamic programming coding interviews

Uploaded by

phulam146
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
You are on page 1/ 136
DYNAMIC PROGRAMMING INTERVIEWS A BOTTOM-UP APPROACH TO PROBLEM SOLVING yAAY vv MEENAKSHI & KAMAL RAWAT FOUNDER, RITAMBHARA TECHNOLOGIES @ DYNAMIC PROGRAMMING CODING INTERVIEWS ABOTTOM-UPAPPROACH TOPROBLEMSOLVING MEENAKSHI & KAMAL RAWAT FOUNDER, RITAMBHARA TECHNOLOGIES © notionpress © Notion Press Old No. 38, New No. 6 McNichols Road, Chetpet Chennai - 600 031 First Published by Notion Press 2017 Copyright © Meenakshi & Kamal Rawat 2017 All Rights Reserved. ISBN 978-1-946556-70-7 This book has been published with all efforts taken to make the material error- free after the consent of the author. However, the author and the publisher do not assume and hereby disclaim any liability to any party for any loss, damage, or disruption caused by errors or omissions, whether such errors or omissions result from negligence, accident, or any other cause. No part of this book may be used, reproduced in any manner whatsoever without written permission from the author, except in the case of brief quotations embodied in critical articles and reviews. The contents of this book including but not limited to the views, representations, descriptions, statements, informations, opinions and references [“Contents”] are the sole expression and opinion of its Author(s) and do not reflect the opinion of the Publisher or its Editors. Neither Publisher nor its Editors endorse the contents of this book or guarantees the accuracy or completeness of the content published herein and shall not be liable whatsoever for any errors, misrepresentation, omissions, or claims for damages, including exemplary damages, arising out of use, inability to use, or with regard to the accuracy or sufficiency of the information contained in this book. DEDICATION This book is dedicated to a yosi MASTER MAHA SINGH who mended relations and excelled in each role he lived SON | BROTHER | FRIEND | HUSBAND | FATHER | FATHER-IN-LAW | TEACHER | FARMER | CITIZEN | DEVOTEE the most upright, conscientious, and able man. We miss you in every possible way Papa. Preface We have been teaching students ‘How to prepare for Coding Interviews’ for many years now. ‘A Data Structure problem like reverse a linked list or add operation getMinimum() to a Stack or implement a thread safe Queue are relatively easy to solve, because there are theoretical reference to start in such problems. The most difficult category of problems asked in coding competitions or interview of companies like Google, Amazon, Facebook, Microsoft etc. fall under the umbrella of Dynamic Programming @P). Most DP problems do not require understanding of complex data structure or programming design, they just need right strategy and methodical approach. A solution is first visualized in the mind before it takes formal shape of an Algorithm that finally translates into a computer program. DP problems are not easy to visualize and hence not easy to solve. ‘The best way to visualize a DP problem is using recursion because DP problems demonsttates optimal substructure behavior. Recursion gives the right solution but usually takes exponential time. This unreasonably high time is taken because it solves subproblems multiple times. DP is a bottom-up approach to problem solving where one subproblem is solved only once. In most cases this approach is counter-intuitive. It may require a change in the way we approach a problem. Even experienced coders struggle to solve DP problems. This book is written with an intention that if a teacher reads it, he will make dynamic programming very interesting to his students. If this book is in the hands of a developer, he will feel super confident in answering algorithm questions in interviews, and anyone who read it will get a robust tool to approach problems asked in coding competitions. Being a good coder is not about learning programming languages, it is about mastering the art of problem solving. Code in this book is written in C language, If you have never written any program in C language, then we suggest you to tead first few chapters of C from any good book and try writing some basic programs in C language. Acknowledgments To write a book, one need to be in a certain state of mind where one is secure inside himself so that he can work with single minded focus. A guru helps you be in that state, We want to thank our ideological guru, Shri Rajiv Dixit Ji and Swamy Ramdev Ji. The time spent on this book was stolen from the time of family and friends. Wish to thank them, for they never logged an FIR of their stolen time. We also wish to thank each other, but that would be tantamount to cone half of the body thanking the other half for supporting it. The body does not function that way. How to Read this Book We have discussed only simple problems in first six chapters. The idea is to explain concepts and not let the reader lost in complexity of questions. More questions are discussed in last three chapters with last chapter completely dedicated to practice questions. Tf you have the luxury of time, then we strongly recommend you to read this book from cover to cover. If you do not have time, then, how you read this book depends on how good you are in coding and how comfortable you are with recursion. If you are good at forming logic and can write reasonably complex programs of Binary Tree and Linked List comfortably, then you may choose to skip the first chapter. I think the below logic may help you find a starting point IF (one day left for Interview) IF (champion of DP) READ Chapter 8,9 ELSE IF (Good @ Programming) READ Chapter 3, 4, 5, 6, 7. ELSE Pray to God, Start from Chapter 1. ELSE IF (more than one week left for interview) IF (champion of DP) READ Chapter 2, 7, 8, 9 ELSE IF (Student OR Researcher) READ complete book ELSE IF (Reading to learn DP) READ complete Book Recursion int main(){ printf("Cannot stop till I die"); main(); ) Most computer concepts have their origin in Mathematics, Recursion is no exception. If you have studied mathematics at high school level, then you must have come across equations like. Creo Plt ts 9 ree OEE SG eo Pret ol & is a symbol for summation. It is read as “Same of first nm numbers is equal to n plus Sum of first (n-1) numbers. When m becomes 1, Sum(1) has a fied value, 1” n, in this case is positive integer. In plain words, Sum is defined in terms of Sum itself. This is called Recursion, In computer programming, “when a function calls itself either directly or indirectly it is called a Recursive Function and the process is called Recursion.” Typically the function performs some part of the task and rest is delegated to the recursive call of the same function, hence, there are multiple instances of same function each performing some part of the overall task. Function stops calling itself when a terminating condition is reached. Recursion is a problem solving technique, where solution of a larger problem is defined in terms of smaller instances of itself. Points to note in recursion are the following: 1. Recursion always have a terminating condition (lsc it will be infinite recursion). In Sum function, the condition, if n=1, then, stop recutsion and return 1, is the terminating condition. 2, Recursive function performs some part of task and delegate rest of it to the recursive call. In above example, the function performs addition of n with the return value of the Sum(n-1) but delegate the computation of Sum(n-1) to recursive call. Writing recursive code is not as difficult as it may appear, in fact, in most cases it is relatively simpler because we ate not solving the complete problem. Recursive function code is a two-step process 1. Visualize the recursion by defining larger solution in terms of smaller solutions of the exact same type (with natrow parameters), and, 2. Add a terminating condition. Example 1.1 translate the logic of calculating sum of first n positive integers to C language program. Example 1.1: C language function to compute sum of first n positive numbers: int sum(unsigned int n){ Lf (n = 4) return 1; else return n + sum(n-1); Code: 1.1 ‘This code has an issue, If someone call it for n=0, then it will behave in an undefined manner. As a good coding practice, we must check our program against boundary values of input parameters. If we call the function as sum(0); then, it will skip our terminating condition (n == 1) and enter into the recursion calling itself forn = -1, and this will make the result undefined’. We should be able to catch this issue while self-reviewing our code and should be able to correct it: int sum(unsigned int n) { // First terminating condition if(n == 0) return 0; // Second terminating condition if(n == 1) return 1; return n + sum(n-1); Code: 1.2 We have just skipped the else part, because we ate returning from the terminating conditions. Code 1.2 may be written in a more compact form as shown below int sum(int n){ return (n==0)? 0: ((n==1)? 1: (n+sum(n-1))); Code: 1.3 Which of the two codes should we prefer to write? "In C language, conversion from signed to unsigned is not defined for negative numbers. For example, the value of y in the below code is not defined: int x = -1; unsigned int y = x; When Sum is called for n=0. Then, it will skip the terminating condition and call the function recursively forn = -1. Some coders, have an undue bias in favour of writing compact codes, this is especially true during the interview. Either they think that such code impress the interviewer, or they may just be in the habit of writing such code. Thumb rule for good code is, “when there is a choice between simple and obfuscated code, go for the simpler one, unless the other has performance or memory advantage.” This rule is not just for interview, its genetic. The code that we write is read by many people in the team, the simpler it is the better it is. One extra advantage of writing simple code duting the intetview is that, it provide us some space to correct mistakes there itself, because it leaves more white space on the paper. The rule is just for spacing of code and choosing between two statements doing exactly same work. In no way a check should be omitted in favor of simplicity or clarity or extra space on the paper. Never miss the terminating condition, else the function may fall into infinite recursion. It is not mandatory to write a recursive function to compute sum of n numbers. It can be done using a loop without making any recursive call, as demonstrated in Code 1.4 Non-recursive code to compute sum of fitst n_ numbers int sum(int n){ int sum = 0; for(int i=l; i<=n; i++) sum += i; return sum; Code: 1.4 Question 1.1; Factorial function is defined recursively for all non-negative integers as: Fact (n) = n * Fact (n-1) if n>1 =1 if n=1 Write both recursive and non-recursive function that accepts an unsigned int n and return factorial of n?. ? Example7.1 in Chapter 7 gives the solution to this question. Question 1.2: Given an array, arr, of integers, write a recursive function that add sum of all the previous numbers to each index of the array. For example, if input array is Then, your function should update the artay to {a ]s3 | 6 | a0] as | 2a Example 1.2: Recursion to compute n> power of a number x" is defined as below: Function for above recursion is: int power(int x, int n){ if(0 == n) return 1; else if (1 return x; else return x * power(x, n-1); Code: 1.5 Recursive function in Code 1.5 accept two parameters. One of them remains fixed, and other changes and terminates the recursion, Terminating condition for this recursion is defined as IF (n EQUALS 0) THEN return 1 But we have used two terminating conditions, IF (n EQUALS 0) THEN return 1 IF (x EQUALS 1) THEN return x ‘This is to avoid unnecessary function calls when x is 1. In next chapter we will learn that every function call is an overhead, both in terms of time and memory. The four things that we should focus on while writing a function (in this ordet) are: 1. It should serve the purpose. For every possible parameter the function must always return expected results. It should not be ambiguous for any input. ‘The time taken by function to execute should be minimized. ‘The extra memory this function consumes should be minimized. 4. Function should be easy to understand. Ideally the code should be self-explanatory to an extent that it does not even require any documentation (comments). At no point during coding or while in the interview do we really care about how many lines of codes does patticular function rans into as long as length of code is justified (we are not writing duplicate piece of code). In next chapter, we will learn that a recursive solution takes mote time and more memory than the corresponding iterative (non-recursive) solution, In Example 1.1, both iterative and recursive solutions are equally easy to code, In such situations we should always go for non-recursive solution. Ye INTERVIEW TIP If both recursive and non-recursive (iterative) solutions are equally easy and take almost equal time to code, then always write the iterative solution. It takes less time and less memory to execute, ‘The advantage of recursion is that, sometimes, a solution that otherwise is very complex to comprehend, can be very easily visualized recursively. We just need to solve the problem for base case and leave rest of the problem to be solved by recursion. Consider Example 1.3 below: Example 1.3: Tower of Hanoi Tower of Hanoi is a Mathematical Game. There are 3 pegs (Rods), Source, Destination and Extra marked as S, D and = respectively, and there ate n discs, each of different size, which can be inserted into any of these three pegs. 3 Recursion is just replacing a loop in the iterative solution. All discs are initially inserted into Source peg in decreasing order (smallest at the top) as shown in Picture 1.1 (for n=4). L a Ss D E Picture: 1.1 We have to move all the discs from Source peg (S) to Destination peg (D). The final state should be as shown in Picture 1.2 ‘There are 2 restrictions: 1. Only one disc can be moved at a time, 2. Atany point in the process we should never place a larger disc on top of a smaller disc. Write a function that accept chatacters representing three rods (S, D & E) and the number of discs (n), and print the movement of discs between pegs such that all discs are moved from the initial state (inside 8) to the final state (inside D). Signature of the function is /* s, ad, e represents three pegs * (source, destination and extra). * n is number of discs (All initially in s)*/ void towerOfHanoi(char s, char d, char e, int n) This may appear to be a complex problem to solve otherwise, but if we think recursively, then the problem can be solved in three simple steps Step-1: Move n-1 discs from S to E using D Let us assume that somehow n-1 discs are moved from S$ to E, and we have used D as the third peg (extra). This problem is similar to the original problem (of moving n discs from 5 to D using E). After this step, the state of pegs and discs is as shown in the picture 1.3 s D E Picture: 1.3 Step-2: Move the n’th disc from S to D Move Disc No. n from S to D. This is a single step of execution. Step-3: Move n-1 discs from E to D using S This problem is again of the same nature as step-1. Here we are moving n-1 discs from E to D using peg-S as extra. If we carefully notice, Step-1 and Step-3, being problems of the same type, are recursive calls to the same function. Code 1.6 is the recursive function for Tower of Hanoi: void towerOfHanoi(char s, char d, char e, int n){ // TERMINATING CONDITION if(n <= 0) return; towerOfHanoi(s, e, d, n-1); printf ("Move Disk-%d FROM %d TO 8d”, n, s, 4); towerOflanoi(e, d, s, n-1); Code: 1.6 ‘The terminating condition here is when there is no disk (n==0) . Notice that we have put the condition as less-than ot equal to, to handle cases when n is negative, alternatively, we can change signature of function to receive unsigned int in place of int. If we call this function for 3 discs (n=3) like below towerOfHanoi('s', 'd', 'e', 3)7 The output is Move Disk-1 FROM s TO d Move Disk-2 FROM s TO e Move Disk-1 FROM d TO e Move Disk-3 FROM s TO d Move Disk-1 FROM e TO s Move Disk-2 FROM e TO d Move Disk-1 FROM s TO d Now we can appreciate, how helpful recursion can be even if it takes more time and more memory to execute. Head Recursion and Tail Recursion ‘A Recursive function typically perform some task and call itself. If the call is made before the function performs its own task, then it is called Head- Recursion (Recursion is performed at the head of the function body). If recursive call is made at the end, then itis ‘Tail-Recursion. In Code 1.1, function sum (3), call function sum(2) first and then perform the add operation’ (return value from sum (2) added with 3). This makes function sum, a head-recursive function. To see the difference, consider Code 1.7 having two recursive functions to traverse a link list: A recursive function is head-recursive if recursive call is made before it performs its own task. tail-recursion is when recursive call is made at end of the function (after it performs its own tasks). /* Head Recursion, * First traverse rest of the list, then * print value at current Node. */ 4 Tip: In C language the order of e valuation of operands for plus operator (+) is not defined. It means that in the below statement: x = funl() + fun2()7 x will be sum of return values of two functions but, whether fun1 is called first or funz is not defined in the language. This has to be defined by the compiler. void traversel (Node* head) { if (head != NULL) { traversel (head->next) ; printf (“td”, head->data); } /* Tail Recursion. * First traverse rest of the list, then * print value at current Node. */ void traverse2 (Node* head) { if(head != NULL) { print£("%d", head->data) ; traverse2 (head->next) ; Code: 1.7 Tf below linked list is passed as input to the two functions in Code 1.7: Te? [+> 14> NN Then, traverse function prints the list in backward order and traverse2 prints it in forward order. Output of traversel: 4 3 21 Output of traverse2: 1 2 3 4 A tail recursion is very easy to re-write in form of a loop. So there should ideally be no teason to write a tail recursion in code unless we are writing it to demonstrate recursion itself. Head and tail recursion ate just given here as concepts. Most of the times (esp. in case of dynamic programming problems) recursion is more complex and may not be simple head ot tail recursion. Consider one of the most rampant examples of recursion, in-order traversals of a Binary Tree. In in-order traversal, we first traverse left subtree in in-order, then traverse (eg, print) the root and finally traverse the right subtree in in-order as shown in Picture 1.4. Clearly the in-order function is defined in terms of in-order of left and right subtrees, hence recursion. Algorithm: (A) 1. Traverse Root 2. Traverse Left Subiree in PreOrder © © 3. Traverse Right SubTree in PreOrder op ECra Q® ®® Picture: 1.4 If we take structure of the Node as below: Struct Node{ Node *left; // Pointer to Left subtree int data; Node *right; // Pointer to Right subtree Le Then code of in-order traversal is as shown in Code 1.8 below: /* Print In-Order traversal of the tree */ void inOrder(node* r) { if(x == NULL) return; inOrder (r->left) + printf£(“d ”, r->data); inOrder (r->right) ; Code: 1.8 In the above code, recursion cannot be termed as either head or tail recursion, ‘The terminating condition that we have taken in Code 1.8 is when root is NULL, It will have extra function calls for leaf nodes, because function is called for left and right subtree even when both are NULL. A better solution is to check that a subtree is not NULL before calling the function for that subtree. /* Print In-Order traversal of the tree */ void inOrder(node* r) { if(x == NULL) return; if(r->left != NULL) inOrder (r->left); printf("td “, r->data); if(r->right != NULL) inOrder (r->right); Code: 1.9 Tt may look like small improvement, but it will reduce our number of function calls to almost half, because in a binary tree the number of null pointers is always greater than the number of valid pointers. In Code 1.8, we are making one function call for each pointer (null or non-null). But in code 1.9, the function calls are against non-null pointers only. Hence the total number of function calls in Code 1.9 is almost half as compared to function calls made in code 1.8. Consider the below binary tree Thete are 8 null pointers, 6 as children of leaf () nodes and tight child of £ and left child of c. @ © If we consider root also as one pointer (pointing to the root node, A), then total number of ©) ©) © swanall pointers are 7 (one pointing to each ade in the trec). © Putting such small checks not just optimize our code but also shows our commitment toward writing better code. Next chapter discuss how does memory actually looks when a recursive function is called in contrast to an iterative function. How to Solve a Problem Using Recursion Our focus usually is to solve a problem, with recursion, we can actually code without solving a problem, if we can somehow define the large problem in terms of smaller problems of same type. Our focus is on solving the problem fot top case and leave the rest for recursive calls to do. Consider the below example: Example 1.4: We all know about Bubble Sort, where an array is sorted in n passes as shown in the below code: void bubbleSort (int *arr, int n) { for(int ¢ i arr[j+1]) swap(6arr[j], &arr[j+1])¢ # j arr[j+1]) swap(Sarr[j], sSarr[j+1]); bubbleSortRec (arr, n-1); Code: 1.12 We cannot be good coders without mastering the art of recursion. ‘My suggestion to everyone reading this book is that when you write simple programs like linear search, binary search, sorting, etc. try to also implement them recursively. It will be a good net practice before the actual match. Question 1.1: Below code print the mathematical table of n. void printTable(int n){ for(int i=l; i<=10; i++) { printf (“$d * %d = %d\n”, n, i, (n*i)); Code: 1.13 Write a recursive function that prints the mathematical table of n. Hint: You may have to pass/ accept i as parameter. How it Looks in Memory ‘Application’ a | oo T Before discussing how a recursive calls looks inside memory, we should understand how memory is divided internally and what part of the program goes in which section of the memory. This chapter is specific to C language, but the concepts are similar in other popular languages also. ant g = 5; — Goes in Data area. aint main() 4 > 20 (default value) static int a; . int b; Josie, ConA of main) ant *p; cefautt — guvtage (int*) malloc (sizeof(int)); 10; } act AR of Picture 2.1 shows lifecycle of a C language program: Source Files User-defined__ Header Files ° Standard a header files £ Picture: 2.1 Read any good C language book to learn about compilation and linking of a C language program, this book assume that you have working knowledge of C language. After compiling and linking, * the binary executable of program gets generated (exe on windows). When this executable binary is actually executing (running) it is called a process. When a process is executed, first it is loaded into memory (RAM). Area of memory where process is loaded is called process address space. Picture 2.2 shows broad layout of process address space (Pidure 2.2 is independent of platform, actual layout may differ for operating system and for program). » Executable Code Global / Static Data HEAP —}-+ Dynamic memory (Allocated using | malloc, ealloc, new, et.) STACK —}-+ Activation Records of functions Command line arg. & environment variables Picture: 2.2 This memoty is allocated to our program by the operating system. The process address space has following segments 1. Code segment (or Text segment) 2. Data segment 3. Stack segment 4. Heap segment In next section, we are discussing these segments one by one. Code segment v This segment contains machine code (in form of executable instructions) of the compiled program. Y — Itis read-only and cannot be changed when the program is executing, VY May be shareable so that only a single copy is in memory for different executing programs®. Y Size of code segment is fixed at load time. 5 Most IDEs compile, link and execute the program using a single button click. But internally all these steps are performed. § Shareable code is outside the scope of this book. Data Segment v v v All global and static data variables are allocated memory in this segment. Memory is allocated in this area when the progtam is loading (before main function is called). That’s why global and static variables are also called load-time variables. All load-time variables (global and static), are initialized at the load- time. If no initial value is given for a load-time variable then it is initialized with the zero of its type’. Internally, this segment is divided in two areas, initialized and uninitialized. If initial value of a variable is given it goes in the initialized data area else it goes in the uninitialized data area, All the uninitialized variables are then initialized with zeros. The main reason why they are stored separately within data segment, is, bacause the uninitialized data area can be memset to zero in a single operation. Size of data segment is fixed at load time and does not change when program is executing. Stack Segment v Stack segment contains Activation Records (also called Stack Frames) of all the active functions. An active function is a function that is curtently under the call. Consider Code 2.1 below int main(){ funl(); } void funl(){ fun2(); } void fun2(){ } void fun3(){ // NEVER CALLED Code: 2.1 7 Zero of int data type is 0. Zero of pointer data type is NULL. When main is called, it is the only active function. Then main calls fun1. At this point fun1 is executing but both main and fun1 are active. When funl calls fun2, then the execution is in fun2, but main, fun1 and fun2 are all active and has their activation records in Stack. When function fun2 returns, then activation record of fun2 is poped from the Stack and execution is back in fun1. At this point main and fun1 are active and has their activation records in the Stack. fun3, is never active, because it is never called and hence its activation record never gets created on the Stack. ¥ When a function is called, its Activation Record is created and pushed on the top of stack. Y When a function returns then the corresponding Activation Record is poped from the Stack. Y Size of Stack keeps changing while the program is executing because the number of active functions keep changing. ¥ Non-static local variables of a function are allocated memory inside Activation Record of that function when it is active. v Variables that are allocated memory on Stack are not initialized by default. If initial value of a variable is not given then it is not initialized and its value will be garbage (this is different from load-time variables allocated memory in Data Segment). Y Activation record also contains other information required in function execution. Y Stack Pointer (SP register) keeps track of the top of the Stack. Y Point of execution is always inside the function whose activation record is on the top of Stack. Function whose activation record is inside Stack, but not at the top is active but not executing. ¥ If a function is recursive then multiple activation records of the function may be present on the Stack (one activation record for each instance of the function call), Heap Segment v When we allocate memory at run time using malloc (), calloc(), and realloc() in C language (new and new{] in C++), then that memory is allocated on the Heap. It is called dynamic memory or run-time memory, Y InC language we cannot initialize the memory allocated on Heap. In C++, if we use new operator to allocate memory, then we can initialize it using constructors. Y Memory allocated in heap does not have a name (unlike memory allocated in and Stack segments). The only way to access this memory is via pointers pointing to it. If we lose address of this memory, there is no way to access it and such a memory will become memory leak. It is one of the largest sources of ertor in C/C++ programming. Y Both Heap and Stack segment shares 2 common area and grows toward each other. After compilation and linking, the executable code (in machine language) gets generated. The first thing that happens when this executable code is executed is that it is loaded in the memory. Loading has following steps: Y Code goes in code area. Code is in the form of binary machine language instructions and Instruction Pointer (IP register) holds the address of current instruction being executed. ¥ Global and static variables are allocated memory in the data area. Data area has two sub-parts, Initialized and Un Initialized data area, if initial value of a variable is given by us, it gets allocated in the initialized data area, else memory to the vatiable is allocated in the un-initialized data area and it is initialized with zero. ¥ Global and static variables are initialized. If we have given the initial value explicitly, then variables are initialized with that value otherwise they are initialized with zeros of their data types. int x = 5; // initialized with 5 int y; // initialized with 0 After these steps, we say that *he program is loaded. After loading is complete, the main function is called and actual execution of the program begins. Read the entire program given in Code 2.2 carefully: // Go in data area at load time. Initialized with 0. int total; /** Code (machine instructions) of function goes in * code area. When this function is called, then ® Question: Who calls the main function ? * Activation Record of the function is created on * Stack. +f int square(int x){ // * goes in AR’ when this function is called. return x*x; /** Code of function goes in the code area. When this * function is called (at run-time), its AR gets * created on Stack and memory to non-static local * variables (x and y) is allocated in that AR. * count, being a static variable, is allocated in * data area at load time. int squareOfSum(int x, int y) { static int count = 0; // Load-time var printf ("Fun called %d times", ++count) ; return square (xty); /** Code goes in code area. When main is called, its * activation record gets created on Stack and memory * to non-static local variables (a and b) is * allocated in that Activation Record. any int main() { int a=4, b=2; total = squareOfSum(a, b); printf ("Square of Sum = %d",total); Code: 2,2 This program computes (a+b)? and print the result. To keep it simple, we are using the hard coded values 4 and 2 for a and b respectively. The function squareOfSum also keeps a count of how many times it is called in a static variable count, and print the count every time it is called. Code 2.2 may not be the best implementation, but it serves our purpose. Read the code again, especially the comments before each ° AR = Activation Record function and make sure that we understand everything. After compilation and linking, the executable of the program is created and when this executable is run, the first thing that happens is that this executable is loaded in the memory (RAM). At this point the main function is not yet called and the memory looks like Picture 2.3: CODE 0 |totjal Q__|count STACK@) @@dV4H DATA Picture: 2.3 After loading is complete, main function is called. When a function is called, its Activation Record is cteated and pushed in the Stack. The AR has Y Local (non-static) variables of a function (a andb for main). v¥ Other things stored in the Activation Record. In the diagrams, we are only showing non-static local variables in Activation Records, Aftet main function is called, the memory looks as shown in Picture 2.4 CODE O_ |total O__|count STACK™ @@dVaH DATA 4 AR of 2 |b main() At any time, the point of execution (Instruction Pointer) is in the function whose AR is at the top of Stack. Let us understand what all happens internally duting a function call. When a function is called: 1. State (register values, Instruction Pointer value, etc.) of calling function is saved!” in the memory. 2. Activation record of called function is created and pushed on the top of Stack. Local variables of called function are allocated memory inside the AR. 3, Instruction pointer (IP register) moves to the first executable instruction of called function. 4, Execution of the called function begins. Similarly when the called function returns back (to the calling function), following work is done: 1, Return value of the function is stored in some register. 2. AR of called function is popped from the memory (Stack size is reduced and freed memory gets added to the free pool, which can be used by either the stack or heap). 3, — State of the calling function is restored back to what it was before the fanction call (Point-1 in fanction call process above). 4, Instruction pointer moves back to the instruction where it was before calling the function and execution of calling function begins from the point at which it was paused'!, 5, Value returned from called function is replaced at the point of call in calling function. 1° Value of local variables of a function under execution are stored in the AR of function which is preserved in the stack. But Registers will also have some values, these values also need to be saved (because Registers are needed by the called function). This state is saved in the memory. " ‘This is conceptually similar to Context Switch of process contexts in a muti-processing operating system when one process is preempted to execute another process and after some time control returns back to the first process and it starts executing from the same point where it was preempted. Clearly, a function call is a lot of overhead both in terms of time and memory. One of the reasons behind the popularity of macros in C language (even after all the evil that they bring along) is this overhead in function call. Another was the type independence that macros bring!2, Some compilers optimize the performance by replacing function call with entire code of the function during compilation, hence avoiding the actual function call overheads. This is called inline expansion. For example, in Code 2.2, the compiler may just put entire code of function square inside squareOfSum and remove the function call all together as shown below. int squareOfSum(int x, int y){ static int count = 0; // Load-time var printf (“Fun called %d times”, ++count) ; return (x+y) * (x+y); Code: 2.3 Recursive functions are very difficult to expand inline because compiler may not know the depth of function call at compile time. Example 2.1: Let us also sce how memory looks like if we miss the terminating condition in recursion. Code 2.4 is example of infinite recursion. int main() { int x = 0; xtt; Lf (x<5) { printf (“Hello”); main(); Code: 2.4 When the program is executed after compilation, it is first loaded in the memory and then the main function is called. At this point (after 2? In C++, both the benefits are given in the form of inline functions and templates and they are not error prone like macros. calling main) the memory looks as shown in Picture 2.5. Code atea has the code, The Data area is empty because there is no load-time (global or static) variable, Stack has only one activation record of function main. ‘CODE, DATA STACK [0 _] Je AR of main() Picture: 2.5 Initial value of x is 0, after increment x become 1, since x<5, the condition is true and main is called again. A new AR for this newly called main is created on the Stack and this AR also has local variable x that is different from variable x in AR of previous call (see Picture 2.6). Value of this new x is again 0, and main is called again. Every time main is called, the value of x in the new activation record is 0. CODE DATA el o JAR of main() STACK x 1 JAR of main () Picture: 2.6 Every instance of main is actually using a different x (from their own instance of AR). Code 2.4 will continue to print “He11o”, until a point when no space is left in the Stack to create new AR. At this point main cannot be called farther and the program will crash. An important thing to note is that the program will not print “Hello” infinitely. It is printed, till the memory stack overflows. Recursive v/s Non-Recursive Inside Memory Let us consider Example 1.1 from chapter-1 again. int sum(int n){ if (n==1) return 1; else return n + sum(n-1); Code: 2.5 When we call this function for n=3, as sum(3); It will call sum(2) ; which will in-turn call sum (1) 7 At this point (when execution control is in sum(1)), the memory stack will have three instances of activation records of function sum, each having a local variable n, as shown in Picture 2.7. a{_1_| JAR of sum(1y al 2] JeAR of sumc2) n [3] [AR of sum(3) Picture: 2.7 In the iterative version (Code 1.4) there is only one function call to sum(3) and three local variables n, i and sum on the Activation Record (AR) of the function as shown in Picture 2.8. [a of sum (3) Picture: 2.8 In recursive version, one activation record is created for each value of n. If n=1000 then 1000 ARs are created. Therefore the extra memory taken is O(n). Table 2.1 gives a comparison of asymptotic running time and extra memory taken for recursive and non-tecutsive sum functions. The asymptotic time may be same, 0 (n) for both the cases, but actual time taken for recursive version is much more than the iterative version because of the constant multiplier. Recursive Non-Recursive Time O(n) O(n) Memory O(n) (1) Table: 24 Example 2.2: Let us consider one more example. Code 2.6 is the recursive function to computes factorial of n: int factorial(int n){ if(1==n || O==n) return 1; else return n * factorial (n-1); Code: 2.6 If function is called for n=4, fact (4); from some other function to compute factorial of 4. During successive function calls, the memoty looks like Picture 2.9. =e oe Se ree ¥ sreactian sreacteay [im 3)?" facta) eteenm nen re ee emma wesc are a Picture: 2.9. When the functions return value to their caller functions, the AR will be poped from the stack and the stack will look like Picture 2.10 (return values shown on right side), Picture: 2.10 There will also be other function’s AR in the Stack (eg, main function). They are not shown to save the space. Code 2.7 shows the non-recursive code to compute factorial of n. int factorial (int n){ int £ = 1; for(int i=2; i<=n; i++) f-£* i; return f; Code: 2.7 ‘The memory image of function in Code 2.7 is shown in Picture 2.11. Compare it with the memory taken by the recursive code. mw Ql a Flnl_4 Picture: 2.11 Code 2.7 may have mote local variables, but there is just one AR in the memory irrespective of the value of n. Recursion is a huge overhead. Both in terms of memory and execution time. The examples of recursion seen till now ate simple linear recursions. One of the major problem with recursive function comes when recursive calls starts overlapping at the level of subproblems. Overlapping subproblems is discussed in detail in chapter 4. Memory Layout as a Problem-Solving Tool A clear understanding of lifecycle of program execution and how a program is loaded in memory comes handy in solving many mote questions other than recursion. Consider the below examples: Example 2.3: What is value of x in the below code? static int x = strlen("Hello"); The above code is compile-time error. To put it simply, “static and global variables cannot be initialized with the return value of a function.” We are trying to initialize a static variable with the return value of function strlen. We know that static variables ate initialized at load time. But wait, functions cannot be called at load time. A function can only be called when loading is complete and the program is executing (first function that gets called is main). How can we initialize, at load-time, with something that is not available until execution time. Hence, Error! What if we break up the statement in two parts? static int x; // initialized with zero x = strlen("Hello"); Now thete is no problem. At load time variable x is initialized with zero. During execution the function strlen ("Hello") is called and x is assigned the value 5. Example 2.4: What value will get printed if we call function fun? void fun() { int a = 5; static int b = a; printf ("value: 34", b); Code: 2.8 No, the answer is not 5 ot 0. The above code is also a compile time ERROR. We know, static variables are initialized at load time. In code 2.8 we are initializing b with a, but variable a, is not available while loading. It will be allocated memory in the activation record when function £un is called and fun is called at execution-time, It is called only after the loading is complete and when the code starts executing. Also, if there are more than one instances of any function in the Stack (in case of recursive functions). Then each AR have a separate copy of local variable a, but there is only one copy of static variable (allocated in the data area). By that logic also static variable (single copy) cannot be initialized with a local variable (possible zero or multiple copies). Load-time variables cannot be initialized with local variables. Conclusion 1. A function will have multiple ARs inside stack if and only if it is recursive. 2. Global and static variables can only be initialized with constants. 3. The memory to load-time variables is allocated before any function is called. 4. "The memory to load-time variables is released only after the execution is complete. We have not discussed the Heap area because the purpose was to explain recursion and not pointers or dynamic memory allocation or deallocation. To learn how heap area is used, read some good book on pointers in C language. Optimal Substructure Am I just recursion? Optimal substructure means, that optimal solution to a problem of size n (having n elements) is based on an optimal solution to the same problem of smaller size (less than n elements). ic while building the solution for a problem of size n, define it in terms of similar problems of smaller size, say, k (k 2 } Fibonacci sequence appears in Indian mathematics, in connection with Sanskrit prosody dated back to 450 B.C. Like most other things, the serics went from east to west and was named after the guy who introduced it to west, rather inal founders, that guy happens to be Fibonacci. The simplest algorithm to compute n* term of Fibonacci is direct translation of mathematical definition: int fib(int n){ if (m==1 || n==2) return 1; else return fib(n-1) + fib(n-2); Code: 4.1 Code 4.1 is a recursive code. We may want to put an extra check and throw an exception if n is negative or zero, so that our code does not run into infinite recursion if called with zero or negative value as argument. We skipped this check to keep the code simple. It may not be obvious, but Example 4.1 also has optimal substructure property. To find optimal solution (the only solution in this case) for n™ term we need to find the optimal solution for n-1* tetm and n-2% term. ‘The equation of time taken by the function in Code 4.1 is T(m) = T(n-1) + T(n-2) + O(1) ‘This is an equation for exponential time. The reason why it is taking exponential time for such a simple algorithm is because it is solving the subproblems (computing Kk term, k

You might also like