Chapter STOCHASTIC PROCESSES 3,0 INTRODUCTION stochastic process, in probability theory, is a process involving the operation of chance. More generally, a stochastic process refers to a family of random variables indexed against some other variable or set of variables and is one of the most general objects of study in probability. Some basic types of stochastic processes include Markov processes, Poisson processes (such as radioactive decay), and time series, with the index variable referring to time. 3.1 FUNDAMENTALS In very general terms, probability is a measure of the likelihood ° that a particular event will occur in any one trial, or experiment, carried out in prescribed conditions. Each separate possible result from a trial is called an outcome. © Notation: the probability that a certain event 'A' will occur is denoted by P(A). For example, if A represents the event that a component, picked at random, is faulty, written A = {faulty component}, then P(A) denotes the probability of picking a faulty component We measure the probability for Random Events How likely an event would occur The set of all possible events is called Sample Space 35ing and Simulation tion to Modelling ai General Introduc' 36 In each experimen t, an event may occur with a co, . probability (Probability Measure) ‘ e Example: « Tossing a die with 6 faces Nea « The sample space is (1, 2, 3,4, 5, ee ¢ Getting the Event " 2" in on experim as a Probab, e Senne in manufacturing, it would be both time-cong ty and uneconomical to subject every single component Prod to full inspection, It is usual, therefore, to examine a sq th batch of components, taken at random, as being epresentay of the whole output. The larger the random sample, the _ nearly representative of the whole population (tota] Output components) is it likely to be. The determination of probability may be undertaken from two approaches: i. Empirical (or experimental) probability and ii. Classical (or theoretical) probability 3.2 EMPIRICAL PROBABILITY e Empirical Probability is based on Previous known results, The relative frequency of the number of times the event has Previously occurred is taken as indication of likely occurrences in the future, ° Take an example. A Tandom batch of 249 components is subjected to strict inspection and 20 items are found to be defective. Therefore if we pick any one component at random from this sample, the chance of it being faulty is"20 in 240" ie "Lin 12" ° SoifA= {faulty component} © Then P(A) = 1 in 12 = V/12 = 0.0833 = 8.33% ® The most usual forms are P(A) = 1/12 or P(A) = 0.0833 ° Therefore, a run of 600 Components from the same machine would be likely to contain ..., defectives,r ECTATION 43, EXP The result does not assert that the . * smany run of 600 compon ere will be : ents, but havi of the event occurring in any are F One es redict the likely number of e trial (1, i : time similar trials. This expectation E les ne vent will occur in N : 18 defined by the product of the number of trials N ang the ili F . fa will occur in any one trial Probability P(A) that the event ¢ E=NxP(A) exactly 50 defectives found the probability /12), we can use it to l, Le 3.4 SUCCESS OR FAILURE e Throughout, we are concermed with the probability of the occurrence of a particular event. When it does occur in any one trial, Wwe record a success: when it fails to occur, we record a failure - whatever the defined event may be. ; e If, in N trials, there are x Successes, there will also be (N - x) failures, so that: eo xt+(N-xX)=N e Therefore, x + W-x N N e But, x/N = P(success) = P(A) e And (N -x) = P(failure) = P(mot A) “NS e Therefore P(A) + P(not A) =1 Sih The event 'not A’ is called the complement of event A and is often written A, i.e © P(A) + P(A) = «(2) 3.5 PROBABILITY * Therefore, if the probability of picking a defective tool from a box that contains 5 of such tools is 1/5, the probability of picking non-defective tool will be ..i ay that: / Hence, suffice it to say 0 of possible events is betwee, e The probability of every se and 1, inclusive. en © The probability of the whole set © E SIZE ' a 3.6 eee original sample from which the probability fig, 7 , oye, 7 ; he catablished affects the reliability of the 1 ene Probabilities derived from small samples har‘ ily a thy ° probabilities associated with the whole population. Th large, the sample, the more reliable are the results 7 For example, If 15% of a trip of 120 bags of cement are found to be counterfeit, what is the number of acceptable number of acceptable bags likely to be present in the trip? SOLUTION > E=Nx P(A) 3.7 MULTIPLE SAMPLES ¢ Asimple random samples of n components taken from a whole Populati i components from the same Population could well inc] different number of defectives from that in the first SB ‘ ° We do not have a production line on hand, but we Can simulate the same kind of problem with the aid of a Pack of playin: cards (jokers removed). ying ° Then, the total population N = the number of cards in the Whole pack, i.e. 52.e If we shuffle well and then deal out a random sample of 12 cards (n), we can count how many cards in the sample are (say) spades (x). These take the place of the defective components. e Replace the 12-card sample in the pack; shuffle well and take asecond sample of 12 cards. Counting the spades in this second sample will most likely give a different total from the result of the first trial. © We will extend this phenomenon into a useful experiment. * To determine the number of spades in each of 40 trials of a random 12-card sample from a full deck of playing cards and to compile a cumulative proportion (running average) of the resultsGeneral Intro — s oCEDURE 3.8 ExPeRIME mek of playing cards, excluding the joker, cart with a se the cards thoroughly : Pick arandom sample of 12 cards i the nt and record the number of carg. t |, From are spades. are spa he pack and shuffle thorough " i urn the sample cards to t for a total of 40 such trials sample, cou iv, Rel Repeat the process vi. Compile a table showing: the number of trial, r (1 to 40) the number of spade cards in each trial, x the cumulative total of spade cards, cum x |. the cumulative proportion (running average) of the Tesuf ier total of spade cards in r trials/no of trials to day vii. Display graphically the distributio: ct tc mn oer ee a. b. ao The table results will look like this but with different entries: Plot the following graphs: © Graph show 7 ing the distributi, each trial istribution of the ny imber of spades at il, The runnj ing ave; for the trial ' average for IsStochastic Processes 41 yain POINTS SO FAR... a, The empirical probability of an number, % of successes experig event A occurring is the divided by n need in n previous trials, je.P(A) = x/n (the relative frequency) p, The number of successes E expected j 3) i is ina sample of m trials p= mxP(A) j.e. Expectation = (number of tri f in any one trial) Of trials) x (probability of success 49 CLASS WORK p Itisknown from past records that 8% ici Geen ’o of moulded plastic items a. The probability that item is (j ee acceptable any one item is (i) defective, (ii) p. The number of acceptable items like i Tne batch of 4500 s likely to be found in a 340 CLASSICAL PROBABILITY ¢ Theclassical approach to probability is based on a consideration of the theoretical number of ways in which it is possible for an event A to occur. As before: a. In any trial, each separate possible result is called an outcome b. the particular occurrence being looked for is the event c. When it occurs, we have a success; when it does not occur, we have a failure. d. The classical probability P of an event Aoccurring is defined by: P(A) = no of ways in which event A can occur total number of all possible outcomes .mulation General Introduction £0 Modelling and Simu! , anormal abla, f rollins * 1,2,3,4,5,6 ° Ifwe Consider the chance result o: coe Se cur only, ie, the total number of Possillege this \ the event We are considering = eS Way out of the six possible outco! therefore P(six) = 1/6 ‘al and P(not six) = 5/6 fa trial, o eM! ° If out of n Possible Cree ir the ev Event A to occur in x ways a} Y ways, then ible f; it is possib ocet tA not to Cur), N=x+y And P(a) =X = n Ss EVENTS n 3.11 CERTAIN AND IMPOSE every ime, the : is certai -nn= ve y= 0 “ero, ere vt any time, then: tane oa cannot possibly aan auf a tan ev =n_ therefore, P(A) = Pamcertaiaryyie ; therefore P(certainty)=1 and --(6) In most cases, Probability values lie between the extreme values, So: If one card is drawn at Tandom from a ful] Pack of the Probability of obtaining, a. Aheart is. b, Aking ig © Acard other than aking ig d. MPlack card ig 7 playing Cards,LY EXCLUSIVE 2 MUTUAL AND MUTUAL 0 PycLUSIVE EVENTS NON a. Mutually exclusive events are events w] together. For example, in rolling a die, the event of throwing a6and that of throwing a5 Cannot occur at the same time. Similarly, when drawing a card from drawing an ace and that of drawi the same single trial, hich cannot occur ing a king cannot occur in b. Mutually non-exclusive events are the events that occur simultaneously. For example, in rolling a die, the event of obtaining a multiple of 3 and the event of obtaining a multiple of 2 can occur together if a 6 is thrown. Similarly, with cards, the event of drawing a black suit and that of drawing a queen can occur together if the ........ or the sacisnest is drawn. Exercise State whether the following pairs of events are mutually exclusive or mutually non-exclusive. SETS A SETS B A= {ace} B = {black card} A= {heart} B = {ace of spades} A= {red card} A= {5 of clubs} A= {card < 10} B = {10 of clubs} B = {king of diamonds}: 4 mo jeneral ttn iv toto Ming and St —— JAN ADDIT 10) I thore are IN LAW OF PROBABILIFY piel «ive a N possible al vents ‘A ang’ ’ I outcomes toa ttt ‘omes Lod ihe A event A OF ey iy Nand and ¥ give a sive an ever eile! Muttially exctys 1H, then, prove pet occuring | ii ve, the probability” : 8 but clearly . Morty — 4 ay not both i y n wnt y/n po) par! , ps) 90 giver ol There hetefore P(A or B) = PA) yell ility nd BY [ithe ji he events A and Bare ne eur Logether, then the probal Por BY = Peay 1 PU) MA" For example, in rolli 3 of 8) = % i.e “orig, 3/6 = yy | 1 (ie. 248) = 0 © PO gora mUulUPle of » ) ple of; The probability of 8 ie of 3 PA) The probability of s¢¢ So, the probability would seem to be r P(A) + PCB) = 2/8 13/6 = 9/0 : ; | Wait a minute, (6) appears 10 poth sets of Outcome | ive, the events are ly exclu id this outcome hi, | been counted avice. | ‘The probability ofsC0" | P (six) = 1/6 = PA and B) | | | | ving alle fscoring multi! outcome a I sive, au one not mutual ing one of the 6m ust therefore be removed ‘Therefore, in this example, P(A or) = 2/3 Because P(A) = 2/6, PCB) = 3/6; P(A and B) = 1/6 Therefore P(A orB) = P(A) + P(B) - Pt ja ) -P(Aand B) = 2/6 + 3/6 - 1/6 = 2/3 single card is drawn fr om a i pack of 52 playing cards. a. event A = {dj = {drawin, seven} ig an ace} and evs ent B = {drawi ing @Stochastic Processe ability of drawing eitl | jobability ol g either an im 4 event A= (drawing yed card}, then pvorB) ACE OF A ge aking) and ev en, Le. (Nori) eM © {drawing a for mutual sna Bie. Aor Bean oceur, but events A and B ie. jot both a tie same time: Sanna por) = PAY + PB) both ea oth can occur, in any one trial: P(A or B) = P(A) + P(B) - P(A and B) 3.44 INDEPENDENT EVENT AND DEPENDENT EVENTS Events are independent when the occurrence of one event does not affect the probability of the occurrence of the second 7 7 For example, in rolling a die on two occasions, the outcome ofthe first throw will not affect the probability of throwing a six on the second throw. : ° Events are dependent when one 7 robability of the occurrence of the seal oe re Thus, the probability of drawing an ace from a pack of cards is 4/52 = 1/13 Ifthe card is replaced so that the pack is complete and shuffled, the probability of drawing an ace on the second occasion is similarly 1/13 (independent events) However, if the ace is drawn on the first attempt and not replaced, the probability of drawing an ace on the second occasion becomes 3/51 This is because there are now only 3 remaining aces in the incomplete pack of 51 cards (dependent events) 3.15 MULTI-PLICATION LAW OF PROBABILITIES Let us consider the probability of the occurrence of both events A andB where event A = {throwing a 6}when rolling a die and event B = {throwing an ace} from a pack of cards.MoGe ee eee q yal nraduction wo ao General tntrodi sare clearly independent events: oe place) = 4/52 = 1/13 i" 1/6; PCB) 16) P(A) = PC wae now both roll the die and draw a card as one trig) th - 2 die i - the . 6 possible outcomes from the die tae for cach One th, ae "5 e somes from the cards, gj Of 1% there are 2 possible outcome » Biving ol ti by th putcom : ' ‘ nT ilities of obtaining a 6 and an ae, i ace} Rep, so the probability of event A and event B occurring jg p(Aand B)=4 = 1x4 = 1x4 = P(A) x P(By 6x52 6x52 6 52 So when A and B are independent events: P(A and B) = P(A) x P(B) 3.16 CONDITIONAL PROBABILITY Weare concerned here with the probability of an event B Occur: given that an event A has already taken place. This is denotet’® the symbol P(B|A). If A and B are independent events th Cd by that event A has already occurred will not affect the probabil; fac event B. in that case: Mty of P(B|A) = P(B) (a0) If A and B are dependent events, then event A havii a J ide VINg occu; will affect the probability of the occurrence of ts an example: a . A box contains five 10Q resistors and tw: “ elve 3 i The resistors are all unmarked and of the same phystet ee EXAMPLE @ If one resistor is pi te Picked out at rando i Probability of its Tesistance being 10Q. ineraine te oneside hn : - Hee to be 10? and it is retained on ie ofteseanes 0G ity that a second selected resistor Let A = : A= {109 Tesistor}; 5 eo, andB = 09 wea Mponents Wstor}; 12 componentsSt tochastic Processes 47 TIONS out p44 12=17 3 en p(A) = 5/17 ; _ The box now contains four 109 Tesistor: esistOFS- ‘obabili ein ‘phen the probability of B, (A having ocq pi6 23/4 ‘urred) = P(B\A) = So the probability of Setting a l0Qresistor at ning it, and getting a 300 resistor at the first selection, tail vera, and BA) = 15/68 at the second selection is pocatse p(Aand BJA) = P(A) x P(BIA) = 5/17 x 3/4 = 15/68 50, fA and B are independent events: p(A and B) = P(A) x P(B) andifA and B are dependent events: pA and B) = P(A) x PBIA) .. EXAMPLE Abox contains 100 copper plugs, 27 of which are oversize and 16 undersize. A plug is taken from the box, tested and replaced: a second plug is then similarly treated. Determine the probability that (a) both plugs are acceptable (b) the first is oversize and the second is undersize, (c) one is oversize and the other undersize. Let A = {oversize plug} B = {undersize plug} N= 100 Oversize = 27 Undersize = 16 Acceptable = 100 - (27 + 16) = 57 (a) P, (ist plug acceptable) = 57/100 P, (2™ plug acceptable) = 57/100 Therefore P,, (1st acceptable & 2nd acceptable) = PxP,= 57_x S7_= 3249 = 0.3249 100 100 10000duc” General It 97/100 16/100 spherelore ond undersize) p,, (1st oversize & yy y 16 432. = 0.0435 12 2 j= ee Too 10000 part (b) of the problem by, al t is undersize and the seo, Z — nets 100 which includes This section 7 co Ths se when the firs' covers the ca over: P,(1 undersize) = 16/100 3 p* (2 oversize) = 27/100 Therefore, P,, (1" undersize & 2nd oversize) = 16 x 7 = = 0.0432 100 100 10000 Therefore, P(one oversize and one undersize) = P{(1st oversize & 2n undersize) or d (1* undersize & 2nd oversize) } = P+ P= 432 +_ 432 = 864 = 0.0864 10000 10000 70000 rr ution is straightforward the main tools ** Independent Events P(AorB) = i i set aon P(B) ie. ‘or’ is associated with + = PCA) x P(B) ie. ‘ops i 7 Dependen hie ) i.e. ‘or’ is associated with x (AorB) = P PCA ang » (A) + Pp). = P(A) x Pla)EROS ns 100 copper plugs, 27 overs; Angi taken, tested pe not replaced a Ser ae pean F mil arly. Determine the probability that ee a. both plugs are acceptable, p. the first is oversize and the second undersize, c. one is oversize and the other undersize. 3.17 DISCRETE PROBABILITY DISTRIBUTION ifwe consider a single trial of tossing a coin, th likely possible outcomes, ahead’ or a'tail! the possible outcomes in successive trials, 1, there are two equally (ie. anot-head). Listing , we have: 1 trial: 2 H gtrials: © TT TH HT HH 3trials: TIT TTH THT THH HTT HTH HHT HHH similarly, with 4 trials, the possible outcomes would be those detailed for 3 trials with an extra T or H attached in each case. The possible outcomes with 4 trials are therefore: HEAD TAIL TITT THT THTT THHT HITT HTHT HTT HHHT TTTH TTHH THTH THHH HTTH HTHH HHTH HHHH If we concentrate on x, the number of heads at the conehision or each set of trials, and f, the frequency of x, we can tabulate Tesults, thus:A i it tion 50 General Introduction to Modelling and simula J NUMBER OF NUMBER OF| NUMBER OF POSSIBLE Heapsx>o123 4 OUTCOMES | TRIALS = probability , Whi = Frequenc and P = Pro! Ther is clealy a pattern here and we a ida that tk probability values are the separate terms of the exP' (y + 1/2)n where nis the number of trials in each case, i.e. the tem of the binomial expansion of (a+ b)n where a = b= 1/2 You will remember that the binomial coefficien' Pascal's triangle: 1 ts are given | 14 64 1 So with 5 trials, the probabilities of obtaining x heads are Ne vs that, as ever, the total probability Wn =(1+5 41041045 + 1)/32we Stochastic Pp ‘cesses 51 cannot al n studying revision of permutations and ¢o; revs make a 4 PERMUTATIONS AND cong "permutations: If we have n i BINATIONS them in different orders of selec : arrangement is called a permutation Each diff TMUtation wi mination without a brief fresh start in the next section. ns Therefore, we shall For example, permutations of th together are ABC, ACB, BAC, BCA, S feat A,B,C taken 3P3 = 6. The upper 3 denotes the numbe and we say that umber of i . the arrangements are made: the lo of tems from which 5 ae wer 3 indicates t] of items we are using in each arrangement, he number Similarly, taking the same 3 letters A,B, C two ata ti can form the permutations. ‘ime, we So that °P, = ........ NB: Remember that the order of the letters does matter. BA is a different permutation from AB, etc. => AB, BA, AC, CA, BC, CB Therefore, *P, = 6 In general, to find nPr, ie. the number of permutations of n different items taken rat a time, we can argue as follows. Of the r positions in each ordered arrangement, The first place can be filled in n different ways, The second place can be filled in (n - 1) different ways, The third place can be filled in (n - 2) different ways, The rth place can be filled in (n - r + 1) different ways, Therefore, the r places can be filled in n(n-1)(n- 2)...(n-1 + 1) different ways ie. =n(n-1)(m-2)...Qa-r+ 1).2 OY 52 General nro roduction to Modelling and Simulation Modeling and Simulation So, 5P4 = 120 oa Because, 5P4,n= 5 andr =4 Therefore, 5P4=5x4x3x2 120 divide result (12) by ifr? n, we can multiply and (n-r)(n-r-1)...3%2%1 - r+ yx(n-O-T D329, *) Then, nPr = ynPr = n(n-1)(n-2)..- (n-ry(a-T~ 13X21 Therefore, ool factorial 2 (n-r)! factorial (n-1) Using (13):°P, = 3 gxaxl =H Ising (13): “Ps a 7 : _what do we mean by 0!? Wait a minute. eviously, We established that 3p3 =6 Pre Therefore, °P, = oe = 6 oO! +-++(14) Therefore, 0! = 1 In that case *P, Therfore, nPn= n! $0, ®P, = 5040... Because, pee 10 40x 9x8x7XOX5X4X3X2K1 6! 6x5x4x3x2x1 =10x9x8x7 = 5040 2. Combinations: Returning to our letters A, B, C we now make selections without regard to the order of the letters in each group, ie, AB is now the same as BA, etc. each group is called a Fae and*C,, where nis the total number of items and ris e number in each selection, gives the number of possible combinations.Stochastic Voce cumple yore + (ARG). Giant that NCA, ACH,. ete, 1, * C= spheretores "Cy 150, 1) = FAB BEL CAY Ket, og Ali, that AB ANC BA are differen pn va ent combinitions, lie general, 10400 "Ce, the mun 0 ent irems taken pata time, we aifiay one combination ofr hen ati vl different Permutation: ee rall her grolnalum sce of ( x vl) different permut ations, butare not nber of combinations ‘aN procec Vbe rear of n s follows ‘anged within itself Arranged, we should have a total ations, the total number of ermutations fom n items taken rat atime, i nC, xa! ay, a ieni ‘Therefore, ©, \ rirt 00, 10! 1Ox9XBxX7x6x5x4x3x2x1 we “Aol (4x3 x 2x1) x(6x5x4x3 x21) 1O0x9xX8x7 4x3x2x1 = 6210 EXERCISES: 1. Evaluate the following: i Ol vii, P. i vill. "C, iii, “P, ix, °C, iv "P, x °C, YP xi. °C, : : vi. 5p xii. °C, : 154 : General Introduction to Modelling and Sinsiloo™ rsiZe and 9 undersiz, a second Plug is 2. a. Abox i ‘opper plugs: # ce ty tha . contains 25 ci ° . i PP pla pabilil t(j) i oversize and the A plug is taken, tested but not © d th d the othe, then treated similarly, Deter™i?® both plugs are acceptable (ii) the yersize 4 second undersize; (iii) o8¢ ° undersize. simulation: b. Describe the concept of Moat® carlo jfferentiate between c. With the use of relevant sce? 08 ‘exclusive events. mutually exclusive and mutually ol ow many WAYS can d. Fora set {The English Alphabets}, ombined? the letters B, E, G, K, M and Q bee