Page 1 of 9

MAT 326 – Probability and Statistics for Engineers

Chapter 7: Multivariate Probability

In chapters 5 and 6 we explained how each time when we are doing a random operation,
then we can introduce one random variable X discrete or continuous”, we also talked
about f(x).(the probability distribution “density” function of X. Sometimes we may need
to introduce two or more random variables. In this chapter we shall deal with random
operations for which we need two random variables, say X, Y, in this case we shall need
to talk about f (x, y) the joint probability distribution (density) function of X and Y. We
shall assume that either both X, and Y are continuous or both are discrete

Example 1: Discrete Case

In a supermarket, there are 3 checkout counters A, B, and C. Two customers arrive at the
counters at different times. Assume that a customer chooses one checkout counter at
random and independently of other customers.
Let X be a R V to represent the number of customers selecting the first counter A.
Let Y be a R V to represent the number of customers selecting the second counter B.
So X: 0, 1, 2, and Y: 0, 1, 2

Let us find for example

P ( X=0 , Y= 0 )= p (Both customers selected the third counter C)
¿ p (1st customer selected C). p (2nd customer selected C) “independent”
1 1 1
¿ ⋅ = .
3 3 9
By the same way (or by using a tree-diagram) as it is shown below

1.st 2.nd outcomes

A X= 2 , Y=0 We have 9 equally likely outcomes

B X=1 , Y=1
P (each one) =1/9
A C X=1 , Y=0
A X=1 , Y=1 So we can calculate P (X=x ,Y=y)
B B X=0 , Y=2
C X=0 , Y=1 for all the values of X and Y
C
A X=1 , Y=0
B X=0 , Y=1

C X=0 , Y=0

The calculated values can be used to construct the following table.

 Page 2 of 9

Marginal Probability
X
0 1 2 for Y
1 2 1 4
0
9 9 9 9
2 2 4
Y 1 0
9 9 9
1 1
2 0 0
9 9
Marginal Probability 4 4 1
1
for X 9 9 9

Such a table is called the joint probability distribution table of X and Y

Definition:
Let X, Y be in general two discrete random variables. We can talk about a special
function f (x, y) called the joint probability distribution function of X and Y, which is
defined in such a way f (x, y) = P (X= x, Y=y) for all values of x and y.
So, in order for f (x, y) to be used as joint probability distribution of X, and Y we must
have 1) f (x, y) ≥ 0 for all x and y
2) ∑ ∑ f ( x , y)=1
all y all x
3) f (x, y) = P (X= x ,Y= y) for all x and y

Remark: The joint probability distribution f (x, y) can be used to find the marginal
probability distribution functions of X and Y.
We have f 1 ( x )=∑ f (x , y)
¿x

And f 2 ( y ) =∑ f ( x , y )
¿y

For our last example P (X =1) = f 1 ( 1 ) = P (X=1, Y = 0) + P (X=1,Y = 1) + P(X=1,Y = 2)

y=2
2 2 4
= + +0= = ∑ f ( 1, y ) in general
9 9 9 y=0

f ( x ) = P (X=x) = ∑ f (x , y) and
1 f ( y ) = = P (Y=y) = ∑ f (x , y)
2
¿y ¿x

Examples 2: Suppose that X and Y are two discrete random variables with
x. y
f (x, y) = for x= 0, 1, 2 and y= 1, 2, 3
18
xy 6 x x
So f 1 ( x )=¿ ∑ f ( x , y )=∑ = = for x= 0,1,2
y y 18 18 3
xy y
And f 2 ( y ) =∑ f ( x , y ) =∑ = for y=1,2,3
x x 18 6
 Page 3 of 9

x+ y
Example 3: f (x, y) = for x= 0, 1, 2 and y= 1, 2, 3
27
x+ 2 y+ 1
So f 1 ( x )= for x = 0.1.2 and f 2 ( y ) =¿ for y=1,2,3
9 9

Continuous Case
Definition: Each time we have X and Y as two continuous R Vs, then we shall have the
function called the joint probability density function of X and Y
A lways the followings aretrue
1) f ( x , y ) ≥ 0for all x and y
∞ ∞

2) ∫ ∫ f ( x , y ) dxdy =1
−∞ −∞

3) P ((X, Y) ∈ A ¿ = ∬ f ( x , y ) dA where dA stands for dx dy or sometimes dy dx

¿A

Example (4) Let X and Y be two cont. random variables their j.p.d.f is given by

f(x , y) = {K ( x +2 y )0foreles0<where
x< 1∧0< y < 2

a) Find K ------- since f (x, y) is a j.p.d.f , therefore ∬ f ( x , y ) dx dy=1

−∞
1
This gives k =
5
2 0.5
1 13
b) P (0 < x < 0.5, 1 < y < 2) =
5 ∫ ∫ ( x+ 2 y ) dx dy = 40
1 0

Remark: Again, in this case we can get the marginal probability density functions for X
and Y, were
∞ ∞
f 1(x) = ∫ f ( x , y )dy and f 2(y) = ∫ f ( x , y )dx
−∞ −∞

For our example we have

{
∞
1
2 1
( 2 x +4 ) for 0< x <1 ¿
f 1(x) = ∫ f ( x , y )dy = ∫ ( x +2 y ) dy =¿ 5
5 0
−∞ 0 elsewhere

{
∞
1
1 1
( 0.5+ 2 y ) for 0< y <2
f 2(y) = ∫ f ( x , y )dx = ∫ ( x +2 y ) dx= 5
50
−∞ 0 elsewhere

Example 5
 Page 4 of 9

Let X 1, and X 2 be continuous with f ( x 1 , x 2 ) =

1 0.5
{
2 ( 1−x1 ) 0 ≤ x 1 ≤1 , 0 ≤ x 2 ≤ 1
0 elsewhere.
3
So, P ( 0 ≤ X 1 ≤ 0.5 , 0.5 ≤ X 2 ≤1 ) =∫ ∫ 2 ( 1−x 1 ) d x 1 d x 2. =
0.5 0
8

{
− ( x+ y )
e
Example 6 : f(x , y) = 0 elsewhere for x > 0 and y > 0

So f 1(x) = ∫ e
−(x+ y)
dy = e− x for x > 0 and f 2(y) = e− y for y> 0
0

Example 7: f (x , y) = {x y for00<elesx <1∧0<

where
y< 2

∞
y
S0 f 1(x) = ∫ f ( x , y )dy = 2x for 0 < x <1 and f 2(y) =
2
for 0 < y <2
−∞
 Page 5 of 9

Sec. 7.2: Conditional Probability Distributions

In general, the joint probability distribution (density) of X and Y can be used to calculate
the conditional probabilities for X given Y or for Y given X.

General discrete Case: X and Y are discrete

P(¿) P ( X ≤ a , Y ≥ b)
P ( X ≤ a∨Y ≥ b ¿ = = Provided p(Y ≥ b)≠ 0
P (Y ≥b) p (Y ≥ b)

x+ y
Back to example 3: Given f (x, y) = for x= 0, 1, 2 and y= 1, 2, 3
27
x+ 2 y+ 1
We get f 1 ( x )= for x = 0.1.2 and f 2 ( y ) =¿ for y=1,2,3
9 9

P(¿) P ( X <1 , Y =2) 2/27 2

Calculate P (X<1 | Y=2) = = = =
P (Y =2) P(Y =2) 1/3 9
P (X ≤1 , Y >1) P ( 0 , 2 ) + P ( 0 , 3 ) + P ( 1, 2 ) + P(1 , 3)
Calculate P (X≤ 1|Y > 1 )= = =
P ( Y >1 ) P ( Y =2 )+ P (Y =3)
2 3 3 4
+ + +
27 27 27 27 12
=
3 4 21
+
9 9
P(¿) P ( 0 , 2 ) + P ( 0 , 3 ) + P ( 1, 2 ) + P(1 , 3)
Calculate P (Y>1 |X≤ 1¿ = = =
P ( X ≤1) P ( X=0 ) + P( X =1)
2 3 3 4
+ + +
27 27 27 27 12
=
2 3 15
+
9 9

General continuous Case: X and Y are continuous

P(¿) P ( a1 ≤ X ≤ a 2 , b1 ≤Y ≤ b2 )
P ( a 1 ≤ X ≤ a2|b1 ≤ Y ≤ b2 )= =
P ( b1 ≤Y ≤ b2 ) P ( b 1 ≤ Y ≤b 2 )
Provided P ( b1 ≤Y ≤ b2 ) ≠ 0

{
1
( x+2 y ) for 0< x <1∧0< y< 2
Back to example 4: f (x , y) = 5
0 eles where

{
1
( 2 x + 4 ) for 0< x <1
Where f 1(x) = 5
0 elsewhere

{
1
( 0.5+2 y ) for 0< y <2
and f 2(y) = 5
elsewhere
0
 Page 6 of 9

2 0.5
1 13
and we got before P (0 < X < 0.5, 1 < Y < 2) =
5 ∫ ∫ ( x+ 2 y ) dx dy = 40
1 0

13
P(¿) P (0< x <0.5 , 1< y <2) 40
So, P (0 < x < 0.5 ¿ 1 < y < 2) = = = =¿13/28
P (1 ≤ Y ≤ 2) P (1<Y <2) 7
10

Special continuous case: (only when X and Y are continuous)

Definition: Let X and Y be two continuous random variables with joint probability
density function f (x, y) and marginal density functions f 1 ( x ) and f 2 ( y ) .
Note that the above general continuous formula can’t be used to calculate for example
P (a <X <b | Y = k) or P ( c < Y< d | X=k ) When X and Y are continuous, this
0
will lead us to get (undefined). To handle the problem, suppose we want to calculate
0
for example, P (a <X <b | Y = k), in such a case we need first to find the conditional
probability density function of a new continuous random variable (X |Y =k ¿which given
by

{
f (x ,k)
when f 2 ( k ) ≠ 0
g ( X |Y =k )= f 2 ( k )
0 elsewhere
b

And in this case, P (a < X < b | y = k) = ∫ g ( X|Y =k ¿ dx ¿

a

Back to example 5: given f (x , y )= {2 ( 1−x 1 ) 0 ≤ x 1 ≤ 1, 0≤ x 2 ≤ 1

0 elsewhere

{
We have f 1 ( x 1 )=
2 ( 1−x 1 ) 0≤ x 1 ≤ 1
0 elsewhere
and f 2 ( x 2 )={1 0 ≤ x 2 ≤1
0 elsewhere

Let us find P ( X 2 < 0.75| X 1=0.5 ), we need g ( X 2| X 1=0.5 ).

We have f ( 0.5 , x 2 )=2 ( 1−0.5 )=1

f 1 ( 0.5 )=2 ( 1−0.5 ) =1

{
f ( 0.5 , x 2 ) 1 1 for 0 ≤ x 2 ≤ 1
 g ( X 2|X 1=0.5 ) = = = .
f 1 ( x 1=0.5 ) 1 0 elsewhere
0.75 0.75

So, P ( X 2 < 0.75| X 1=0.5 ) = ∫ g ( x 2| x1 =0.5 ) d x 2 ¿ ∫ 1 d x 2=0.75.

0 0

Note that for this example, we can’t calculate, say P ( X 2 < 0.75| X 1=1 ) since
f ( 1 , x 2) 1
g ( x 2| x1=1 ) = = undefined ( f 1 ( 1 )=2 ( 1−1 )=0 ¿
f 1 ( x 1=1 ) 0
 Page 7 of 9

Back to example 6: where f (x , y) = {x y for00<elesx <1∧0<

where
y< 2

∞
y
f 1(x) = ∫ f ( x , y )dy = 2x for 0 < x < 1 and f 2(y) =
2
for 0 < y <2
−∞
Let us find P (0 < X <0.5 ¿ Y =1 ¿
f ( x , 1) x
We need g (X | Y=1) = = = 2 x for 0 < x < 1
f 2 ( y =1) 0.5
0.5

So, P (0 < X < 0.5 ¿ Y =1 ¿= ∫ 2 x dx = 1/4

0

Independent Random Variables

Definition: For both types, two random variables X and Y are said to be independent
f ( x , y )=f 1 ( x ) ⋅ f 2 ( y ) for all the values of x and y
Otherwise, we say they are dependent
Note that when X and Y are discrete, then they are independent
P ( X=x , Y = y ) =P ( X=x ) ⋅ P ( Y = y ) for all the values of x and y.
They are dependent there exist at least one x and one y such that
P ( X=x , Y = y ) ≠ P ( X=x ) ⋅ P ( Y = y )
Remark: We can say that
X and Y are independent P (X | Y) = P (X) and P (Y | X) = |P(Y) for all x,y

Back to our example (1) of the discrete case, we can show that X 1 and X 2 are not
1
independent (they are dependent) since, for example P ( X=0 ,Y =0 )= ,while
9
4 4 16
P ( X=0 ) ⋅ P ( Y =0 ) ¿ . =¿ and hence ( X =0 , Y =0 ) ≠ P ( X=0 ) ⋅ P ( Y =0 )
9 9 81

Back to example 5 of the continuous case, X and Y are independent since

f ( x , y )=2 ( 1−x )=2 ( 1−x ) ⋅1=f 1 ( x ) ⋅f 2 ( y ).

Back to example 4, X and Y are not independent (they are dependent) since there are at
least one x and one y were,
1 1 1
f (x ,y) = ( x +2 y ) ≠ ( 2 x+ 4 ) . ( 0.5+2 y ) = f 1(x). f 2(y)
5 5 5

Expected values of Functions of Random Variables

 Page 8 of 9

Definition: Let X ,Y be two random variables. If g ( X , Y ) is any real function of X and Y,

{
∑ ∑ g(x, y)f (x , y ) If X , Y discrete
E ( g ( X , Y ) )= over y over x
then ∞ ∞

∫ ∫ g ( x , y ) f ( x , y ) dxdy If X , Y continuous
−∞ −∞

Back to example 1 of the discrete case.

Let us find E ( X 1+ X 2 ), i.e. g ( X 1 , X 2 )= X 1+ X 2. We get E (X+Y) = 12 / 9
We also find E ( X 1 ⋅ X 2 ), i.e. g ( X 1 , X 2 )= X 1 ⋅ X 2. We get E (X.Y) = 2 / 9

Back to example 4, continuous, were

1
f (x , y) = ( x +2 y ) for 0< x <1∧0< y <2
5
Let us find E ( X ⋅Y ) , i.e. g ( X , Y )=X ⋅Y and hence
2 1
1 5
E (X.Y) = ∬ f (x , y )dx dy = ∫ ∫ ( x +2 y ) dx dy = …… = =1
0 0 5
5

Definition:
Let X, Y be two random variables, we define the covariance between X and Y (denote it
Cov ( X , Y ) ) to be given by Cov ( X , Y ) =E ( X ⋅Y )−E ( X ) ⋅ E ( Y ) .

Note that if and Y are independent then E ( X ⋅Y )= E ( X ) ⋅ E (Y ) and hence

Cov ( X , Y ) =0, but the converse is not true.

 The covariance helps us to have an idea about the relationship between X and Y,
If Y tends to be large (small) at the same time X is large (small) then cov ( X , Y ) is
positive, on the other hand.
If Y tends to be small (large) when X is large (small) and large when Y is small
then cov ( X , Y ) <0 (negative).
back to example (1)
2 2
0 ( 1 ) +0 ( 2 ) +0 ( 1 ) +0 ( 2 ) +1 ( 2 ) +2 ( 0 ) +0 ( 1 ) +2 ( 0 ) +4 ( 0 )
E ( X 1 ⋅ X 2 ) =¿ ∑ ∑ x . y P ( x , y ) = =¿ ¿
0 0 9
2 6
= while E ( X 1) =( X 2 ) = ⇒
9 9
2 6 6 −2
Cov ( X 1 , X 2) = − . = the answer is negative which means as X increases then Y
9 9 9 9
decreases and vice versa

Back to example 6
2 1 1

E (X.Y) = ∫ ∫ x . y xy dx dy = 8/9, while E(X) = ∫ x ( 2 x ) dx=¿ ¿ 2/3

0 0 0
 Page 9 of 9

2
y
and E(Y) = ∫ y ( ¿ )dy =¿ ¿8/6, therefore Cov (X.Y) = 8/9 – 2/3. 8/6 = 0 (remember we
0 2
showed before, that X and Y are independent)

Theorem:
If X 1, X 2 are two random variables and Y =a1 X 1+ a2 X 2 +k , where a 1, a 2 , k ∈ R , then
1) E ( Y )=E ( a1 X 1 +a2 X 2 +k ) =a 1 E ( X 1 ) + a2 E ( X 2 ) +k .
2 2
2) V ( Y )=V ( a 1 X 1+ a2 X 2+ k ) =a1 V ( X 1 ) +a2 V ( X 2 ) +2 a1 a2 cov ( X 1 , X 2 ).

Example : Given two random variables X and Y with E(X)= 2, E(Y)= 5, V(X)= 9,
V(X)= 4 and Cov(X,Y)= 0.75. Let W= 3X – Y. Calculate E(W) and V(W)

The Multinomial Distribution

This distribution is a generalization of the binomial distribution.
Consider the experiment of repeating n identical independent trials where the sample
space of each trial consists of k outcomesT 1 , T 2 … … , T k with probability P11 , p2 , … … , p k
respectively. Let X 1 , X 2, … … , Xk be random variables, X i =0 ,1 , 2 , … … n ∀ i with value
of X i as the number of T i results in n trials. With such experiments we ask: What is the
probability of getting x 1 , x 2 ,… … , x k results of the type T 1 , T 2 … … , T k respectively. The
probability distribution that gives us such probability is
n! x x x
P ( X 1=x 1 , X 2=x 2 , … … , Xk=x k ) = P1 P2 … Pk were
1 2 k

x 1! x 2! … xk !
k k

∑ x i=n, ∑ Pi =1, and for each i, x i=1 , 2, … … , n.

i=1 i=1
We call this distribution by the multinomial distribution.
Note that if k =2, then for X 1=x 1 , we get X 2=x 2=n−x 1 and when P= P1 we get q=1-P
And hence P ( X 1=x 1 , X 2=n−x 1 ) =P ¿ and this is in fact the binomial distribution. P
¿
Exercises: Problem 5.31

Remark: Note that when we have a multinomial distribution, then

We can show that E ( X i) =n pi V ( X i ) =n p i qi for i=1 , 2 ,… … ,k
i.e. for all i X i is a binomial random variable with n and Pi=P(success)
And cov ( X i , X j )=−n Pi P j for i≠ j.

Exercises: 7.25, 7.26, 7.30 and one special example

Based on time we solve 1, 2, 4, 6, and 8