Joint Statistical Meetings - Section on Quality & Productivity (Q&P)

A Bayesian control chart for Service Quality Control

Bianca M. Colosimo and Q. Semeraro

Dipartimento di Meccanica - Politecnico di Milano
Piazza Leonardo da Vinci 32, 20133 Milano, Italy
([email protected])

Key Words: Service Quality, Control charts, isfaction should be considered as different concepts,
Bayesian Networks. most of the studies reported that they are highly cor-
related and, besides, indistinguishable for long-term
Abstract: customers (Dabholkar et al., 2000). Therefore, no
Service quality is becoming a strategic factor to re- specific distinction will be considered in the follow-
tain existing customers and acquire new ones. It ing. A further controversial question discussed in
can be directly related to the extent to which the the literature concerns whether the service quality
service successfully fulfills the needs of a customer should be measured as perceptions or disconfirma-
and is often measured through questionnaires that tions (i.e., difference between expectation and per-
are submitted to a population of customers. Differ- ception). Considering industrial practice the sec-
ent approaches have been proposed in the literature ond approach is almost always used, probably be-
to analyze collected data. This paper presents a new cause of the inherent difficulty in measuring expec-
technique aimed at overcoming some limitations of tation (Dabholkar et al., 2000). Hence, we will adopt
current methods and to go further exploring the way the first approach as reference assuming that service
in which this analysis can be used to design a control quality is measured by directly asking the customer
chart to monitor service quality with time. to express a judgement concerning the degree of sat-
isfaction/dissatisfaction related to the specific ser-
1. Introduction vice offered. The overall level of service quality is
often determined by a set of more specific features,
During the last decade, quality issues are more and called service quality characteristics, which can dif-
more often related to services as well as products. ferently affect the global satisfaction (e.g., the time
This trend can be explained considering changes to access the service, the queuing time, the ability of
which are affecting the global competitive scenario. the front-end operators, etc.). Therefore, it is usual
Most of the firms competing in large-scale markets practice to design a questionnaire containing a set
are in fact focusing on services offered together with of questions about the level of satisfaction with each
products (e.g., marketing, logistics, post-sales assis- specific quality characteristic. Then, the question-
tance, etc.) as a mean to customize their offer when naire is submitted to a population of clients and the
products are identical. Besides, the number of com- analysis of collected data allows to determine the
panies offering only service is impressively increas- actual level of service quality (overall satisfaction)
ing, as an effect of the recent impulse toward busi- and to further investigate the ”importance weight”
ness which are related to ”transforming” information of each quality characteristic, i.e., to what extent
instead of products. As a consequence, the quality each characteristic influences the global satisfaction.
service control is becoming a primary key to acquire
and retain customers and it has to be considered in Most of the approaches presented in the literature
a strategic perspective. do not directly focus on tools to monitor it with
Despite of the agreement in considering service time, but they offer a valuable reference on the sta-
quality control a relevant issue, the definition of a tistical analysis of data collected through question-
general methodology is still an open problem (Dab- naires. In particular, most of the models proposed
holkar et al. 2000). A first controversial issue con- in the literature (Cronin and Taylor, 1994; Parasur-
cerns how service quality can be defined and, as a aman et al., 1994) use factor analysis to handle the
consequence, measured. According to Juran (2002), multivariate nature of service quality. Besides the
service quality can be defined as the extent to which traditional critique that refers to factor analysis as
the service successfully fulfills the needs of a cus- ”too subjective” due to the lack of a ”well identi-
tomer, i.e., his/her satisfaction. Although some au- fied criterion for judging the quality of the model
thors outlined that service quality and customer sat- investigated” (Johnson and Wichern, 1992), the use

615
 Joint Statistical Meetings - Section on Quality & Productivity (Q&P)

of factor analysis in the quality service framework conclusions and the directions for further research.
has the drawback of requiring the translation of cat-
egorical variables – the judgments of the customers
2. Using Bayesian Networks to model
– into continuous ones, adopting a score scale. To
overcome this problem, Franceschini and Rossetto service quality
(1999) proposed an approach to avoid the phase of Assume service quality is characterized by I charac-
translation of judgments into scores, suggesting the teristics, which represent features of the service that
use of Multiple Criteria Decision Aiding (MCDA) are supposed to influence customer satisfaction. For
to analyze collected data. In this model the uncer- each of these characteristics the customer is asked
tainty characterizing the customer responses is mod- to express a degree of satisfaction, e.g., an answer
elled through the use of fuzzy numbers. In this last among a set of possible ones ranging from ”com-
approach, as well as in most of the methods pre- pletely unsatisfied” to ”completely satisfied”. Since
sented in the literature, the importance weights are the number of possible outcomes for each question is
assumed to be known, i.e., it is assumed that some often greater than two, answers by customers can-
questions in the questionnaire are used to directly not be modelled as a Bernoulli trial (yes/not, i.e.,
ask the customers to specify an importance weight 0,1) and the binomial distribution cannot be used
for each quality characteristic. Only few exceptions to model the number of customers satisfied with
to this general rule (as the ’unweighted SERVPERF’ that characteristic. Therefore, a multinomial distri-
in Cronin and Taylor, 1992) are reported in the lit- bution (i.e., the natural extension of the binomial
erature as performing equally likely. Since most of when the possible outcomes are more than two),
the solutions proposed in industrial practice do not will be adopted to model the generic variable Xi
include questions related to the importance weight, (i = 1, 2, ..., I), with x1i , x2i , ..., xci i representing all its
the effect of each indicator on the overall satisfac- possible outcomes (degree of satisfaction). The set
tion should be directly estimated by data collected. of characteristics monitored through the question-
Furthermore, data collected in reality include miss- naires can be modelled as a vector of categorical ran-
ing entries, while all the approaches presented in the dom variables X = {X1 , X2 , ..., XI }. It can be as-
literature assume all the data are completely avail- sumed, without loss of generality, that the first vari-
able. A statistical analysis should therefore handle able X1 represents the answer related to the overall
the possibility of missing values. satisfaction.
This paper presents a new approach to service Since variables in X are not independent, a
quality control aimed at overcoming limitations out- Bayesian Network (BN) can be used to repre-
lined in the methods presented in the literature. sent conditional dependencies among variables in X
The approach presented is based on Bayesian Net- (Spieghelhalter et al., 1993; Cowell et al., 1999 and
works (BNs) and permits to handle the discrete na- Heckermann, 1998). A BN for a set of variables X
ture of collected data, to estimate the relationship is basically constituted by a Directed Acyclic Graph
between the service quality characteristics and the (DAG), representing the assumptions of conditional
overall satisfaction when importance weights are un- independence between variables in the graph, and
known (i.e., they are not directly asked to the cus- a set of local distribution functions (P ). In par-
tomer) and to include the possibility of missing data. ticular, in the DAG each node corresponds to one
The effectiveness and the applicability of the method variable in the set X and an arc directed from Xi
to analyze data in real cases has been shown in to Xh represents the conditional dependence of Xh
(Colosimo, 2001). In this paper the use of BNs is from Xi , i.e., the hth variable has a distribution
further explored in the traditional framework of Sta- that depends on the value assumed by the ith one.
tistical Process Control (SPC) to design a control In this case Xh is called a child of Xi and, equiv-
chart for monitoring service quality with time. alently, Xi is referred as a parent of Xh . The set
This paper is organized as follows. The second of parents of Xi is therefore a subset of the ran-
section introduces the Bayesian Network as a tool to dom variables in X and is denoted with Pai ⊂ X.
represent service quality. The third section describes The generic realization
of the parents of  Xi , Pai ,
1 2 qi
how data collected through questionnaires can be is denoted
 with pa i = pa i , pa i , ..., pa i , where
used to determine the Bayesian Network which rep- qi = Xi ∈Pai ci . When the DAG is known, the par-
resents the current level of service quality. Starting ents of the generic variable Xi can be identified and
from this model, the fourth section presents the de- the variable is thus characterized by a local distri-
sign of the control chart for monitoring service qual- bution function that is a collection of multinomial
ity with time. Finally, the fifth section presents the distributions, one for each configuration of its par-

616
 Joint Statistical Meetings - Section on Quality & Productivity (Q&P)

data collected D = {x1 , ..., xN } are therefore a set of

cases. Given this set of cases, the BN which better
represents collected data has to be determined (fol-
lowing the literature on BNs, we will call this phase
learning phase). In order to learn a BN from data,
two steps have to be performed:
1. the qualitative learning (or selection of the
model) which consists in determining the DAG:
this phase permits to identify, for each variable
Xi , its parents Pai ;
Figure 1: A simple example of DAG in a Bayesian
Network 2. the quantitative learning: the derivation, for a
given DAG, of all the conditional distributions
required to compute the full joint probability.
ents Pai . Denote with θijk the parameter charac- Once these two phases have been performed, the
terizing the probability that the ith variable is in BN derived represents the statistical relationships
the k th state, given the j th configuration of the par- among variables (and in particular between the dif-
qi
ents. Hence, θ i = ((θijk )ck=2
i
)j=1 represents the vec- ferent specific service features and the overall sat-
tor of parameters characterizing the local distribu-
 ci isfaction represented by the variable X1 ) which, in
tions of the ith variable (where θij1 = 1 − k=2 turn, explain the degree of satisfaction with each ser-
θijk ). As an example, consider the problem of rep- vice quality characteristic. The qualitative learning
resenting the effect of the second and third charac- is executed comparing competitive models on the
teristics (X2 and X3 ) on the global satisfaction (X1 ) basis of a ’score criterion’ that, following traditional
and, for simplicity, assume that the three variables approaches in BNs (Heckermann, 1998 and Spiegel-
are characterized by two possible outcomes: 1 (low halter et al., 1993), is the logarithm of the rela-
satisfaction) or 2 (high satisfaction). Assume most tive posterior probability, log p(D, M h ), that can be
of the customers consider both the characteristics computed for a given DAG (representing the model
X2 and X3 are relevant for their overall satisfac- M h ) using the collected data D as:
tion. Under this assumption we will observe a 2 (or
a 1) for X1 together with a 2 (or a 1) for both X2 log p(D, M h ) = log p(M h ) + log p(D|M h ) , (1)
and X3 , i.e., the distribution of X1 will be condi-
tioned by the values assumed by the variables X2 where log p(M h ) is the logarithm of the prior prob-
and X3 . The DAG is thus characterized by three ability assumed for the hth model and log p(D|M h )
nodes and two arcs, one directed from X2 to X1 is the logarithm of the marginal likelihood, i.e. the
and another directed from X3 to X1 (Figure 1(a)). probability of observing the data in D given that the
Therefore the parents of X1 are Pa1 = {X2 , X3 }, model M h is true. Unless some specific information
while Pa2 = {∅} and Pa3 = {∅}. Since all the permits to distinguish among different models, all
variables have two possible outcomes, q1 = 4 and the models are a priori equally likely and therefore
pa1 = {(1, 1), (1, 2), (2, 1), (2, 2)}. The set of local the first term in (1) is equal for all the models and
probability function P for the DAG in Figure 1(a) can be neglected. In this case, the score criterion
is therefore given by: θ 1 = (θ112 , θ122 , θ132 , θ142 ), can be directly computed as the log of the marginal
while, since X2 and X3 have no parents, θ 2 = (θ22 ), likelihood log p(D|M h ), denoted in the following as
and θ 3 = (θ32 ). In Bayesian models the parame- logML. The adoption of the logML as score criterion
ters are modelled as random variables, therefore they has interesting properties, as outlined in (Dawid,
can be simply added in the DAG, leading to the full 1984). Firstly, the logML can be rewritten in term
graphical representation reported in Figure 1(b). of sequential prediction of the data in D. Therefore,
the model selected adopting this score criterion can
3. Identification of the Bayesian Net- be considered as the best sequential predictor of the
data under the log utility function. Secondly, as the
work using collected data
number of data collected increases, the logML tends
The set of answers reported in the questionnaire can asymptotically to the Bayesian Information Crite-
be considered as a realization x of X and will be rion (BIC) of Schwarz (1978), which has a direct in-
referred as a case. Assume questionnaire has been terpretation in terms of model fitting and parsimony
submitted to a population of N customers. All the of the model selected.

617
 Joint Statistical Meetings - Section on Quality & Productivity (Q&P)

Since the computation of the logML depends on al., 1995). In particular the data are said to be
the local conditional probabilities P , the phase of ”Missing At Random” (MAR) if the probability of
quantitative learning which consists of estimating observing a missing entry of the ith variable does
the vector θ, has to be performed for all the mod- not depend on the variable itself. Otherwise, miss-
els compared. In particular, under the assumption ing data mechanism is defined as ”Not Ignorable”
of complete data and parameter independence, the (NI). With reference to the BN representation, miss-
estimation of each vector θ ij can be computed inde- ing data are MAR when the probability of observ-
pendently (Cowell et al., 1999). Assume a Dirichlet ing a missing value of Xi depends at most from the
prior (the traditional conjugate prior for multino- configuration of its parent Pai . When this proba-
mial sampling) for each θ ij , i.e.: bility does not depend neither by Xi nor by Pai ,
the missing data mechanism is said to be ”Missing
π(θ ij |M h ) = Dir(θ ij |αij1 , ..., αijci ) , Completely At Random” (MCAR). Although BNs
are suitable to represent all the missing data mech-
where αijk are the hyperparameters. This choice
anisms, the assumption of NI requires additional in-
of the prior has two main advantages. The first is
formation on the pattern of missing data. When
computational and is related to the adoption of con-
this information is not available, the MAR hypoth-
jugate families in Bayesian statistics. In fact, when
esis is often assumed. In this case, the computa-
the parameters of a multinomial distribution have a
tion of the posterior distribution becomes intractable
Dirichlet prior, the posterior, i.e., the distribution of
as the number of missing cases increases. To over-
the parameters once data have been observed, is still
come this problem, the Gibbs sampling (based on
a Dirichlet:
Markov Chain Monte Carlo) approach and the Ex-
π(θ ij |D, M h ) = Dir(θ ij |αij1 +Nij1 , ..., αijci +Nijci ) , pectation Maximization algorithm are traditionally
adopted in the literature (Gelman et al., 1995). A
where Nijk denotes the number of cases in D in new algorithm, which has been shown to determine
which Xi = xki and Pai = paji . The second ad- performance comparable with Gibbs sampling while
vantage is related to the flexibility of the Dirichlet reducing computational time, has been recently pro-
in modelling many different assumptions on the pa- posed in (Ramoni and Sebastiani, 1998; Sebastiani
rameters (the same flexibility of the beta distribu- and Ramoni, 2000). Hence, in situations where a
tion, which is the conjugate prior for binomial sam- large set of questionnaires have to be analyzed, this
pling). In particular, the lack of any prior infor- choice can be particularly indicated.
mation on the parameters entailed can be modelled According to the learning strategy adopted, the
using a Dirichlet distribution with equal hyperpa- final BN should be the one that has the maximum
rameters and precision set to 1. In this case, all the marginal likelihood among all the possible models
possible realizations of each variable given its par- (i.e., using the exhaustive enumeration of all the pos-
ents are a priori equally likely and, because of the sible models M h ). Unfortunately, given I variables,
prior precision set to 1, the prior acts as a ’head- the set of all the possible models is more than expo-
start’, suddenly forgotten when observations accu- nential in I, thus determining a NP-hard problem.
mulate. The marginal likelihood corresponding to Since the exhaustive enumeration is practically in-
the model represented though the DAG can be thus feasible, the problem is traditionally tackled in the
computed as: literature by heuristic algorithms. Among these al-
qi
I  ci
gorithms, the greedy search with random perturba-
 Γ(αij )  Γ(αijk + Nijk ) tion of the order of the variables considered is the
p(D|M h ) = ,
i=1 j=1
Γ(αij + Nij ) Γ(αijk ) one that has reported the best results, for a given
k=1
(2) computational time (Heckerman, 1998).
ci ci
where Nij = k=1 Nijk and αij = k=1 αijk .
Unfortunately, the computation of the likelihood, 4. A control chart for service quality
given by equation (2), is not straightforward when control
there are missing data. Since the questionnaires
completed by customers are likely to contain missing Once the phase of learning is ended, the BN obtained
data, this issue must be considered in order to pro- should represent the actual level of service quality
pose a realistic solution to the quality service prob- and it can be used to design a control chart on the
lem. When the set of data collected is not complete, overall satisfaction. The approach proposed is based
different assumptions can be made on the mecha- on the idea originally reported in (Box, 1980) and
nism that generates missing observations (Rubin et then extended for multinomial sampling in (Wood,

618
 Joint Statistical Meetings - Section on Quality & Productivity (Q&P)

1986) and (Spiegelhalter et al., 1994), of using the

predictive distribution to check an identified model.
The main advantage of this approach is that it does
not require the definition of any specific alternative
model (Box, 1980).
Denote with θi·k the unconditional probability of
obtaining the k th outcome for the ith variable (the
second sub-index refers to the unconditional com-
putation, i.e. considering all the possible configu-
rations paji , j = 1, ..., qi , of the parent variables).
Assuming the learning phase has been performed
with N cases. Using the BN derived, the vectors
θ i· = (θi·1 , ..., θi·ci ) can be estimated for all the nodes Figure 2: A control chart for service quality control
in the network. Under the null hypothesis that the
BN model derived is true, the checking function for
the ith variable can be computed as: 5. Conclusions and further develop-
ments
ci
α+1  (Ni·k − θ̂i·k N )2
gi = ∼ χ2ci −1 , (3) The problem of controlling service quality with time
α+N θ̂i·k N
k=1 has been tackled proposing a new approach based on
Bayesian Networks (BNs). Compared with models
where k = 1, ..., ci represents the possible outcome proposed in the literature, the adoption of BNs has
of the ith multinomial variable; α is the precision of shown the main advantage of considering the cate-
the Dirichlet distribution after the phase of learning gorical nature of the indicators of satisfaction and
(given by the prior precision plus the number of cases including the possibility of missing data.
used in the phase of learning); Ni·k is the number of
In order to completely exploit the potentiality of
outcomes equal to k observed in the set of N cases
the new approach, a first direction of future research
for the ith variable (the second subindex refers to the
concerns the use of BNs as a supporting tool for the
computation performed for any configuration paji ,
phase of questionnaires design. In fact, the BN de-
j = 1, ..., qi , of the parents of Xi ); χ2ci −1 denotes the
rived after learning can be used to identify indica-
Chi-Square distribution with ci − 1 degrees of free-
tors that are missing or redundant in the question-
dom. The reader interested in the analogy between
naire. Missing information can be detected when
the checking function gi and the traditional Pearson
the BN derived after the learning phase shows that
Chi-Square statistic can refer to (Wood, 1986).
just few variables are parents of the overall satis-
Assuming the overall satisfaction is represented faction, i.e., the overall satisfaction is almost in-
by the first variable X1 , the predictive function g1 , dependent from quality characteristics examined in
given by equation (3) for i = 1, can be used to design the questionnaire. In this case, the dimensions that
the control chart for the overall satisfaction. Assum- are poorly modelled should be better identified and
ing a first type error α , the upper control limit is questions concerning these features should be in-
given by: cluded in the questionnaire. On the other hand,
U CL = χ2c1 −1,α
, (4) the use of ’evidence propagation’ in BNs could be
useful in reducing the number of questions, through
where χ2c1 −1,α
is the value of the χ2c1 −1 such that the identification of redundant variables. In this
the cumulative distribution is equal to 1 − α . case evidence propagation can be used to analyze
Therefore each time a new set of data is collected, the way in which a change in an indicator of sat-
g1 can be computed and the value obtained can be isfaction (node of the network) impacts the overall
plotted against the limit defined by U CL, as rep- satisfaction (node 1 in the BN presented), in order
resented in Figure 2. If an out of control signal is to identify indicators that are slightly affecting the
detected, the same approach can be used for all the service quality and that can be therefore neglected
other nodes in the BN, using predictive functions gi in future analysis.
given by equation (3) for i = 2, ...I. Hence, the nodes Although the paper has been specifically focused
in the network (service quality characteristics) that on services, it could be useful for controlling qual-
are more responsible for the out of control signal can ity of many products in a near future. In fact,
be identified. products oriented to large-scale markets are more

619
 Joint Statistical Meetings - Section on Quality & Productivity (Q&P)

and more often technologically undistinguishable D.B. (1995) Bayesian Data Analysis, Chapman &
and their attractiveness is basically characterized by Hall/CRC.
non-tangible aspects (e.g., the esthetic appeal or the
comfort of cars in the automotive industry). These Heckerman, D. (1998) A Tutorial on Learning
quality characteristics have to be measured by di- with Bayesian Networks in Learning Graphical
rectly asking the customers to express a judgement models, edited by M.J. Jordan, Kluwer Academic
about their satisfaction, as for measuring service Publishers, 301-354.
quality.
Johnson, R.A. and Wichern, D.W. (1992) Applied
6. Acknowledgements Multivariate Statistical Analysis. Third edition,
Prentice-Hall.
The authors are grateful to E. del Castillo for the
suggestions that resulted in a clearer presentation of Juran, J.M. (2002) Quality Control in Service
the paper. Industries, available at http://www.juran.com/.

References Parasuraman, A., Zeithaml, V.A. and Berry, L.L.

Box, G.E.P. (1980) Sampling and Bayes’ Inference (1994) Reassessment of Expectations as a Com-
in Scientific Modelling and Robustness, Journal parison Standard in Measuring Service Quality:
of the Royal Statistical Society, Series A, 143(4), Implications for Further Research, Journal of
383-430. Marketing, 58, 111-124.

Colosimo, B.M., (2001) Service Quality Control Ramoni M., and Sebastiani, P. (1998) Parameter
using Bayesian Network, V Convegno A.I.TE.M., Estimation in Bayesian Networks from Incomplete
Bari (Italy), 18-20 September 2001. Databases, Intelligent Data Analysis, 2(1).

Cowell, R.G., Dawid, A. P., Lauritzen, S.L. and Rubin, D.B., Stern, H.S. and Vehovar, V. (1995)
Spiegelhalter, D.J. (1999) Probabilistic Networks Handling ’don’t know’ survey responses: the case
and Expert Systems, Springer-Verlag New York. of the slovenian plebiscite, Journal of the American
Statistical Association, 90, 822-828.
Cronin, J.J. and Taylor, S.A. (1992) Measuring
Service Quality: A reexamination and Extension, Schwarz, G. (1978) Estimating the dimension of a
Journal of Marketing, 56, 55-68. model, Annals of Statistics, 6, 461-464.

Cronin, J.J. and Taylor, S.A. (1994) SERVPERF Sebastiani, P. and Ramoni M., (2000) Bayesian
Versus SERVQUAL: Reconciling Performance- Inference with Missing Data Using Bound and
Based and Perceptions-Minus-Expectations Mea- Collapse, Journal of Computational and Graphical
surement of Service Quality, Journal of Marketing, Statistics,9(4).
58, 125-131.
Spiegelhalter D.J., Dawid, A. P., Lauritzen L. and
Dawid, A.P. (1984) Statistical Theory: The Pre- Cowell R.G. (1993) Bayesian Analysis in Expert
quential Approach, Journal of the Royal Statistical Systems, Statistical Science, 8(3).
Society, Series A, 147(2), 278-292.
Spiegelhalter D.J., Harris, N.L., Bull, K., Franklin,
Dabholkar, P.A., Shepherd, C. D. and Thorpe, D.I. R.C.G, (1994) Empirical Evaluation of Prior Beliefs
(2000) A Comprehensive Framework for Service About Frequencies: Methodology and a Case
Quality: An Investigation of Critical Conceptual Study in Congenital Heart Disease, Journal of the
and Measurement Issues Through a Longitudinal American Statistical Association, 89(426), 435-443.
Study, Journal of Retailing, 76(2), 139-173.
Wood, J.B., (1986) An Application of predictive
Franceschini, F. and Rossetto, S. (1999) Service Checks in the Bayesian Analysis of Multinomial
Qualimetrics: the QUALITOMETRO II method, Data, Statistician, 35(1), 65-72.
Quality Engineering, 12(1), 13-20.

Gelman, A., Carlin, J.B., Stern, H.S. and Rubin,

620