12th International Conference on Information & Communication Technology and System (lCTS) 2019

Sentiment Analysis of Restaurant Customer

Reviews on TripAdvisor using Naive Bayes
Rachmawan Adi Laksono Kelly Rossa Sungkono Riyanarto Sarno
Depa rtment ofInformation Technology Depa rtment ofInfo rmatics Depa rtment ofInformatics
Management Institut Teknologi Sepuluh Nop ember Institut Teknologi Sepuluh Nopember
Institut Teknologi Sepuluh Nop ember Surabaya, Indonesia Surabaya, Indonesia
Surabaya, Indonesia [email protected] [email protected]
[email protected]

Cahyaningtyas Sekar Wahyuni

Department ofInformatics
Institut Teknologi Sepuluh Nopemb er
Surabaya, Indonesia
[email protected]

Abstract- Sentiment analysis is one method for classifying done with the help of Python, while for classifying data using
documents to identify positive or negative opinions. Customer Waikato Environment for Knowledge Analysis (WEKA)
satisfaction has an essential point for customer service. software with the Naive Bayes method and also using
Customer behaviour is currently doing a lot of reviews in online TextBlob which is a python-based sentiment analyzer to
media such as on trip advisor. A restaurant is a business that compare. Naive Bayes is chosen because this method has been
requires more attention in the service to consumers by widely implemented in sentiment analysis [9]-[11] .
improving service to customers continuously. This study tries to
classify Surabaya restaurant customer satisfaction using Naive This aim of this study is to analyze restaurant customer
Bayes. Data sampling is crawling by using WebHarvy Tools. reviews from online TripAdvisor in the best ten Surabaya
The result from this research shows that these two methods get restaurants. Another aim is to find the best method for
the customer response accurately and Naive Bayes method is analyzing restaurant customer review data by comparing
more accurate than TextBlob sentiment analysis with a different Naive Bayes method and TextBlob [12], [13] sentiment
accuracy of 2.9%. analysis since the two methods have fundamental differences
in terms of calculations.
Keywords- customer satisfaction, Naive Bayes,
sentiment analysis, textBlob II . LITERATURE REVIEW
Customer satisfaction is an essential concern in the field
I. INTRODUCTION
of marketing and research in terms of consumer behaviour.
Customer satisfaction is an opinion or feeling between As in the habits of hotel consumers when they get excellent
expectation and reality obtained by consumers [1]. Today, service, they will transmit to others mouth to mouth [14] .
many customers write opinions in the form of reviews about
their obtained satisfaction on online media, such as Text mining or retrieval of data from a collection of
TripAdvisor. Customer reviews on the online media become documents stores frequently with the help of analysis tools or
important as it might increase the popularity of the product or manuals [13] . Through the analysis process of several text
service sold by the seller. mining perspectives, information can be produced that can be
used to increase profits and services.
A restaurant is a business [2], [3] that prepares and serves
food for customers and exchanges for a certain amount of Sentiment analysis is used to find opinions from the
money. Although improving quality through this method is author about a specified entity [15]. Sentiment analysis of a
considered useful, only some restaurants use customer review is an opinion inve stigation ofa product [16]. The basis
satisfaction analysis to improve their services. Also, many of sentiment analysis is using Natural Language Processing
algorithms might be used for doing the study [4], [5]. (NLP), text analysis and some computational portions to
extract or omit unnecessary parts to see the pattern of the
Restaurant customer satisfaction research through reviews sentence negative or positive [17].
on online media such as TripAdvisor is still rare. Generally,
restaurant customer satisfaction analyses through product In the l Sth century, Reverend Thomas Bayes developed
data, nutrition data and food preparation [6]. One of the a method known as Naive Bayes that used probability and
previous studies is an analysis of hotel customer satisfaction opportunity approaches. The workings of the Naive Bayes
in Ponorogo district. The study used hotel customer review algorithm can be seen in Equation (1). Naive Bayes calculates
data on TripAdvisor [7]. future probability predictions from data or experiences that
have been given, based on the opportunity point of view [18].
Restaurant reviews on TripAdvisor are still in the form of One characteristic of the Naive Bayes Classification is the
text, customer reviews are included in the text mining existence of independent input variables which assume the
category, the results of these data will be classified into two presence of an articular feature from a class that is mutually
values, positive or negative [8]. Retrieving data on independent of other features [19] .
TripAdvisor using WebHarvy software, for preprocessing
review data such as remove stopword, remove punctuation is

978-1-7281-2133-8/19/$31.00 ©2019 IEEE 49

 As a comparison, the results of this study are also
(1) compared to the TextBlob sentiment analysis, a sentiment
analyzer that has a Natural Language Toolkit (NLTK) and
where: Pattern processing basis [21]. TextBlob can also be used for
P(HJ Ix) = states the probability arises Hjif known x. text mining, text processing modules for python winners, and
even text analysis. TextBlob also provides simple APIs for
P(xIHj) = The likelihood function of H, to x general Natural Language Processing (NLP) processing such
P(H j) = Prior probability as part-of-speech tagging, tokenizing sentences, noun phrase
P(x) = evidence extraction, sentiment analysis, classification, translation [22] .
WEKA, is one ofthe workbenches that has many choices
Methods of precision, recall, and accuracy is used to of machine learning methods for research. Initiated since
check the accuracy of the results of the process. A confusion 1992, the use of WEKA allows users to try and compare
matrix is created to provide performance classification data. several different machine learning methods on the new data
Elements of confusion matrix in Fig . 1 are True positive (TP) set quickly, and there are several Algorithms available in
when both human and method predict are positive and True several languages [23]. WEKA has been very accepted in
Negative (TN) when both human and method predict are various academics, the business environment, to be used as a
negative. False negative (FN) is used when the human data mining research tool. The analysis is working on
prediction is positive while method prediction is negative and environment using WEKA 3.8.2.
False Positive (FP) is used when the human prediction is One ofits abilities is the classification method either with
negative while method prediction is positive [20] . supervision or no supervision. Some popular classifications
The level of accuracy between what the user wants and are the Naive Bayes method, Decision tree J48, Random
the results of the system process is called Precision, can be Forest, k-Nearest Neighbors (k-NN), Sequential Minimal
seen in Equation (2), whereas Recall is the average success Optimization (SMO) or also called the Support Vector
of the system in the process of finding information, listed in Machine (SVM) method..
Equation (3), Precision and Recall calculations are used to
avoid measurement errors for deviation values as shown in III. RESEARCH METHOD
Equation (4) . Accuracy is the degree of truth between the This paper takes restaurant review data on TripAdvisor,
predictive value, and the actual value is shown in Equation especially restaurants in Surabaya by web crawling method,
(5) . Precision value is obtained by dividing TP with the and will be analyzed using the Naive Bayes method on WEKA
positive results obtained by the method. Because this paper and a comparison with the TextBlob sentiment analysis. There
analyzes customer review, so the positive results are the are several steps of the research method.
reviews classifying as positive reviews. Recall value is
A. Data collection
obtained by dividing TP with positive results based on the
real data . F Score value uses the Recall value and Precision Retrieval of customer review data is done by crawling by
value. The accuracy is the division of the amount of TP and using the WebHarvy tool as in the methods Fig. 2. Data is
TN to the amount of data. taken from web TripAdvisor, as in Fig. 3. TripAdvisor is the
largest travel community review site on the web . This web
Precision = TP
(TP+FP)
(2) was introduced in the year of2000, and now covers 212.000
hotels, more than 30.000 destinations, and 74.000 attractions
Recall =~ (3) around the world. Fig . 3 shows an overview of reviews from
(TP+ FN )
TripAdvisor.
F Score = (2xPrecision x Recall) (4)
(Precision +Recall)
In Fig . 3, there are 2 data shown, in the red box is the title
ofthe review, while the green box is a review ofthe customer.
Accurac = (TP+TN) (5) This paper will use customer review data as a data source.
Y (TP+TN+FP+FN)

where: TP = the positive review of the real data is classified

as the positive review obtained by the method Web
TN = the negative review ofthe real data is classified Data Mining : Data
TripAdvisor
Crawling Review
as the negative result obtained by the method Restaurant
Data
FP = the negative review ofthe real data is classified Review
as the positive review obtained by the method
FN = the positive review of the real data is classified Fig. 2. Data mining process
as the negative review obtained by the method
B. Data processing and analysis
Fig. 4 shows the work steps on processing and analysis
Real data data . On the processing data and analysis, there are several
Positive Review Nezatlve Review
steps that the procedure must be followed. The Trim
Positive
Review
TP FP lowercase process is done to change all the letters in small
Methods bricks not mixed large and small so that the uniforms are like
Negative
Review
FN TN this: "Good", "NoiSy" becomes "noisy" good.

Fig. I . Confusion matrix

50
 Overview Rev ieVJs Location O& A Details

r----------
~ R e vi e w e d 1 we ek ago

~ !?~J is;i~ L:! ~ __:

0 v ia mobile

~ i'ts- gura-rri'..-..-ba karl's-tiighly- rs co-ni"rri'..-nd ed! Ayam i)akai" is anotfier- - -:

IW ll k lVlI 1
I recomendat ion. E a t with so rnbct p e n c it a n d kan gkung h o tpla t e I
L J
Surabaya ,
Indonesia Date of v isit: December 20 18
f'i 4 3 •• 1 2
.~ T h a n k iw il k iw il

~ '~Hv iewe (j 1 week ~g(}

Ni ce
F o o d was delicious . Service was good. Not so expe nsive . You must t ry
th ..ir "Aya m Bakar" Servi ng t im.. quite fast. Overa ll ok >
Yulic HV'1
S urabaya,
Indon esi a Date o f v isit: November 20 18
[3 3
.~ T h 'lnk YulleH'N

Fig. 3. TripAdvisor Web

Stopword removal is a process of removing words that

often appear but do not have meaning in languages, such as
Data Mi ning: Data review "the", "a", "an", "in". Like the examp le below: "let me preface
Web TripAdvisor this review," said gardeners" became" let preface review,
Restaurant rev iew saying, mad keen gardeners ".
The remove punctuation process is the process of
removing punctuation that often appears and usually does not
have much meaning like "-, /,:,? " After the preprocessing text
is complete, the next process will be text calcification using
Preprocessing:
the Naive Bayes method and the comparison using Blob Text.
-Trim lowercase
- -Remove IV. RESULT
punctuation
Crawling techniques take the process of receiving
-Stopword removal
customer review data on Tripadvisor, especially restaurants
in Surabaya. The data taken is data on restaurant names and
reviews. The results of the data collection obtain 337 data,
269 data are used for training data and 68 data for testing data .
~ ~ Some data resu lts from the data collection process as shown
in TABLE I. There are three kinds of information in TABLE
Clasification Clas ification Pro cess I, i.e. several reviews (No.), the name of the restaurant, and
Process: : the reviews. There are only ten reviews that are shown in
-Naive Bayes -TextBlob Sentiment TABLE I.
clasification Ana lyzer
The next step is to prepare the review data for training

t t and testing data. Trim lowercase or make the letters uniform

into lowercase letters. TABLE II are examples of trim
Data Data lowercase processing resu lts. As seen in the first review of
Clasificat ion : Clasification : TAB LE II, the sentence "About the steak, well, the taste was
TextBlob nice" has been processed into "about the steak, well, the taste
Naive Bayes
was nice".
The next preprocessing is shown in Fig. 5. Fig. 5 explains
t the process of removing punctuation and words that often
appear in the text but have no meaning. As in Fig. 5, an
Compare Result : examp le of the results of eliminating punctuation and
-Naive Bayes & removing stop words.
TextBlob

Fig. 4. Data analysis process

51
 TABLE!. PART OF TH E RESULT OF D ATA COLLECTION
Time taken to Build mod el: 0.27 seconds
Correctly Clas sified 237 88.1041 %
No. Restaurant Name Reviews
Incorectly Classified 32 11.8959 %
About the steak, well, the taste
I Steak Hut Manyar Kertoarjo === Confusion Matrix ===
was nice
a b <-- class ified as
2 Steak Hut Manyar Kertoarjo The price is standard
63 141 a=-1
I think, they need more
3 Steak Hut Manyar Kertoarjo
innovation for their menu
181741 b = 1
4 Steak Hut Manyar Kertoarjo Abo ut the place, felt comfortable
Fig. 6. Training data and resu lt
overall, worth to try this
5 Steak Hut Manyar Kertoarjo
restaurant
TABLE III. TH E CLASSIFICATION OF TESTING DATA BY USING NAIvE
6 Steak Hut Manyar Kertoarjo best luck BAYES AND TEXT BLOB

Affordable price, good ambience,

7 Steak Hut Manyar Kertoarjo
recommended for family event No. Review NB TB EX

So many promotion from any 1. steak well taste nice

8 Steak Hut Manyar Kertoarjo I I I
credit card
Great service & affordable price
9 Steak Hut Manyar Kertoarjo 2. place felt comfortable I I I
for steak
Usually I visited this restaurant
10 Steak Hut Manyar Kertoarjo 3. overall worth try restaurant I I I
once in a month

4. best luck -I * I I

TABLE II. TRlM LoWERCAST RESULT affordable price good ambience

5. I I I
recommended fam ily event
No . Before Trim Lowercase After Trim Lowercase
6. many promotion cred it card -I * I I
About the steak, well, the about the steak, we ll, the taste
I
taste was nice was nice 7. great service affordable price steak I I I
2 The price is standart the price is standart
8. usually visited restaurant month I -I * I
I think, they need more i think , they need more
3
innovation for their menu innovation for their menu recommended steak restaurant
9. I I I
surabaya
Abo ut the place, felt
4 about the place , felt comfortable
comfortable
10. crowded I I I
overall, worth to try this overall, worth to try this
5
restauran t restaurant I!. cozy place decen t food I I I

6 best luck best luck 12. nice environment atmosphere I I I

Affordable price , good
affordable price, good ambience, 13. nice mea l larger portion perfect I I I
7 ambience, recommended for
recommended for family event
family event
14. cozy place good service I I I
So many promotion from any so many promotion from any
8
credit card credit card 15. good food perfect salad I I I
Great service & affordable great service & affordable price
9 16. chicken cordon blue delicious I I I
price for steak for steak

Usually I visited this usually i visited this restaurant 17. love I I I

10
restaurant once in a month once in a month
18. thumbs up -I * I I

19. lunch fami ly surabaya I I I

"nice and clean restaurant, good service. good food,
good taste good service steak hut
the best is garlic steam fish.. so yummy... price also 20 .
manyar ketoarjo
-I * I I
ok"
21. recommended
g~
I I I
"nice clean restaurant service good food best
garlic steam fish yummy price ok" 22 . great lunc h family I I I

23 . nice steak hut salad nz sirloi n steak -I * I I

Fig. 5. Remove stopwords proc ess
24. come next -I * I I

convenience environment great taste

25 . I I I
steak

also love burger new mozarella

26. I I I
chicken schni tzel

52
 thanks good service delicious beef 56. you starving dont go there -1 1* -1
27 . -1* 1 1
steak i will back here
quite take time especially restaurant
57. -1 1* -1
28 . thanks lot -I * 1 I crowded

steak hut restaurant specialties steak 58. lobster expensive -1 -1 -1

29 . -1* 1 1
menu
59. large restaurant crowded local foreign 1* 1* -1
usually promotion price cooperation
30. 1 -1* 1
credit card certain bank 60. prepare serve late menu -1 -1 -1

good deliciou s food clean spacious 6 1. always crowded -1 1* -1

31. place great large groups good friendl y 1 1 1
service
thing parking fee flat rate longer
62. -1 1* -1
expen sive ticket is
32. good food good price comfort place 1 1 1
63. place always crowd dinner time 1* 1* -1
33. must visit indonesi an food surabaya 1 1 1
64. variety foods many nothing special 1* 1* -1
34. grilled fish shrimp etc. I I I
65. enjoyed ice cream solace -I 1* -I
35. think need innovation menu -1* 1 1
66. The food terribly overrated 1* -1 -I
36. crowded visit weekend 1* 1* -I
The chicken dry tough marinad e taste
area nonsmoking smoking small 67. -1 -1 -I
37. 1* -I -I like kecap manis
space
The pric e expensive quality poor
come thought small restaurant totally 68. -I -I -I
38 -1 -1 -1 rating
wrong
where: NB : Naive Bayes
long time tried dinner fish bit small
tempe also small portion nasi goreng TB : TextBlob
39. 1* -I -I
merah amaz ed us super tiny bit
EX : Expert
portion small bowl its

food price expensive quality average TABL E IV. C ONFUSION M ATRIX OF N AiVE B AYES AND T EXTBLOB
40 . -1 -1 -1
portion small Naive Bayes TextBlob

41. sorry coming back again -1 -1 -1 TP FP 25 10 34 19

42 . disappointed female staff -1 -1 -1 FN TN 9 24 2 13

want ask spoon serve food without

43 . -1 1* -1
spoon take food and one coming
After going through preprocessing the data, a model was
44 . and response -1 1* -1 made with training data using WEKA 3.8.2 with Naive Bayes
classifiers. The model data is stored and will be used for
look back and yell louder voice
45 .
tunggu sebentar
-1 1* -1 evaluation with other data or testing data, Fig. 6 is the result
of the data model. As seen in Fig. 6, this research obtains 63
46. end she is coming table all -1 1* -1 TN and 174 TP by using Naive Bayes . The total of reviews
that are correctly classified is 237 reviews .
and sometime ask staff take food
47 . -1 -1 -1
without spoon food After training data with the Naive Bayes method, then re-
evaluate the model with testing data. The testing data was also
know staff mak es bad experience tested in the TextBlob Sentiment analyzer for comparison of
48. -1 -1 -1
coming here
accuracy with the Naive Bayes method. The following is the
49 . ac working well here -1 1* -1 result of testing data with the Naive Bayes and TextBlob
models that have been implemented written as in TABLE III.
50. not cold -1 1* -1 TABLE III shows 68 testing data. Each review will be
classified by using Naive Baye s as the method of this
little bit expensive crabs prices beside 1* research, TextBlob as the compari son method, and by an
5 1. park ing area difficult imagine raining 1* -1
season expert. 1 is denoted the positive review and -1 is denoted as
the negati ve review. The sign (*) means the result is different
52. noisy lots loud people here -1 1* -1 from the expert judgment.
53. pri vacy dont come romantic dinner -1 1* -1 All the results obtained by Naive Bayes and Textblob will
be analyzed by using confusion matrix. TABLE IV shows the
54.
good seafood want go prep ared noisy
1* 1* -1 confusion matrix of those two sentiment analysis method.
evening Based on TABLE IV, Naive Bayes obtains 25 TP and 24 TN.
55. restaurant quite big crowded 1* 1* -1
Then, TextBlob collects 34 TP and 13 TN. Naive Bayes has
a higher FN number than TextBlob.

53
 Conversely, Naive Bayes has a lower FP number than Comment Titles in Booking. corn Using Probabilistic Latent
TextBlob. The accuracy of the confusion matrix listed in Semantic Analys is," in 2018 6th International Conference on
Information and Communication Technology (lCoICT), 2018, pp.
TABLE IV is processed by using Equation (5). The accuracy 514-519.
of Naive Bayes is shown in Equation (6), and the accuracy of
TextBlob is shown in Equation (7) . The results of the [8] D. Grabner, M. Zanker, G. Fliedl, M. Fuchs, and others,
calculation of the above formula for the Naive Bayes method Classification of customer reviews based on sentiment analysis.
Citcsccr, 2012 .
have data accuracy of 72.06%, whereas by using the
TextBlob analyzer obtained an accuracy of 69.12%. From [9] B. Y. Pratama and R. Sarno, "Personality classification based on
these results, the Naive Bayes method using WEKA tools has Twitter text using Na ive Bayes, KNN and SYM ," in Data and
Software Engineering (ICoDSE), 2015 International Conference
a higher percentage of accuracy than the TextBlob sentiment on,2015,pp.170-174.
analysis.
[10) F. S. Fitri, M . N. S. Si, and C. Setianingsih, "Sentiment Analysis
Accuracy Naive Bayes = 72.06% (6) on the Level of Customer Satisfaction to Data Cellular Services

Accuracy TextBlob = 69.12% (7)

Using the Naive Bayes Class ifier Algorithm," in 2018 IEEE
International Conference on Internet of Things and Intelligence
System (IOTAIS), 2018, pp . 201--206.
[11) U. W. Wijayanto and R. Sarno, "An Experimental Study of
V. CONCLUSION Supervised Sentiment Analysis Using Gaussian Naive Bayes," in
2018 International Seminar on Application for Technology of
Based on the result of this paper that done on restaurant Information and Communication , 2018, pp. 476--481.
customer reviews in Surabaya, customer satisfaction analysis
might be used by the Naive Bayes classification method and [12] H. Gauba, P. Kumar, P. P. Roy, P. Singh, D. P. Dogra, and B.
Raman, "Prediction of advert isement preference by fusing EEG
TextBlob sentiment analysis that might be able to learn response and sentiment analysis," Neural Networks, vol. 92, pp .
sentiment from customers, since customer satisfaction is 77--88,201 7.
essential in terms of the restaurant business. The results also
[13] A. Hasan, S. Moin , A. Karim , and S. Shamshirband, "Machine
indicate that the Naive Bayes method has a value of 72.06% learning-based sentiment analysis for twitter accounts,"
accuracy and is slightly better (2.94%) than TextBlob Mathematical and Computational Applications, vol. 23, no . I, p.
sentiment analysis. Further research can be done by 11,2018.
increasing the number and variety of review data, or by other
[14] K Berezina, A. Bilgihan, C. Cobanoglu, and F. Okurnu s,
methods, to increase the value of accuracy. "Understanding satisfied and dissatisfi ed hotel customers: text
mining of online hot el review s," Journal ofHospitality Marketing
ACKNOWLEDGMENT & Management, vol. 25, no. I, pp . 1-24,2016.
The authors would like to thank to Institut Teknologi [15] R. Feldman, "Techniques and applications for sentiment analysis ,"
Sepuluh Nopember, The Ministry of Research, Technology Communications ofthe ACM, vol. 56, no. 4, pp . 82-89,2013.
and Higher Education of Indonesia, Direktorat Riset dan
[16] E. Haddi, X. Liu, and Y. Shi, "The role of text pre-processing in
Pengabdian Masyarakat, and Direktorat Jenderal Penguatan sentiment analysis," Procedia Computer Science, vol. 17, pp. 26-
Riset dan Pengembangan Kementerian Riset, Teknologi, dan 32,2013 .
Pendidikan Tinggi Republik Indonesia for supporting the
[17] R.-E. Fan, K-W. Chang, C.-J. Hsieh , X.-R . Wang, and C.-J. Lin,
research
"LIBLINEAR: A library for large linear classific ation," Journal of
machine learning research, vol. 9, no. Aug, pp. 1871-1874,2008.
[18] M. Jupri and R. Sarno , "Taxpayer compliance classification using
REFERENCES C4. 5, SYM, KNN, Naive Bayes and MLP," in 2018 International
[I) K Ilieska, "Customer satisfaction index--as a base for strategic Conference on Information and Communications Technology
marketing management," TEM Journal, vol. 2, no . 4, p . 327, 2013 . (ICOIACT), 2018, pp. 297-303 .

[2] Y. Caesarita and K R. Sungkono, "Identifying Bottlenecks and [19) T. Pranckevicius and V. Marcinkevicius, "Comparison of naive
Fraud of Business Process using Alpha ++ and Heuristic Miner bayes, random forest, decision tree, support vector machines, and
Algorithms (Case study : CV . Wicaksana Artha )," pp . 143-148, logistic regre ssion classifiers for text reviews classification,"
2017 . Baltic Journal ofModern Computing, vol. 5, no . 2, pp. 221-232,
2017 .
[3) R. Sarno and K R. Sungkono, "Coupled Hidden Markov Model
for Process Disco very of Non-Free Choice and Invisible Prime [20] D. L. Gupta , A. K Malv iya, and S. Singh , "Performance analys is
Tasks," Procedia Computer Science, vol. 124, pp. 134-141,2018. of classification tree learning algorithms," International Journal of
http ://doi.org/IO.lOI6/j.procs.2017.12.139. Computer Applications, vol. 55, no. 6,2012.

[4] M . Fikri and R. Sarno, "A Comparative Study of Sentiment [21] B. Agarwal, N . Mittal , P. Bansal, and S. Garg, "Sentiment analysis
Analysis using SVM and SentiWordNet," Indonesian Journal of using common-sense and context information," Computational
Electrical Engineering and Computer Science (lJEECS), vol. 13, intelligence and neuroscience, vol. 2015 , p. 30, 2015 .
no. 3,2019.
[22] S. Vijayarani, R. Janani , and others, "Text mining: open source
[5] B. Rintyarna, R. Sarno, and C. Fatichah, "Enhancing the tokenization tools-an analysis, " Advanc ed Computational
performance of sentiment analysis task on product reviews by Intelligence : An International Journal (ACI/) , vol. 3, no. I, pp . 37-
handling both local and global context," International Journal of 47,2016.
Information and Decision Sciences, vol. 11,2018.
[23] M. Hall, E. Frank, G. Holmes, B . Pfahringer, P. Reutemann, and 1.
[6] J. E. Mills and L. Thomas, "Assessing customer expectations of H. Witten, "The WEKA data mining software : an update," ACM
information provided on restaurant menus: A confirmatory factor SIGKDD explorations newsletter, vol. II , no. I, pp. 10-18,2009.
analysis approach," Journal of Hospitality & Tourism Research ,
vol. 32, no . I, pp . 62-88,2008.
[7] D. A. K Khotimah and R . Sarno, "Sentiment Detection of

54