Food Calorie Measurement Using Deep

Learning Neural Network

Team Member:-
Shivani Saraf (RA1911004010644)
Ram Bagaria (RA1911004010629)
Aakanksh Singh (RA1911004010600)

Guide:
Dr. K. Harisudha
Assistant Professor, Department of ECE
SRMIST, Kattankulathur
 01 02 03
PROPOSED
Contents

Introduction LITERATURE SYSTEM

SURVEY

04
Introduction to
05 06
WSN CONCLUSION
Bibliography &
EXPERIMENTAL Future Scope
RESULTS
 • Obesity in adults and children is considered a global epidemic . The main cause of obesity is
a combination of excessive food consumption and lack of physical activities.
• Preliminary studies among adolescents suggest that innovative use of technology may
improve the accuracy of dietary information of young people .
Introduction
• There is overwhelming evidence that metabolic complications, which are caused by obesity,
increase the risks for developing adverse health consequences such as diabetes, blood
pressure, dyslipidaemia and hypertension. People in general understand the links between
diet and health.
• Empowering patients with an effective longterm solution requires novel mechanisms that
help them make permanent changes to their dietary quality and calorie intake.
• Mobile phone food record is important among both adolescents and adults.
• Accurate methods of dietary assessment will help strengthen the ability of researchers to
identify diet–disease and diet–gene relationships.
• Deep learning can be a powerful method to significantly increase the accuracy of food
classification and recognition.
• Experimental results of combination of different segmentation method such as color,
texture, graph-cut segmentation and deep learning neural network combination show a 99%
accuracy of food recognition in single food portions.
 LITERATURE SURVEY

• Some of the most common food intake measuring methods that have been
developed in the last few years.
• The main advantages and drawbacks of these methods, in order to
demonstrate the novelty and contribution of our proposed system.
• One of the first clinical works in this area is the 24- Hour Dietary Recall (24HR).
• This procedure lists the daily food intake using a special format for a period of
24 hours.
• Another method is the Food Frequency Questionnaire (FFQ), which uses an
external verification based on double labeled water and urinary nitrogen.
• FFQ focuses on describing dietary patterns or food habits, but not calorie
intake.
• Disadvantage of the 24HR and FFQ are: the delay of reporting the eaten food,
the underreporting of the size of food portions, relying on memory.
 AUTHOR PAPER OUTCOME

• M. Livingstone, P. Robson and a. • 24- Hour Dietary Recall (24HR). This procedure lists the daily food intake
J.Wallace, L. Bandini, A. Must, H. Cyr, S. using a special format for a period of 24
Anderson, J. Spadano and W. Dietz hours. Estimation of food portion size is
made using standardized cups and spoons.
The record of food amounts are converted
into nutrient intakes amounts using food
composition tables.

• W. Luo, H. Morrison, M. d. Groh, C. • Food Frequency Questionnaire (FFQ) FFQ focuses on describing dietary patterns
Waters, M. DesMeules, E. Jones- or food habits, but not calorie intake,
McLean, A.-M. Ugnat, S. Desjardins which uses an external verification based
and M. L. a. Y. Ma on double labeled water and urinary
nitrogen.

• Y. Kato, T. Suzuki, K. Kobayashi, Y. • A web application for an obesity The application acquires and registers data
Nakauchi, prevention system based on individual about diet, exercise, sleep, and fat mass,
lifestyle analysis by using a web application and health
information sensors

• T. Miyazaki, G.C. De Silva, K. Aizawa • Image-based Calorie Content Estimation The images in the dictionary are used for
for Dietary Assessment dietary assessment, but with only 6512
images, the accuracy of such approach is
low
• H. C. n Chen, W. Jia, Z. Li, Y. Sun, M. Sun, • 3D/2D model to-image registration for The food is segmented from the background
quantitative dietary assessment image using morphological operations while
the size of the food is estimated based on
user-selected 3D shape model.
A set of segmented objects partitioned into
• Fengqing Zhu, Marc Bosch, Nitin Khanna, • Multiple Hypotheses Image similar object classes based on their
Carol J. Boushey Segmentation and Classification With features, for solving the idea they have
Application to Dietary Assessment applied different segmentation method.
Second, automatic segmented regions were
classified using a multichannel feature
classification system.
PROPOSED SYSTEM
• The user captures the picture of the food.

• Our approach is to generate a pre-trained model file with the help of

Faster R-CNN network. This is performed by initially capturing a set of
images of one particular class and then labeling them with object
name-set.

• Algorithm implementing stochastic gradient descent.

• After classifying the food object the system then calculates the total
amount of food portion on the plate in order to estimate the calorie
value.
CNN (Convolution Neural Network)

• Computer vision is an interdisciplinary field that has been gaining

huge amounts of traction in the recent years.

• Another integral part of computer vision is object detection.

• The difference between object detection algorithms and classification

algorithms is that in detection algorithms, we try to draw a bounding
box around the object of interest to locate it within the image.

• Not necessarily draw just one bounding box in an object detection

case, there could be many bounding boxes representing different
objects of interest within the image.
• The major reason for not proceeding with CNN model as building a
standard convolutional network followed by a fully connected layer is
that, the length of the output layer is variable — not constant, this is
because the number of occurrences of the objects of interest is not
fixed.

• It cannot be implemented real time as it takes around 45 minutes to

train the model.
• Low Accuracy.
Faster R-CNN (Convolution Neural Network)
• In Faster R-CNN input image to the CNN to generate a
convolutional feature map.

• From the convolutional feature map, identified region of proposals

and warp them into squares and by using a RoI pooling layer,
reshape them into a fixed size so that it can be fed into a fully
connected layer.
• From the RoI feature vector, a softmax layer to predict the class of the
proposed region and also the offset values for the bounding box .

• The reason “Fast R-CNN” is faster than R-CNN is because you don’t
have to feed 2000 region proposals to the convolutional neural network
every time. Instead, the convolution operation is done only once per
image and a feature map is generated from it.

Before Threshold Before Threshold

(Threshold=0.6)
Input:

Output:
 EXPERIMENTAL RESULTS
• we have combined Graph cut segmentation and deep neural network.

• The combination of these two methods allow to improve the accuracy of food
classification and recognition.

• Before the implementation of the image recognition algorithm in the Android

application, the first step in our approach is to generate a pre-trained model file
with the help of CNN network.

• System could recognize the food portions very accurately in 3 seconds.

• The results of table I show that we have 99% accuracy in our single food
portions. Results shows the average calories are so close to the real one and
also the small range of standard error also shows the accuracy of the system.
 Conclusions and Future W o r k

• Empower the user by a convenient, intelligent and accurate system that helps them
become sensible about their calorie intake.
• We employed a rather unique combination of graph cut segmentation and deep
learning neural networks as a means of accurately classifying and recognizing food
items.
• The combination of those two methods provides a powerful instrument to attain a
100 % accuracy of food recognition in our system.
• The implementation of the virtualization approach of the application which allows
us to benefit from cloud based resources.
• Future work is to increase our database of images and use the approach presented
in this paper to test mixed food portions.
 [1] World Health Organization. (2011, October) Obesity Study. [Online]. http://www.who.int
/mediacentre/factsheets /fs311/en/index.html

[2] World Health Organization. (2012) World Health Statistics 2012. [Online].
http://www.who.int/gho/publications/world_health_statistics/2012/en/index.html
Bibliography
[3] Fengqing Zhu, Anand Mariappan, Carol J Boushey, Deb Kerr, Kyle D Lutes, David S Ebert, Edward J Delp, “Technology-
assisted dietary assessment”, International Society for Optics and Photonics, p.p. 681411-681420, 2008.

[4] Bethany L Daugherty, TusaRebecca E Schap, Reynolette Ettienne-Gittens, Fengqing M Zhu, Marc Bosch, Edward J Delp,
David S Ebert, Deborah A Kerr, Carol J Boushey, Novel “Technologies for Assessing Dietary Intake: Evaluating the Usability of
a Mobile Telephone Food Record Among Adults and Adolescents”, Published online 2012
.
[5] P.Pouladzadeh, S.Shirmohammadi, and R.Almaghrabi, “Measuring Calorie and Nutrition from Food Image”, IEEE
Transactions on Instrumentation & Measurement, Vol.63, No.8, p.p. 1947 – 1956, August 2014.

[6] P. Pouladzadeh, S. Shirmohammadi, A. Bakirov, and A. Bulut, Abdulsalam Yassine “Cloud-Based SVM for Food
Categorization”, Multimedia Tools and Applications, Springer, Vol. 74, Issue 14, pp. 5243-5260.

[7] P.Pouladzadeh, S.Shirmohammadi, A.Yassine, “Using Graph Cut Segmentation for Food Calorie Measurement”, IEEE
International Symposium on Medical Measurements and applications, p.p.1-6, Lisbon, June 2014.
 Future Scope
1. This can reduce Heart disease, obesity.

2. A person may also boost their mood and gain more energy by
maintaining a balanced diet.

3. Counting calories can help you loose weight by bringing

awareness to what you eat each day.