AUTOMATIC IDENTIFICATION OF ANIMALS IN THE WILD: A COMPARATIVE STUDY BETWEEN C-CAPSULE NETWORKS AND DEEP CONVOLUTIONAL NEURAL NETWORKS.

Author

Joel Kamdem TetoFollow
Ying Xie, Kennesaw State University

Date of Submission

Fall 11-27-2018

Degree Type

Thesis

Degree Name

Master of Science in Computer Science (MSCS)

Department

Computer Science

Committee Chair/First Advisor

Dr. Selena He

Track

Big Data

Chair

Dr. Dan Lo

Committee Member

Dr. Ying Xie

Committee Member

Dr. Sumit Chakravarty

Related Publications

[1] Abhishek Chauhan & Mukund Babu & Nischel Kandru, Sanket Lokegaonkar. Empirical Study on convergence of Capsule Networks with various hyperparameters.

[2] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton (2017). ImageNet Classification with Deep Convolutional Neural Network. Communications of the ACM.

[3] Amara Dinesh & R.Karthila & Latha Parameswara.Novel Deep Learning Model for Traffic Sign Detection Using Capsule Networks. https://arxiv.org/pdf/1805.04424.pdf

[4] Anonymous authors.Paper under double-blind review. Generalized Capsule Networks with Trainable Routing Procedure. ICLR 2019

[5] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens(ILSVRC 2015). Rethinking the Inception Architecture for Computer Vision.

[6] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, and Dumitru Erhan. (September 2014, ILSVRC14).Going deeper with convolutions.

[7] Deep Learning Tutorial Release 0.1, LISA Lab, University of Montreal, March 09, 2015.

[8] Edgar Xi, Selina Blind, Yang Jin(December 2017,arXiv) Capsule Network Performance on Complex Data.

[9] Geoffrey E. Hinton, Sara Sabour, Nicholas Frosst. (2015, NIPS). Dynamic Routing Between Capsules. In 31st Conference on Neural Information Processing Systems.

[10] Geoffrey Hinton & Sara Sabour & Nicholas Frost. (2018) Matrix Capsules with EM Routing. ICLR 2018

[11] Gering, D. (2002). Linear and nonlinear data dimensionality reduction. fulfilment of the Area Exam doctoral requirements.

[12] Gomez A, Diez G, Salazar A, Diaz A (2016) Animal identification in low-quality camera-trap images using very deep convolutional neural networks and confidence thresholds. 2016 International Symposium on Visual Computing (Springer, Cham, Switzerland), pp 747–756.

[13] http://tensorflow.com

[14] https://www.udacity.com, Deep Learning Nano-degree.

[15] James J. DiCarlo, Davide Zoccolan, and Nicole C. Rust (2012, NIH). How does the brain solve visual object recognition?

[16] Jan Eric & Lensen Matthias & Fey Pascal Libuschewski.(2018) Group Equivariant Capsule Networks. https://arxiv.org/pdf/1806.05086.pdf

[17] Jürgen Schimdhuber.( Technical Report IDSIA-03-14). Deep Learning in Neural Networks: An Overview.

[18] Karen Simonyan, Andrew Zisserman. (2017, ICLR). Very Deep Convolutional Networks for Large-Scale Image Recognition. In Conference Paper at ICLR 2015.

[19] Kuntal Kumar Pal & K. S. Sudeep (2016).Preprocessing for image classification by convolutional neural networks. 2016 IEEE International Conference On Recent Trends in Electronics, information & Communication Technology.

[20] MD. Rayed Bin Wahed. (2016) Comparative Analysis between Inception-V3 and other Learning Systems using Facial Expressions Detection. Department of computer Science & Engineering BRAC University.

[21] Michael A. Tabak, Mohammed S. Norouzzadeh, David W. Wolfson, Steven J. Sweeney. Kurt C. VerCauteren. Machine Learning to classify animal species in camera trap images: applications in ecology. [https://www.biorxiv.org/content/biorxiv/early/2018/06/13/346809.full.pdf]

[22] Misha. (February 2018, Medium.com). Capsule Networks for food Classification.

[23] Mohammed Sadegh Nourouzzadeh, Anh Nguyen, Margaret Kosmala, Ali Swanson, Meredith Palmer, Craig Packer, and Jeff Clune. Automatically identifying, counting and describing wild animal in camera trap images with deep learning. Communications of the ACM.

[24] Naturomics:https://github.com/naturomics/CapsNetTensorflow/blob/master/imgs/capsuleVSneuron.png

[25] Nguyen, Hung & Maclagan, Sarah & Nguyen, Tu & Nguyen, Thin & Flemons, Paul & Andrews, Kylie & Ritchie, Euan & Phung, Dinh. (2017). Animal Recognition and Identification with Deep Convolutional Neural Networks for Automated Wildlife Monitoring. 10.1109/DSAA.2017.31.

[26] Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol. 2010.Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion, Journal of Machine Learning Research 11 (2010) 3371-3408.

[27] Rinat Mukhometzianov and Juan Carrillo. (arXiv) CapsNet comparative performance evaluation for image classification.

[28] Rob Schapire. COS 511: Theoretical Machine Learning. Princeton.edu

[29] Saurabh Srivastava & Prerna Khurana & Vartika Tewari. Identifying Aggression and toxicity in Comments using Capsule Network.TCS research

[30] Sergey Karayev, Tobias Baumgartner, Mario Fritz, Trevor Darrel. (NDSEG). Timely Object recognition.

[31] Stanford Tutorial: http://ufldl.stanford.edu/wiki/index.php/Stacked_Autoencoders

[32] Tomas Iesmantas and Robertas Alzbutas.(arXiv) Convolutional Capsule Network for classification of breast cancer histology images.

[33] UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.

[34] Xuanxin Liu, Fu Xu, Yu Sun, Haiyan Zhang, Zhibo Chen (2018). Convolutional Recurrent Neural Networks for Observation-Centered Plant Identification. Journal of Electrical and Computer Engineering.

[35] Youngjoo Kim, Peng Wang, Yifei Zhu, and Lyudmila Mihaylova.(September, 2018) A Capsule Network for Traffic Speed Prediction in Complex Road Networks. Department of Automatic Control and Systems engineering, the university of Sheffield, United Kingdom

[36] Zhong-Qui Zhao, Peng Zheng, Shou-tao Xu, and Xindong Wu (March 2017, journal of Latex). Object Detection with Deep learning: A review.

Abstract

The evolution of machine learning and computer vision in technology has driven a lot of

improvements and innovation into several domains. We see it being applied for credit decisions, insurance quotes, malware detection, fraud detection, email composition, and any other area having enough information to allow the machine to learn patterns. Over the years the number of sensors, cameras, and cognitive pieces of equipment placed in the wilderness has been growing exponentially. However, the resources (human) to leverage these data into something meaningful are not improving at the same rate. For instance, a team of scientist volunteers took 8.4 years, 17000 hours at a rate of 40 hours/week to label 3.2 million images from the Serengeti wild park. For our research, we are going to focus on wild data and keep proving that deep learning can do better and faster than the human equivalent labor for the same task. Moreover, this is also an opportunity to present some custom Capsule Networks architectures to the deep learning community while solving the above-mentioned critical problem. Incidentally, we are going to take advantage of these data to make a comparative study on multiple deep learning models, specifically, VGG-net, RES-net, and a custom made Convolutional-Capsule Network. We benchmark our work with the Serengeti project where Mohammed Sadegh et al. recently published a 92% top-1 accuracy [23] and Gomez et al. had a 58% top-1 accuracy [12]. We successfully reached 96.4% top-1 accuracy on the same identification task. Concurrently, we reached up to 79.48% top-1 testing accuracy 33on a big, complex dataset using capsule network, which out-performed the best results of Capsule networks on a complex dataset from Edgar Xi et al. with 71% testing accuracy [8,33,27].