Semantic image annotation using convolutional neural network and wordnet ontology

  • Authors

    • Jaison Saji Chacko CHRIST (Deemed to be University)
    • Tulasi B CHRIST (Deemed to be University)
    2018-08-02
    https://doi.org/10.14419/ijet.v7i2.27.9886
  • Convolutional Neural Networks, Deep Learning, Image Annotation, Semantic Labeling, WordNet Ontology.

  • Abstract

    Images are a major source of content on the web. The increase in mobile phones and digital cameras have led to huge amount of non-textual data being generated which is mostly images. Accurate annotation is critical for efficient image search and retrieval. Semantic image annotation refers to adding meaningful meta-data to an image which can be used to infer additional knowledge from an image. It enables users to perform complex queries and retrieve accurate image results. This paper proposes an image annotation technique that uses deep learning and semantic labeling. A convolutional neural network is used to classify images and the predicted class labels are mapped to semantic concepts. The results shows that combining semantic class labeling with image classification can help in polishing the results and finding common concepts and themes.

  • References

    1. [1] Millers, George A“WordNet: a lexical database for English.†Communications of the ACM 38.11 (1995): 39-41. https://doi.org/10.1145/219717.219748.

      [2] Stamou, Giorgos, et al., “Multimedia annotations on the semantic web.†IEEE Multimedia 13.1 (2006): 86-90 https://doi.org/10.1109/MMUL.2006.15.

      [3] Little, Suzanne, Ovidio Salvetti, and Petra Perner. “Semi-automatic semantic annotation of images.†Seventh IEEE International Conference on Data Mining Workshops (ICDMW 2007). IEEE, 2007. https://doi.org/10.1109/ICDMW.2007.22.

      [4] Verma, Yashaswi, and C. V. Jawahar. “Image Annotation by Propagating Labels from Semantic Neighborhoods.†International Journal of Computer Vision (2016): 1-23.

      [5] Dureja, Aman, & Payal Pahwa. “Image retrieval techniques: a survey.†International Journal of Engineering & Technology [Online], 7.1.2 (2018): 215-219. Web. 7 May. 2018.

      [6] Berners-Lee, Tim, James Hendler, and Ora Lassila. “The semantic web.†Scientiï¬c american 284.5 (2001): 34-43. https://doi.org/10.1038/scientificamerican0501-34.

      [7] Reena Pagare and Anita Shinde, “A Study on Image Annotation Techniquesâ€, Harlow, England: International Journal of Computer applications, Volume 37- No.6, January 2012.

      [8] Luque, Edson F., Daniel L. Rubin, and Dilvan A. Moreira. “Automatic Classiï¬cation of Cancer Tumors using Image Annotations and Ontologies.†2015 IEEE 28th International Symposium on Computer-Based Medical Systems. IEEE, 2015.

      [9] Seifert, Sascha, et al., “Semantic annotation of medical images.†SPIE medical imaging. International Society for Optics and Photonics, 2010.

      [10] Rubin, D. L., Rodriguez, C., Shah, P., and Beaulieu, C. “Semantic Annotation and Markup of Radiological Images.†AMIA Annual Symposium Proceedings (2008), Volume 2008, p. 626.

      [11] O. Marques, N. Barman, “Semi-Automatic Semantic Annotation of Images Using Machine Learning Techniquesâ€, Proc. of ISWC, pp. 550565, 2003.

      [12] Carneiro, Gustavo, and Nuno Vasconcelos. “Formulating semantic image annotation as a supervised learning problem.†Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on. Vol. 2. IEEE, 2005.

      [13] Feng, Linan, and Bir Bhanu. “Semantic Concept Co-Occurrence Patterns for Image Annotation and Retrieval.†IEEE transactions on pattern analysis and machine intelligence 38.4 (2016): 785-799. https://doi.org/10.1109/TPAMI.2015.2469281.

      [14] LeCun, Yann, et al., “Gradient-based learning applied to document recognition.†Proceedings of the IEEE 86.11 (1998): 2278-2324. https://doi.org/10.1109/5.726791.

      [15] LeCun, Yann, Koray Kavukcuoglu, and Cl´ement Farabet. “Convolutional networks and applications in vision.†Circuits and Systems (ISCAS), Proceedings of 2010 IEEE International Symposium on. IEEE, 2010. https://doi.org/10.1109/ISCAS.2010.5537907.

      [16] Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. “Imagenet classiï¬cation with deep convolutional neural networks.†Advances in neural information processing systems. 2012.

      [17] Maxime Oquab, L´eon Bottou, Ivan Laptev, Josef Sivic. “Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks.†IEEE Conference on Computer Vision and Pattern Recognition, Jun 2014, Columbus, OH, United States. https://doi.org/10.1109/CVPR.2014.222.

      [18] Boutell, Matthew R., et al., “Learning multi-label scene classiï¬cation.†Pattern recognition 37.9 (2004): 1757-1771. https://doi.org/10.1016/j.patcog.2004.03.009.

      [19] Szegedy, Christian, et al., “Rethinking the inception architecture for computer vision.†Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.

      [20] Pan, Sinno Jialin, and Qiang Yang. “A survey on transfer learning.†IEEE Transactions on knowledge and data engineering 22.10 (2010): 1345-1359. https://doi.org/10.1109/TKDE.2009.191.

      [21] Sai V, Bhavya, Narasimha Rao G, Ramya M, Sujana Sree Y, & Anuradha T. “Classiï¬cation of skin cancer images using TensorFlow and inception v3.†International Journal of Engineering & Technology [Online], 7.2.7 (2018): 717-721. Web. 9 May. 2018.

  • Downloads

  • How to Cite

    Saji Chacko, J., & B, T. (2018). Semantic image annotation using convolutional neural network and wordnet ontology. International Journal of Engineering & Technology, 7(2.27), 56-60. https://doi.org/10.14419/ijet.v7i2.27.9886

    Received date: 2018-03-05

    Accepted date: 2018-05-17

    Published date: 2018-08-02