Quality Analysis of Object Categorization Benchmarks: Corel, Caltech, and IAPR Datasets

Chih-Fong      Tsai; Wei-Chao      Lin; Chihli      Hung

Abstract

Image retrieval aims at searching similar semantic contents of images to the users queries. However, the major challenge of image retrieval is to make computers understand high-level concepts of images, e.g. objects. In other words, automatic image annotation by object recognition and categorization can achieve semantic-based image retrieval. However, object categorization is a very important and challenging problem in computer vision. To evaluate some proposed object categorization algorithm, a pilot experiment is usually conducted based on a chosen small scale dataset in order to quickly obtain some useful results for later large scale experiments. Although there are a number of object categorization benchmarks, it is time consuming to consider all of them for the pilot experiment. This patent paper analyzes three wellknown benchmarks, which are Corel, IAPR, and Caltech, and tries to identify the levels of easiness and difficulty for image annotation. In addition, relevant patents are also reviewed in this manuscript. In the experiments, local region-based image features were extracted from these datasets and the Euclidean distances between images were analyzed in order to understand the distance distribution of images in the feature space. Moreover, image annotation accuracy using SVM and k-NN classifiers were examined. The experimental results showed that Caltech is the easiest dataset, which allows classifiers to provide the highest rate of annotation accuracy. On the other hand, IAPR is the toughest dataset and Corel stands at the middle position.

Keywords: Benchmarks, corel, caltech, IAPR, Image retrieval, image annotation, object categorization, Caltech Dataset, SVM and k-NN