An Efficient Visual Representation and Indexing Scheme for Large Scale Video Retrieval

Abstract

This chapter presents technical advancements toward web scale video retrieval. Two methods for efficient visual representation and efficient visual indexing of web videos are discussed. First, a video representation named interest seam image is presented, which considers both spatial and temporal information contained in a video. Therefore it is more discriminative than previous video representations (such as those based on key-frames). Second, an indexing system is presented, which is capable of dealing with web-scale data. The system combines both local and global descriptors, and embeds geometric configuration information of interest points into the index to simultaneously improves retrieval accuracy, speed and memory footprint. The efficacy and efficiency of the proposed methods are demonstrated in large scale experiments on real web-videos.

Keywords: Video Representation, Efficient Indexing, Video Retrieval, Seam, MinHash, Temporal, Keyframe, global descriptor, GIST, geometric information