Florida International University - University of Miami Trecvid 2019 Conference

Tao, Y, Wang, T, Machado, D et al. (2020). Florida International University - University of Miami Trecvid 2019 .

cited authors

  • Tao, Y; Wang, T; Machado, D; Garcia, R; Tu, Y; Reyes, MP; Chen, Y; Tian, H; Shyu, ML; Chen, SC

abstract

  • This paper presents the framework and results from the team “Florida International University-University of Miami (FIU-UM)” in the TRECVID 2019 Ad-hoc Video Search (AVS) [1] task. We submitted 7 fully automatic runs as follows. • run1: unweighted concept fusion + arithmetic mean + weighted W2VV score integration • run2: weighted concept fusion + geometric mean + W2VV score with threshold integration • run3: weighted concept fusion + geometric mean + weighted W2VV score integration • run4: weighted concept fusion + geometric mean • run5: unweighted concept fusion + geometric mean + W2VV score with threshold integration • run6: unweighted concept fusion + geometric mean • novelty run: weighted concept fusion + crawling concepts with description + geometric mean Our framework includes the following processing steps: (1) automatically parsing the query and generating a concept tree, (2) generation of CNN features from keyframes, (3) generation of concept scores from multiple pre-trained models for image classification, object, scene, and action detection, (4) just-in-time concept learning for keywords not found in the concept bank, (5) word to visual vector (W2VV) image-text matching scores, and (6) integration of the scores based on the concept tree. The performance results show that our fourth run (run4), which includes our best-weighted combination scores and geometric mean, outperforms all the other runs. This year, the FIU-UM team achieved the highest novelty score among all the teams. The submission details are listed as follows. • Class: F (fully automatic runs) • Training type: E (used only training data collected automatically, using only the official query textual description) • Team ID: FIU-UM (Florida International University - University of Miami) • Year: 2019.

publication date

  • January 1, 2020