This paper demonstrates the framework and results from the team “Florida International University - University of Miami (FIU-UM)” in TRECVID 2013 Semantic Indexing (SIN) task [14, 15]. Four runs were submitted, and the summary of these four runs are as follows: • M A FIU-UM-1 1: SVM - Support vector machine (SVM) based ranking using key frame (KF) features. • M A FIU-UM-2 2: SVM+DASD - SVM based ranking using KF features, domain adaptive semantic diffusion (DASD) is applied to refine results of this round. • M A FIU-UM-3 3: SVM+AAN - SVM based ranking using KF features, concept association network (CAN) is applied to refine results of this round. • M A FIU-UM-4 4: Fusion of the results generated from three models corresponding to the aforementioned three runs. In addition, the results of deformable part model for some concepts are also integrated.