R2DCLT: Retrieving relevant documents using cosine similarity and LDA in text mining
Proceedings Paper
Ramya, RS, Singh, G, Sejal, SN et al. (2021). R2DCLT: Retrieving relevant documents using cosine similarity and LDA in text mining
. 19(4), 391-422. 10.1504/IJICT.2021.118576
Ramya, RS, Singh, G, Sejal, SN et al. (2021). R2DCLT: Retrieving relevant documents using cosine similarity and LDA in text mining
. 19(4), 391-422. 10.1504/IJICT.2021.118576
The availability of digital documents over web has increased exponentially and hence there is a need for effective methods to retrieve and organise. Since data is dispersed globally and unorganised, a number of algorithms have been proposed based on relevance calculations. However, it is found that there is a gap between user’s search intention and retrieved results. In this paper, we propose a framework for retrieving relevant documents using cosine similarity (CS) and LDA in text mining (R2DCLT). The uniqueness of this approach is that LDA is applied for the documents and extracted patterns like unigram, bigram and trigram. Documents are ranked based on the CS score. Experiments are conducted on Reuters Corpus volume and custom news dataset. It is observed that R2DCLT outperforms pattern taxonomy and relevance feature discovery models by providing high quality relevant documents with improved response time and dynamically updated document set.
