Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Assigning Topics to Documents by Successive Projections

Abstract : Topic models provide a useful tool to organize and understand the structure of large corpora of text documents, in particular, to discover hidden thematic structure. Clustering documents from big unstructured corpora into topics is an important task in various areas, such as image analysis, e-commerce, social networks, population genetics. A common approach to topic modeling is to associate each topic with a probability distribution on the dictionary of words and to consider each document as a mixture of topics. Since the number of topics is typically substantially smaller than the size of the corpus and of the dictionary, the methods of topic modeling can lead to a dramatic dimension reduction. In this paper, we study the problem of estimating topics distribution for each document in the given corpus, that is, we focus on the clustering aspect of the problem. We introduce an algorithm that we call Successive Projection Overlapping Clustering (SPOC) inspired by the Successive Projection Algorithm for separable matrix factorization. This algorithm is simple to implement and computationally fast. We establish theoretical guarantees on the performance of the SPOC algorithm, in particular, near matching minimax upper and lower bounds on its estimation risk. We also propose a new method that estimates the number of topics. We complement our theoretical results with a numerical study on synthetic and semi-synthetic data to analyze the performance of this new algorithm in practice. One of the conclusions is that the error of the algorithm grows at most logarithmically with the size of the dictionary, in contrast to what one observes for Latent Dirichlet Allocation.
Document type :
Preprints, Working Papers, ...
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03281066
Contributor : Suzanne Sigalla <>
Submitted on : Wednesday, July 7, 2021 - 8:47:10 PM
Last modification on : Tuesday, July 13, 2021 - 3:19:24 AM

Files

main.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03281066, version 1
  • ARXIV : 2107.03684

Collections

Citation

Olga Klopp, Maxim Panov, Suzanne Sigalla, Alexandre Tsybakov. Assigning Topics to Documents by Successive Projections. 2021. ⟨hal-03281066⟩

Share

Metrics

Record views

31

Files downloads

10