Elamur Research⌕
☰
home / papers / 2026 / CobwebTM: Probabilistic Concept Formation for Lifelong and Hierarchical Topic Modeling

CobwebTM: Probabilistic Concept Formation for Lifelong and Hierarchical Topic Modeling

Paper · 2026 · Karthik Singaravadivelan, Anant Gupta, Zekun Wang et al.

Summary

Topic models usually involve a trade-off. LDA-style probabilistic models need a fixed topic count and bag-of-words input. Neural models are heavily parameterized, trained in batch, and tend to forget earlier topics as new data streams in. Hierarchies are also often added only after flat clustering. CobwebTM addresses lifelong and hierarchical topic discovery together. It adapts Cobweb, a classic incremental concept-formation algorithm, to pretrained transformer document embeddings. Each tree node stores a diagonal Gaussian over the embedding dimensions. Each new document travels down the tree, and at every level the algorithm chooses an operation by maximizing an entropy-based category utility. The depth and width of the hierarchy therefore come from the data rather than a preset K. Topic words for each node are then extracted with c-TF-IDF over the documents beneath it. On several corpora, including 20 Newsgroups and AG News, the authors report coherence and hierarchy quality that match or exceed neural and clustering baselines, along with temporally stable topics. The work suggests that a lightweight symbolic method built on strong embeddings is a practical choice for evolving text collections.

Key points

  • Node statistics: Nodes keep a mean and a variance vector that are updated online. Category utility measures how much splitting a parent into its children reduces Gaussian differential entropy.
  • Restructuring operators: At each node, four operators are compared: insert into the best child, create a new singleton child, merge the two best children, or split the best child.
  • Baselines: CobwebTM is compared against Online LDA, BERTopic (with KMeans or HDBSCAN), incremental clusterers such as DBStream and Mini-Batch KMeans, and neural hierarchical models such as BoxTM and TraCo.
  • Qualitative findings: On 20 Newsgroups, CobwebTM keeps ambiguous terms apart, for example computing "drive" versus automotive senses. The authors credit this to per-dimension variance acting as feature weighting.
  • Limitations and code: Results are sensitive to document arrival order, depend on the quality of the encoder, and memory grows with the tree. Code is available at github.com/Teachable-AI-Lab/cobweb-language-embedding.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Georgia Institute of TechnologyAtlanta · United States133 works · 24,497 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksGeneral Social Sciences · 227 works

💡 Concepts 1Key ideas and methods explained on our concept pages.Topic modeling · 47 works

Cite

Cite as Karthik Singaravadivelan, Anant Gupta, Zekun Wang and Christopher J. MacLellan. 2026. CobwebTM: Probabilistic Concept Formation for Lifelong and Hierarchical Topic Modeling. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1753

@inproceedings{singaravadivelan2026cobwebtm,
  title = {{CobwebTM: Probabilistic Concept Formation for Lifelong and Hierarchical Topic Modeling}},
  author = {Karthik Singaravadivelan, Anant Gupta, Zekun Wang and Christopher J. MacLellan},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1753},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1753}
}

References

37 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Derek Greene, Derek O’Callaghan, and Pádraig Cunningham. 2014. How many topics? stability analysis for topic models. In Machine Learning and Knowledge Discovery in Databases, pages 498–513, Berlin, Heidelberg. Springer Berlin Heidelberg. find
  2. Nicki Barari, Xin Lian, and Christopher J MacLellan. 2024a. Avoiding catastrophic forgetting in visual classification using human concept formation. CoRR. find
  3. Maarten Grootendorst. 2022. Bertopic: Neural topic modeling with a class-based tf-idf procedure. arXiv preprint arXiv:2203.05794.
  4. Nicki Barari, Xin Lian, and Christopher J. MacLellan. 2024b. Incremental concept formation over visual images without catastrophic forgetting. Advances in Cognitive Systems. find
  5. Anant Gupta, Karthik Singaravadivelan, and Zekun Wang. 2025. Hierarchical semantic retrieval with cobweb. Preprint, arXiv:2510.02539. — [arXiv:2510.02539](https://arxiv.org/abs/2510.02539) find
  6. David M. Blei, Michael I. Jordan, Thomas L. Griffiths, and Joshua B. Tenenbaum. 2003a. Hierarchical topic models and the nested chinese restaurant process. In Proceedings of the 17th International Conference on Pankaj Gupta, Yatin Chaudhary, Thomas Runkler, and Hinrich Schütze. 2020. Neural topic mo… find
  7. Matthew Hoffman, Francis Bach, and David Blei. 2010. Online learning for latent dirichlet allocation. In Advances in Neural Information Processing Systems, volume 23. Curran Associates, Inc. find
  8. Masaru Isonuma, Junichiro Mori, Danushka Bollegala, and Ichiro Sakata. 2020. Tree-Structured Neural Topic Model. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 800–806, Online. Association for Computational Linguistics. find
  9. Haidar Jomaa. 2025. Space-time minilm. find
  10. Diederik P Kingma and Max Welling. 2013. Autoencoding variational bayes. arXiv preprint arXiv:1312.6114.
  11. Sergei Koltcov, Vera Ignatenko, Maxim Terpilovskii, and Paolo Rosso. 2021. Analysis and tuning of hierarchical topic models based on renyi entropy approach. Preprint, arXiv:2101.07598. — [arXiv:2101.07598](https://arxiv.org/abs/2101.07598) find
  12. Ken Lang. 1995. Newsweeder: Learning to filter netnews. Proceedings of the Twelfth International Conference on Machine Learning (ICML). find
  13. Xin Lian, Zekun Wang, and Christopher J. MacLellan. 2025. Efficient and scalable masked word prediction using concept formation. Cognitive Systems Research, 92:101371. find
  14. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. Preprint, arXiv:1907.11692.
  15. Yuyin Lu, Hegang Chen, Pengbo Mao, Yanghui Rao, Haoran Xie, Fu Lee Wang, and Qing Li. 2024. Self-supervised topic taxonomy discovery in the box embedding space. Transactions of the Association for Computational Linguistics, 12:1401–1416. find
  16. Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. 2025. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. Preprint, arXiv:2308.08747. find
  17. Christopher J. MacLellan, Peter Matsakis, and Pat Langley. 2022. Efficient induction of language models via probabilistic concept formation. Advances in Cognitive Systems. find
  18. Christopher J. MacLellan and Harshil Thakur. 2021. Convolutional cobweb: A model of incremental learning from 2d images. Advances in Cognitive Systems. find
  19. Leland McInnes, John Healy, Nathaniel Saul, and Lukas Grossberger. 2018. Umap: Uniform manifold approximation and projection. The Journal of Open Source Software, 3(29):861. find
  20. David Mimno, Wei Li, and Andrew McCallum. 2007. Mixtures of hierarchical topics with pachinko allocation. In Proceedings of the 24th international conference on Machine learning, pages 633–640. find
  21. Dana Movshovitz-Attias, Yair Movshovitz-Attias, Peter Steenkiste, and Christos Faloutsos. 2013. Analysis of the reputation system and user contributions on a question answering website: Stackoverflow. In Proceedings of the 2013 IEEE/ACM International Conference on Advances in Social Networks Analysi… find
  22. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake VanderPlas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Edouard Duchesnay. 2012. Scikit-learn:… find
  23. Adler Perotte, Frank Wood, Noemie Elhadad, and Nicholas Bartlett. 2011. Hierarchically supervised latent dirichlet allocation. Advances in neural information processing systems, 24. find
  24. Chau Minh Pham, Alexander Hoyle, Simeng Sun, Philip Resnik, and Mohit Iyyer. 2024. Topicgpt: A prompt-based topic modeling framework. Preprint, arXiv:2311.01449.
  25. Michael Röder, Andreas Both, and Alexander Hinneburg. 2015. Exploring the space of topic coherence measures. In Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, WSDM ’15, page 399–408, New York, NY, USA. Association for Computing Machinery.
  26. Eleanor Rosch and Carolyn B. Mervis. 1975. Family resemblances: Studies in the internal structure of categories. Cognitive Psychology, 7(4):573–605. find
  27. D. Sculley. 2010. Web-scale k-means clustering. In Proceedings of the 19th International Conference on World Wide Web, WWW ’10, page 1177–1178, New York, NY, USA. Association for Computing Machinery. find
  28. Simra Shahid, Tanay Anand, Nikitha Srikanth, Sumit Bhatia, Balaji Krishnamurthy, and Nikaash Puri. 2023. Hyhtm: Hyperbolic geometry based hierarchical topic models. Preprint, arXiv:2305.09258. — [arXiv:2305.09258](https://arxiv.org/abs/2305.09258) find
  29. Asahi Ushio, Leonardo Neves, Vitor Silva, Francesco. Barbieri, and Jose Camacho-Collados. 2022. Named Entity Recognition in Twitter: A Dataset and Analysis on Short-Term Temporal Shifts. In The 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th I… find
  30. Felipe Viegas, Washington Cunha, Christian Gomes, Antônio Pereira, Leonardo Rocha, and Marcos Goncalves. 2020. CluHTM - semantic hierarchical topic modeling based on CluWords. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8138–8150, Online. Associa… find
  31. Zekun Wang, Ethan L Haarer, Nicki Barari, and Christopher J MacLellan. 2025. Taxonomic networks: A representation for neuro-symbolic pairing. arXiv preprint arXiv:2505.24601. — [arXiv:2505.24601](https://arxiv.org/abs/2505.24601) find
  32. Xiaobao Wu, Thong Nguyen, and Anh Tuan Luu. 2024a. A survey on neural topic models: methods, applications, and challenges. Artificial Intelligence Review, 57(2). find
  33. Xiaobao Wu, Thong Nguyen, and Anh Tuan Luu. 2024b. A survey on neural topic models: methods, applications, and challenges. Artificial Intelligence Review, 57(2). find
  34. Xiaobao Wu, Thong Nguyen, Delvin Ce Zhang, William Yang Wang, and Anh Tuan Luu. 2024c. Fastopic: Pretrained transformer is a fast, adaptive, stable, and transferable topic model. Preprint, arXiv:2405.17978. — [arXiv:2405.17978](https://arxiv.org/abs/2405.17978) find
  35. Xiaobao Wu, Fengjun Pan, Thong Nguyen, Yichao Feng, Chaoqun Liu, Cong-Duy Nguyen, and Anh Tuan Luu. 2024d. On the affinity, rationality, and diversity of hierarchical topic modeling. In Proceedings of the AAAI Conference on Artificial Intelligence. find
  36. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. Advances in Neural Information Processing Systems (NeurIPS).
  37. Yin Zheng, Yu-Jin Zhang, and Hugo Larochelle. 2013. A supervised neural autoregressive topic model for simultaneous image classification and annotation. Preprint, arXiv:1305.5306. — [arXiv:1305.5306](https://arxiv.org/abs/1305.5306) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/singaravadivelan-2026-cobwebtm-probabilistic-concept-formation-lifelong/