Researchers at the University of Negev are developing DeepDPM: a deep clustering algorithm with an unknown number of clusters


Clustering is an essential unsupervised learning job where class labels are not accessible, unlike in the supervised situation of classification. Furthermore, the number of classes represented by K and their relative sizes are unknown in the totally unsupervised environment on which this research is focused.

Clustering tasks have not been overlooked by Deep Learning (DL). Large and high-dimensional data sets are usually clustered better and more efficiently using DL approaches than using conventional clustering methods. However, while nonparametric approaches to classical clustering have advantages over parametric methods (methods that require a known K), nonparametric deep clustering methods are few.

Unfortunately, the latter is neither scalable nor effective enough. It is advantageous to be able to derive the latent K. Parametric approaches can perform poorly if K is not accurately estimated. In both balanced and unbalanced data sets, using the wrong K can have major negative implications for parametric approaches.

Changing K during training has positive effects on optimization; For example, if you split a single cluster into two, many data labels will change at once. When K is unknown, a popular workaround is to use model selection, where a parametric technique is run multiple times with different K values ​​over a wide range, and then the best K is selected using an unsupervised criterion. K could be a valuable quantity in and of itself.

When K is uncertain, nonparametric Bayesian (BNP) mixture models such as the Dirichlet Process Mixture (DPM) offer an elegant, data-adaptive solution to clustering. However, due to the high computational costs associated with DPM inference, few studies have attempted to use it in conjunction with deep clustering.

Researchers at the University of Negev, in a recent publication, attempted to fill this gap by developing DeepDPM, a powerful deep nonparametric approach. They advocated effectively combining the benefits of DL and DPM. Despite its unfair advantage, DeepDPM gives a result consistent with the best parametric approaches, even when K is known.

DeepDPM, the proposed approach, uses cluster splits and merges to modify K and dynamic design to handle such changes. For expectation maximization algorithms, it also uses a novel amortized inference technique. DeepDPM can be integrated into cluster-based deep pipelines.

DeepDPM is differentiable for most of the training, as opposed to an offline clustering step. Across a wide range of datasets and metrics, DeepDPM outperforms existing non-parametric clustering algorithms (both classical and deep). It also manages class imbalances and scales effectively with huge datasets.

The team compared DeepDPM to both parametric and non-parametric approaches. The study used the MNIST, USPS, and Fashion MNIST datasets and their unbalanced counterparts. DeepDPM dominates almost equally across all datasets and measures, and its performance advantage only grows in unbalanced scenarios, according to the results. It was also observed that non-parametric methods were less affected by imbalance than parametric methods. DeepDPM, like most clustering techniques, would struggle to recover when input capabilities are weak. Parametric approaches can also be a slightly better alternative when K is known and the dataset is balanced.


A deep nonparametric clustering approach, a dynamic architecture that adapts to changing K values, has been described by researchers at the University of Negev. Deep and non-deep nonparametric approaches were surpassed by the method, which yielded state-of-the-art results. The method’s resistance to both class imbalances and the initial K was demonstrated by the researchers. They were the first approach of its kind to publish results on ImageNet and demonstrate DeepDPM’s scalability.

This Article Is Based On The Research Paper 'DeepDPM: Deep Clustering With an Unknown Number of Clusters'. All Credit For This Research Goes To The Researchers of This Project. Check out the paper and codes.

Please Don't Forget To Join Our ML Subreddit

Comments are closed.