A hybrid method for estimating the predominant number of clusters in a data set

Jamil Alshaqsi, Wenjia Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In cluster analysis, finding out the number of clusters, K, for a given dataset is an important yet very tricky task, simply because there is often no universally accepted correct or wrong answer for non-trivial real world problems and it also depends on the context and purpose of a cluster study. This paper presents a new hybrid method for estimating the predominant number of clusters automatically. It employs a new similarity measure and then calculates the length of constant similarity intervals, L and considers the longest consistent intervals representing the most probable numbers of the clusters under the set context. An error function is defined to measure and evaluate the goodness of estimations. The proposed method has been tested on 3 synthetic datasets and 8 real-world benchmark datasets, and compared with some other popular methods. The experimental results showed that the proposed method is able to determine the desired number of clusters for all the simulated datasets and most of the benchmark datasets, and the statistical tests indicate that our method is significantly better.

Original languageEnglish
Title of host publicationProceedings - 2012 11th International Conference on Machine Learning and Applications, ICMLA 2012
Pages569-573
Number of pages5
Volume2
DOIs
Publication statusPublished - 2012
Event11th IEEE International Conference on Machine Learning and Applications, ICMLA 2012 - Boca Raton, FL, United States
Duration: Dec 12 2012Dec 15 2012

Other

Other11th IEEE International Conference on Machine Learning and Applications, ICMLA 2012
CountryUnited States
CityBoca Raton, FL
Period12/12/1212/15/12

Fingerprint

Statistical tests
Cluster analysis
statistical test
cluster analysis

Keywords

  • cluster analysis
  • cluster number
  • similarity measure

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Education

Cite this

Alshaqsi, J., & Wang, W. (2012). A hybrid method for estimating the predominant number of clusters in a data set. In Proceedings - 2012 11th International Conference on Machine Learning and Applications, ICMLA 2012 (Vol. 2, pp. 569-573). [6406797] https://doi.org/10.1109/ICMLA.2012.146

A hybrid method for estimating the predominant number of clusters in a data set. / Alshaqsi, Jamil; Wang, Wenjia.

Proceedings - 2012 11th International Conference on Machine Learning and Applications, ICMLA 2012. Vol. 2 2012. p. 569-573 6406797.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Alshaqsi, J & Wang, W 2012, A hybrid method for estimating the predominant number of clusters in a data set. in Proceedings - 2012 11th International Conference on Machine Learning and Applications, ICMLA 2012. vol. 2, 6406797, pp. 569-573, 11th IEEE International Conference on Machine Learning and Applications, ICMLA 2012, Boca Raton, FL, United States, 12/12/12. https://doi.org/10.1109/ICMLA.2012.146
Alshaqsi J, Wang W. A hybrid method for estimating the predominant number of clusters in a data set. In Proceedings - 2012 11th International Conference on Machine Learning and Applications, ICMLA 2012. Vol. 2. 2012. p. 569-573. 6406797 https://doi.org/10.1109/ICMLA.2012.146
Alshaqsi, Jamil ; Wang, Wenjia. / A hybrid method for estimating the predominant number of clusters in a data set. Proceedings - 2012 11th International Conference on Machine Learning and Applications, ICMLA 2012. Vol. 2 2012. pp. 569-573
@inproceedings{6e67b4e07501457c9e002be882d07b98,
title = "A hybrid method for estimating the predominant number of clusters in a data set",
abstract = "In cluster analysis, finding out the number of clusters, K, for a given dataset is an important yet very tricky task, simply because there is often no universally accepted correct or wrong answer for non-trivial real world problems and it also depends on the context and purpose of a cluster study. This paper presents a new hybrid method for estimating the predominant number of clusters automatically. It employs a new similarity measure and then calculates the length of constant similarity intervals, L and considers the longest consistent intervals representing the most probable numbers of the clusters under the set context. An error function is defined to measure and evaluate the goodness of estimations. The proposed method has been tested on 3 synthetic datasets and 8 real-world benchmark datasets, and compared with some other popular methods. The experimental results showed that the proposed method is able to determine the desired number of clusters for all the simulated datasets and most of the benchmark datasets, and the statistical tests indicate that our method is significantly better.",
keywords = "cluster analysis, cluster number, similarity measure",
author = "Jamil Alshaqsi and Wenjia Wang",
year = "2012",
doi = "10.1109/ICMLA.2012.146",
language = "English",
isbn = "9780769549132",
volume = "2",
pages = "569--573",
booktitle = "Proceedings - 2012 11th International Conference on Machine Learning and Applications, ICMLA 2012",

}

TY - GEN

T1 - A hybrid method for estimating the predominant number of clusters in a data set

AU - Alshaqsi, Jamil

AU - Wang, Wenjia

PY - 2012

Y1 - 2012

N2 - In cluster analysis, finding out the number of clusters, K, for a given dataset is an important yet very tricky task, simply because there is often no universally accepted correct or wrong answer for non-trivial real world problems and it also depends on the context and purpose of a cluster study. This paper presents a new hybrid method for estimating the predominant number of clusters automatically. It employs a new similarity measure and then calculates the length of constant similarity intervals, L and considers the longest consistent intervals representing the most probable numbers of the clusters under the set context. An error function is defined to measure and evaluate the goodness of estimations. The proposed method has been tested on 3 synthetic datasets and 8 real-world benchmark datasets, and compared with some other popular methods. The experimental results showed that the proposed method is able to determine the desired number of clusters for all the simulated datasets and most of the benchmark datasets, and the statistical tests indicate that our method is significantly better.

AB - In cluster analysis, finding out the number of clusters, K, for a given dataset is an important yet very tricky task, simply because there is often no universally accepted correct or wrong answer for non-trivial real world problems and it also depends on the context and purpose of a cluster study. This paper presents a new hybrid method for estimating the predominant number of clusters automatically. It employs a new similarity measure and then calculates the length of constant similarity intervals, L and considers the longest consistent intervals representing the most probable numbers of the clusters under the set context. An error function is defined to measure and evaluate the goodness of estimations. The proposed method has been tested on 3 synthetic datasets and 8 real-world benchmark datasets, and compared with some other popular methods. The experimental results showed that the proposed method is able to determine the desired number of clusters for all the simulated datasets and most of the benchmark datasets, and the statistical tests indicate that our method is significantly better.

KW - cluster analysis

KW - cluster number

KW - similarity measure

UR - http://www.scopus.com/inward/record.url?scp=84873589657&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84873589657&partnerID=8YFLogxK

U2 - 10.1109/ICMLA.2012.146

DO - 10.1109/ICMLA.2012.146

M3 - Conference contribution

SN - 9780769549132

VL - 2

SP - 569

EP - 573

BT - Proceedings - 2012 11th International Conference on Machine Learning and Applications, ICMLA 2012

ER -