ENHANCING CLUSTER INTERPRETABILITY IN K-MEANS VIA PCA VISUALIZATION, FEATURE NORMALIZATION, AND ROBUST INITIALIZATION METHODS

Аннотация

The k-Means algorithm is one of the most widely used unsupervised learning methods for partitioning data into homogeneous groups. This study explores the efficacy of k- Means clustering on three real-world datasets: a synthetic 2D blob dataset, a customer segmentation dataset based on spending behavior, and a global cities dataset based on geographic coordinates. We analyze the influence of data scaling, initial centroid selection (random vs k-means++), and the number of clusters (k) using silhouette score, elbow method, and PCA visualizations. The study also highlights the limitations of k-Means such as sensitivity tooutliers and non-convex clustershapes, offering guidelines forpractical implementation. Key words: k-Means, unsupervised learning, clustering, silhouette score, elbow method, k- means++, PCAvisualization, customer segmentation, geospatial clustering. Introduction. Clustering is a fundamental task in unsupervised machine learning where the goal is to discover inherent groupings in unlabeled data. Among various algorithms, k- Means is favored for itssimplicity, scalability, and intuitiveworking mechanism. Given a set of objects without predefined categories, k-Means assigns each object to one of clusters by minimizing the distanceto the cluster centroids. Despiteits popularity, k-Means suffers from several practical issues such as sensitivity to initial centroid placement, outlier infkluence, and difficulty in detecting non-spherical cluster shapes. This paper aims to provide an empirical and mathematical investigation of the k-Means algorithm,supported byvisualillustrations and real-world applications. Problem statement. The central goal of this study is to investigate the effectiveness of the k-Means clustering algorithm in discovering meaningful patterns from unlabeled data. Specifically, we aimto answer thefollowing key research questions: page1924 INTERNATIONAL JOURNAL OF ARTIFICIAL INTELLIGENCE ISSN:2692-5206,ImpactFactor:12,23 AmericanAcademicpublishers,volume05,issue06,2025 Journal: https://www.academicpublishers.org/journals/index.php/ijai 1. Howdoesthe number ofclusters affectthequality ofclustering? Since the number of clusters is not known a priori, a mechanism such as the elbow method or silhouette analysis is req uired to select the optimal . We seek to empirically validatethesetechniques across diverse datasets. 2. Whatis theimpact offeaturescaling onclustering performkance? Since k-Means relies on Euclidean distances, features with larger numerical ranges can disproportionately influence the cluster assignment. We assess whether z-score standardization orMin-Max scaling improves clustering consistency. 3. Howdoesthe initialization strategy affectconvergence andstability? The algorithm's outcome may vary depending on the initial centroid positions. We compare random initialization with the more robust k-means++ method, which selects initial centroids to maximizespread. 4. Cank-Means reveal interpretablepatterns in real-worldapplications? Weapply thealgorithmtothree datasets: Asynthetic 2Dblob dataset (control case), o Acustomersegmentation dataset (business case), o Aglobal cities dataset (geospatial case). o Ourobjective is to quantify and visualizetheclustering results under these variables using clusterinertia, silhouettescore, and PCA-based projections. Mathematical foundations of k-Means. The k-Means algorithm partitions a dataset into kk distinct, non-overlapping clusters by minimizing the within-cluster sum of squared errors (WCSS),also referred toas inertia. Let be a dataset of points in a -dimensional space. The algorithm aims to find a partitiodn such that the following cost function is minimizedX: = {x1,x2,…,xn} ⊂ ℝ n d {C1,C2,…,Ck} k 2 J = x−μi where: i=1 x∈Ci  isthecentroid ofcluster ,  denotes the Euclidean norm. μi Ci Thealgor⋅ithm proceeds iteratively tominimizethisobjective. Algorithmsteps: 1. Initialization. Choose initial centroids either randomly or using k- means++, which spreads thecentroids moreevenly to reduce variance. 2. Assignment Step. k μ1,μ2,…,μk Assign each datapoint to thenearest cluster based onEuclidean distance: xj page1925 INTERNATIONAL JOURNAL OF ARTIFICIAL INTELLIGENCE ISSN:2692-5206,ImpactFactor:12,23 AmericanAcademicpublishers,volume05,issue06,2025 Journal: https://www.academicpublishers.org/journals/index.php/ijai 2 2 3. UpdateStep. Ci = {xj: xj −μi ≤ xj −μl ,∀l = 1,…,k} Updatethe centroid ofeach cluster: 1 μi = xj Ci xj∈Ci 4. Convergence Check. Repeat Steps 2–3 until the assignments do not change or the change in is below a tolerance threshold. J Complexity: Time complexity: , where is thenumberofiterations. O(n∗k∗t∗d) Spacetcomplexity: O n+k∗d Despite its practical advantages, k-Means has several notable limitations. It is inherently sensitive to the initial positions of the centroids and can converge to suboptimal partitions if these positions are poorly chosen. The algorithm also assumes that clusters are isotropic and of roughl