Learn · Beginner

K-means clustering: grouping examples around representative centers

K-means is an unsupervised learning method that groups numerical examples around a chosen number of representative centers. It minimizes the sum of squared distances from examples to their assigned centers. The resulting groups can help summarize data, but their existence does not establish that nature—or a collection of chatbot answers—contains exactly those categories.

The problem it solves

Suppose a support team has thousands of messages and wants to see recurring patterns before it has a reliable labeling scheme. After representing each message numerically, the team can ask for a small number of groups. K-means returns assignments and centers that summarize where the numerical examples concentrate. It does not supply authoritative names such as billing dispute or delivery problem.

This is unsupervised learning because the fitting procedure does not require a correct category label for each example. It still contains human choices: how to represent examples, how many groups to request and what distance should count as similarity. Ground Truth’s embeddings lesson explains one way text becomes a vector of numbers.

Assign, average, repeat

Choose k starting centers. Assign each example to its nearest center using ordinary straight-line distance. Replace each center with the arithmetic mean of the examples assigned to it. Repeat the assignment and averaging steps until the result stops changing meaningfully or reaches a stopping limit.

Stuart Lloyd’s least-squares quantization paper describes the classic alternating procedure underlying this approach. With assignments fixed, the mean minimizes squared distance within a group. With centers fixed, choosing the nearest center gives the cheapest assignment. Each step therefore avoids increasing the objective, although that does not guarantee finding its globally best value.

Think of placing three service depots among customer locations. Given the depots, assign each customer to the nearest one. Given those assignments, move each depot to its group’s average location. The procedure improves its own squared-distance objective. Real delivery costs may involve roads, capacity or mountains, so optimizing that objective does not automatically optimize the actual business.

What the objective rewards

Squaring distance makes faraway points expensive. This encourages relatively compact groups and makes unusual points influential. A single extreme observation can pull a mean toward itself. K-means also prefers geometry that its centers can represent: curved or interlocking groups can fit poorly even when a person sees an obvious pattern.

The centers need not be actual observations. A mean of several text embeddings is a point in vector space, not necessarily the embedding of a sentence anyone wrote. To interpret a group, inspect real nearby examples and difficult boundary cases. A convenient center is a summary of geometry, rather than a semantic definition.

Initialization and scale matter

Different starting centers can lead to different final answers because the procedure can settle into local optima. Running multiple initializations and comparing their objective values is a common practical response. David Arthur and Sergei Vassilvitskii’s k-means++ introduces a careful randomized seeding method that favors points farther from already selected centers. Better initialization reduces one source of instability; it does not make an inappropriate representation appropriate.

Feature units can change the answer just as much. If one coordinate is income in dollars and another is age in years, the income coordinate may dominate squared distance. Converting dollars to cents changes the numerical geometry again. Standardization can help when comparable variation is intended, but it is a modeling choice rather than a universal rule. The principal-component analysis lesson explains a related sensitivity to scale.

The choice of k is also substantive. Increasing it generally reduces the fitting objective because more centers can represent the data more closely. That improvement alone does not establish that the extra groups are useful. Stability, held-out behavior, separation and the intended decision all matter. A data summary should be judged by what it helps people understand or do.

A cluster is not a validated interpretation

Today’s ModelCompare experiment concerns manual grouping of related chatbot answers, not k-means. It raises the same interpretation problem. Answers about status, rationalization and belonging can be grouped broadly or narrowly. An automatic clustering method would move part of that judgment into the representation and distance function, rather than eliminate it.

If a text embedding emphasizes shared tone, answers with different claims may cluster together. If it emphasizes topic, opposing positions about the same subject may be neighbors. A label such as shared belief needs evidence about the claims themselves. Neither a compelling visualization nor a low clustering objective supplies that evidence.

Use k-means when compact numerical summaries are a reasonable match for the task. Inspect examples, test stability across choices and describe the representation. It is a useful tool for organizing a dataset, with a precise mathematical target and a narrower authority than the names people later give its groups.

Ground Truth. (2026, October 23). K-means clustering: grouping examples around representative centers. Ground Truth. https://groundtruth.day/learn/k-means-clustering.html

@misc{groundtruth:k-means-clustering,
  title  = {K-means clustering: grouping examples around representative centers},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {oct},
  url    = {https://groundtruth.day/learn/k-means-clustering.html}
}

Topics: education · clustering · unsupervised-learning · embeddings · fundamentals