Learn · Beginner

Decision trees: learning a prediction by asking a sequence of questions

A decision tree predicts an outcome by asking a sequence of questions about an example and following the corresponding branches to a final answer. It matters because the same structure can learn useful rules from ordinary tables while making an individual prediction’s path easy to inspect.

Imagine sorting incoming support requests. The first question might ask whether a payment failed. One branch then asks whether the account is new; another asks how long the user has waited. At the end of each route, a leaf supplies a prediction, such as the likely support category or expected handling time. A learned tree chooses those questions from examples instead of requiring someone to write every rule by hand.

The inputs are features: measurable attributes such as account age, transaction amount or the presence of an error message. During training, the tree receives examples with known outcomes. A candidate split might separate transactions above and below a particular amount. The training algorithm asks whether that split creates groups whose outcomes are easier to predict than the original mixed group.

For classification, one common measure is impurity. A group containing equal numbers of two classes is more mixed than one containing almost entirely one class. A split is useful when its children, weighted by how many examples they contain, are less mixed than their parent. Another approach uses information gain, which measures the reduction in uncertainty about the label. J. R. Quinlan’s Induction of Decision Trees is a canonical account of learning trees through this family of ideas.

For regression, the target is a number rather than a category. A tree might split homes by floor area and location, then predict the average observed price within each final group. Candidate splits can be scored by how much they reduce squared prediction error. The resulting function changes in steps: two homes in the same leaf receive the same basic prediction even if their areas differ slightly.

Most familiar tree-building procedures are greedy. They choose the best available split now and then repeat the process inside each new group. They do not search every possible complete tree before deciding. This makes training practical, but a locally attractive first question can prevent the algorithm from finding a better overall arrangement. A tree is an efficient learned partition, not necessarily the uniquely best explanation of a dataset.

A simple numerical example shows what a leaf means. Suppose ten training requests follow one path and eight required a billing specialist. The tree might label that leaf billing and attach an observed frequency of eight out of ten. That frequency summarizes a small historical group. It is not automatically a trustworthy 80% probability for every future request; calibration and sample size still matter.

The biggest danger is excessive growth. A deep tree can keep adding questions until nearly every training example sits in its own leaf. Training accuracy can then look excellent even if the questions capture accidents. Limiting depth, requiring a minimum number of examples in each leaf or pruning branches constrains this behavior. The general problem is overfitting: a model can describe yesterday’s examples more precisely while predicting tomorrow’s cases less well.

Trees also explain interactions naturally. An account-age threshold may matter only after a payment has failed, while waiting time matters on a different branch. The feature’s effect depends on earlier answers. This is one reason trees are useful on tables where relationships are irregular and conditional rather than well described by one straight-line formula.

Readable paths have limits. A small tree can be audited by a person, but a tree with thousands of leaves becomes difficult to understand as a whole. A chosen feature can also proxy for a sensitive attribute, and an understandable rule can still be discriminatory or wrong. Inspecting a path explains how this model reached its answer; it does not establish that the relationship is causal or ethically acceptable.

Single trees can be unstable: a small change in training examples may change an early split and therefore many later predictions. Ensembles address that weakness by combining trees, while tabular foundation models take a different route through pretraining. Those are separate methods built around different tradeoffs; the fundamental tree operation remains dividing examples through questions.

Today’s observational call-center study offers a useful hypothetical application. A tree might explore which call types correlate with successful intake, but it could not make a human-versus-AI comparison causal merely by producing neat rules. Call assignment and population differences would still matter. A useful tree workflow therefore combines meaningful features, disciplined holdouts, sensible complexity limits and careful interpretation of what the learned questions actually establish.

Ground Truth. (2026, October 16). Decision trees: learning a prediction by asking a sequence of questions. Ground Truth. https://groundtruth.day/learn/decision-trees.html

@misc{groundtruth:decision-trees,
  title  = {Decision trees: learning a prediction by asking a sequence of questions},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {oct},
  url    = {https://groundtruth.day/learn/decision-trees.html}
}

Topics: decision-trees · fundamentals · classification · regression · tabular-data