The following are the main points you should have learned from this chapter:

• •

  Learning is the ability of an agent to improve its behavior based on experience.

• •

  Supervised learning is the problem of predicting the target of a new input, given a set of input–target pairs.

• •

  Given some training examples, an agent builds a representation that can be used for new predictions.

• •

  Linear models and decision trees are representations which are the basis for more sophisticated models.

• •

  Overfitting occurs when a prediction fits the training set well but does not fit the test set or future predictions.

• •

  Gradient-boosted trees are an efficient and effective representation for many learning problems.

• •

  Data is invariably about the past. If data is used for acting in the future, the predictor may no longer work as well, and we might want the future to be different from the past.