A quadratic model seems like it might be a better fit.

But how can we measure that?

Deviance ()^* is minus two times the log likelihood of the data, given the model and a point estimate for the model parameters ():

^* Note: a common definition of deviance requires a comparison to a ‘saturated’ model. For clarity, we use this simpler definition.

• Predictions err in systematic ways
• Misses meaningful patterns in the relationship between predictor(s) and outcome

• Takes random variation to be systematic
• Predicts cases in the sample well, but tends to predict new data very poorly

Fit the model on a subset of the data (e.g. 50%)

Asses model fit on the held-out portion of the data

Penalize deviance score for each added parameter by some ‘reasonable’ value.

Model the average difference in deviance between training and test data.

Strategy 1

Strategy 2

Strategy 3