Model selection when a key parameter is constrained to be in an interval

This article considers model selection procedures based on choosing the model with the largest maximized log-likelihood minus a penalty, when key parameters are restricted to be in a closed interval. Its main emphasis is how these penalties might be chosen in small samples to give good properties of the resultant procedure. We illustrate two model selection problems in the context of Box-Cox transformations and their application to the linear regression model. Simulation results for both problems indicate that the new procedure clearly dominates existing procedures in terms of having higher probabilities of correctly selecting the true model.