Reducing model complexity of the general Markov model of evolution

The selection of an optimal model for data analysis is an important component of model-based molecular phylogenetic studies. Owing to the large number of Markov models that can be used for data analysis, model selection is a combinatorial problem that cannot be solved by performing an exhaustive search of all possible models. Currently, model selection is based on a small subset of the available Markov models, namely those that assume the evolutionary process to be globally stationary, reversible, and homogeneous. This forces the optimal model to be time reversible even though the actual data may not satisfy these assumptions. This problem can be alleviated by including more complex models during the model selection. We present a novel heuristic that evaluates a small fraction of these complex models and identifies the optimal model. The Author 2011. Published by Oxford University Press on behalf of the Society for Molecular Biology and Evolution. All rights reserved.