Six-State Amino Acid Recoding is Not an Effective Strategy to Offset Compositional Heterogeneity and Saturation in Phylogenetic Analyses

Six-state amino acid recoding strategies are commonly applied to combat the effects of compositional heterogeneity and substitution saturation in phylogenetic analyses. While these methods have been endorsed from a theoretical perspective, their performance has never been extensively tested. Here, we test the effectiveness of six-state recoding approaches by comparing the performance of analyses on recoded and non-recoded data sets that have been simulated under gradients of compositional heterogeneity or saturation. In our simulation analyses, non-recoding approaches consistently outperform six-state recoding approaches. Our results suggest that six-state recoding strategies are not effective in the face of high saturation. Furthermore, while recoding strategies do buffer the effects of compositional heterogeneity, the loss of information that accompanies six-state recoding outweighs its benefits. In addition, we evaluate recoding schemes with 9, 12, 15, and 18 states and show that these consistently outperform six-state recoding. Our analyses of other recoding schemes suggest that under conditions of very high compositional heterogeneity, it may be advantageous to apply recoding using more than six states, but we caution that applying any recoding should include sufficient justification. Our results have important implications for the more than 90 published papers that have incorporated six-state recoding, many of which have significant bearing on relationships across the tree of life. [Compositional heterogeneity; Dayhoff 6-state recoding; S&R 6-state recoding; six-state amino acid recoding; substitution saturation.].