Factorial, convergent, and discriminant validity of timss math and science motivation measures: A comparison of arab and anglo-saxon countries

For the international Trends in International Mathematics and Science Study (TIMSS2007) math and science motivation scales (self-concept, positive affect, and value), we evaluated the psychometric properties (factor structure, method effects, gender differences, and convergent and discriminant validity) in 4 Arab-speaking countries (Saudi Arabia, Jordan, Oman, and Egypt) and 4 English-speaking Anglo-Saxon countries (United States, England, Australia, and Scotland). In this article, we also highlight methodological weaknesses in the TIMSS approach to these motivation measures. We found reasonable support for within-group invariance across the math and science domains and between-group invariance across countries for full factor loading invariance and partial item intercept invariance. However, the factor structure is complicated by strong negative-item method effects and correlated unique characteristics associated with the use of math and science items with parallel wording. Correlations involving the motivation factors were reasonably similar across countries, supporting both discriminant and convergent validity in relation to achievement, plans to take more coursework in math and science, and long-term educational aspirations. However, gender differences largely favor girls in the Arab countries (with strong single-sex education systems) relative to Anglo countries (and international norms). The juxtapositions of latent mean differences in achievement and motivation factors are perplexing; students from Anglo countries had substantially higher achievement than students from Arab countries but had substantially lower motivation across all 8 math and science factors.