Factorial, convergent, and discriminant validity of timss math and science motivation measures: A comparison of arab and anglo-saxon countries

Herbert W. Marsh, Adel Salah Abduljabbar, Maher M. Abu-Hilal, Alexandre J S Morin, Faisal Abdelfattah, Kim Chau Leung, Man K. Xu, Benjamin Nagengast, Philip Parker

Research output: Contribution to journalArticle

68 Citations (Scopus)

Abstract

For the international Trends in International Mathematics and Science Study (TIMSS2007) math and science motivation scales (self-concept, positive affect, and value), we evaluated the psychometric properties (factor structure, method effects, gender differences, and convergent and discriminant validity) in 4 Arab-speaking countries (Saudi Arabia, Jordan, Oman, and Egypt) and 4 English-speaking Anglo-Saxon countries (United States, England, Australia, and Scotland). In this article, we also highlight methodological weaknesses in the TIMSS approach to these motivation measures. We found reasonable support for within-group invariance across the math and science domains and between-group invariance across countries for full factor loading invariance and partial item intercept invariance. However, the factor structure is complicated by strong negative-item method effects and correlated unique characteristics associated with the use of math and science items with parallel wording. Correlations involving the motivation factors were reasonably similar across countries, supporting both discriminant and convergent validity in relation to achievement, plans to take more coursework in math and science, and long-term educational aspirations. However, gender differences largely favor girls in the Arab countries (with strong single-sex education systems) relative to Anglo countries (and international norms). The juxtapositions of latent mean differences in achievement and motivation factors are perplexing; students from Anglo countries had substantially higher achievement than students from Arab countries but had substantially lower motivation across all 8 math and science factors.

Original languageEnglish
Pages (from-to)108-128
Number of pages21
JournalJournal of Educational Psychology
Volume105
Issue number1
DOIs
Publication statusPublished - 2013

Fingerprint

Motivation
Middle East
science
Arab countries
speaking
gender-specific factors
mathematics studies
Oman
Students
sex education
Saudi Arabia
science studies
Jordan
self-concept
Sex Education
Mathematics
Egypt
psychometrics
education system
Scotland

Keywords

  • Cross-cultural measurement invariance
  • Math and science gender difference
  • Math and science motivation
  • Negative item method effects
  • Trends in international mathematics and science study

ASJC Scopus subject areas

  • Developmental and Educational Psychology
  • Education

Cite this

Factorial, convergent, and discriminant validity of timss math and science motivation measures : A comparison of arab and anglo-saxon countries. / Marsh, Herbert W.; Abduljabbar, Adel Salah; Abu-Hilal, Maher M.; Morin, Alexandre J S; Abdelfattah, Faisal; Leung, Kim Chau; Xu, Man K.; Nagengast, Benjamin; Parker, Philip.

In: Journal of Educational Psychology, Vol. 105, No. 1, 2013, p. 108-128.

Research output: Contribution to journalArticle

Marsh, Herbert W. ; Abduljabbar, Adel Salah ; Abu-Hilal, Maher M. ; Morin, Alexandre J S ; Abdelfattah, Faisal ; Leung, Kim Chau ; Xu, Man K. ; Nagengast, Benjamin ; Parker, Philip. / Factorial, convergent, and discriminant validity of timss math and science motivation measures : A comparison of arab and anglo-saxon countries. In: Journal of Educational Psychology. 2013 ; Vol. 105, No. 1. pp. 108-128.
@article{c0b8b1025446422c9819c05b902f196a,
title = "Factorial, convergent, and discriminant validity of timss math and science motivation measures: A comparison of arab and anglo-saxon countries",
abstract = "For the international Trends in International Mathematics and Science Study (TIMSS2007) math and science motivation scales (self-concept, positive affect, and value), we evaluated the psychometric properties (factor structure, method effects, gender differences, and convergent and discriminant validity) in 4 Arab-speaking countries (Saudi Arabia, Jordan, Oman, and Egypt) and 4 English-speaking Anglo-Saxon countries (United States, England, Australia, and Scotland). In this article, we also highlight methodological weaknesses in the TIMSS approach to these motivation measures. We found reasonable support for within-group invariance across the math and science domains and between-group invariance across countries for full factor loading invariance and partial item intercept invariance. However, the factor structure is complicated by strong negative-item method effects and correlated unique characteristics associated with the use of math and science items with parallel wording. Correlations involving the motivation factors were reasonably similar across countries, supporting both discriminant and convergent validity in relation to achievement, plans to take more coursework in math and science, and long-term educational aspirations. However, gender differences largely favor girls in the Arab countries (with strong single-sex education systems) relative to Anglo countries (and international norms). The juxtapositions of latent mean differences in achievement and motivation factors are perplexing; students from Anglo countries had substantially higher achievement than students from Arab countries but had substantially lower motivation across all 8 math and science factors.",
keywords = "Cross-cultural measurement invariance, Math and science gender difference, Math and science motivation, Negative item method effects, Trends in international mathematics and science study",
author = "Marsh, {Herbert W.} and Abduljabbar, {Adel Salah} and Abu-Hilal, {Maher M.} and Morin, {Alexandre J S} and Faisal Abdelfattah and Leung, {Kim Chau} and Xu, {Man K.} and Benjamin Nagengast and Philip Parker",
year = "2013",
doi = "10.1037/a0029907",
language = "English",
volume = "105",
pages = "108--128",
journal = "Journal of Educational Psychology",
issn = "0022-0663",
publisher = "American Psychological Association Inc.",
number = "1",

}

TY - JOUR

T1 - Factorial, convergent, and discriminant validity of timss math and science motivation measures

T2 - A comparison of arab and anglo-saxon countries

AU - Marsh, Herbert W.

AU - Abduljabbar, Adel Salah

AU - Abu-Hilal, Maher M.

AU - Morin, Alexandre J S

AU - Abdelfattah, Faisal

AU - Leung, Kim Chau

AU - Xu, Man K.

AU - Nagengast, Benjamin

AU - Parker, Philip

PY - 2013

Y1 - 2013

N2 - For the international Trends in International Mathematics and Science Study (TIMSS2007) math and science motivation scales (self-concept, positive affect, and value), we evaluated the psychometric properties (factor structure, method effects, gender differences, and convergent and discriminant validity) in 4 Arab-speaking countries (Saudi Arabia, Jordan, Oman, and Egypt) and 4 English-speaking Anglo-Saxon countries (United States, England, Australia, and Scotland). In this article, we also highlight methodological weaknesses in the TIMSS approach to these motivation measures. We found reasonable support for within-group invariance across the math and science domains and between-group invariance across countries for full factor loading invariance and partial item intercept invariance. However, the factor structure is complicated by strong negative-item method effects and correlated unique characteristics associated with the use of math and science items with parallel wording. Correlations involving the motivation factors were reasonably similar across countries, supporting both discriminant and convergent validity in relation to achievement, plans to take more coursework in math and science, and long-term educational aspirations. However, gender differences largely favor girls in the Arab countries (with strong single-sex education systems) relative to Anglo countries (and international norms). The juxtapositions of latent mean differences in achievement and motivation factors are perplexing; students from Anglo countries had substantially higher achievement than students from Arab countries but had substantially lower motivation across all 8 math and science factors.

AB - For the international Trends in International Mathematics and Science Study (TIMSS2007) math and science motivation scales (self-concept, positive affect, and value), we evaluated the psychometric properties (factor structure, method effects, gender differences, and convergent and discriminant validity) in 4 Arab-speaking countries (Saudi Arabia, Jordan, Oman, and Egypt) and 4 English-speaking Anglo-Saxon countries (United States, England, Australia, and Scotland). In this article, we also highlight methodological weaknesses in the TIMSS approach to these motivation measures. We found reasonable support for within-group invariance across the math and science domains and between-group invariance across countries for full factor loading invariance and partial item intercept invariance. However, the factor structure is complicated by strong negative-item method effects and correlated unique characteristics associated with the use of math and science items with parallel wording. Correlations involving the motivation factors were reasonably similar across countries, supporting both discriminant and convergent validity in relation to achievement, plans to take more coursework in math and science, and long-term educational aspirations. However, gender differences largely favor girls in the Arab countries (with strong single-sex education systems) relative to Anglo countries (and international norms). The juxtapositions of latent mean differences in achievement and motivation factors are perplexing; students from Anglo countries had substantially higher achievement than students from Arab countries but had substantially lower motivation across all 8 math and science factors.

KW - Cross-cultural measurement invariance

KW - Math and science gender difference

KW - Math and science motivation

KW - Negative item method effects

KW - Trends in international mathematics and science study

UR - http://www.scopus.com/inward/record.url?scp=84880966922&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84880966922&partnerID=8YFLogxK

U2 - 10.1037/a0029907

DO - 10.1037/a0029907

M3 - Article

AN - SCOPUS:84880966922

VL - 105

SP - 108

EP - 128

JO - Journal of Educational Psychology

JF - Journal of Educational Psychology

SN - 0022-0663

IS - 1

ER -