Factor Structure Equivalence across Gender of a Volition Model among Engineering Undergraduates: Comparison of LRT, ΔCFI, Gamma Hat and McDonald’s NCI Approaches
DOI:
https://doi.org/10.16920/jeet/2026/v40i1/26137Keywords:
Measurement Invariance Testing, Measurement Equivalence Testing, Volition, Engineering Students, Engineering Education, Likelihood Ratio Test (lRT), Ordinal Omega Reliability Coefficient, Sensitivity Analysis, Cheung And Rensvold’s Criterion, Steiger’s Gamma’s Hat, Mcdonald’s Nci CriterionAbstract
Test of equal performance of a structural or measurement model in measuring a psychological phenomenon or a construct, in subjects across multiple groups, technically known as measurement invariance or measurement equivalence testing, is an essential practice in Psychometrics, which is not commonly followed in the Indian context. To promote awareness and encourage initiation of its inclusion in tool validation exercises, measurement invariance testing of the volitional model of self-regulated learning varying with respect to gender was conducted using 533 (373 males and 160 females) second and third year computer science and mechanical engineering students of the Punjab state of India. The instruments of data collection were the Academic Procrastination Scale Short Form by Yockey (2016), the Zimbardo Time Perspective Inventory Short Form by Orosz et al., (2017) and the Academic Delay of Gratification Scale by Bembenutty and Karabenick (1998). Apart from the popular approach of measurement invariance testing like Likelihood Ratio Test (LRT), lesser known invariance testing techniques like Cheung and Rensvold (2002), Steiger’s Gamma’s Hat and McDonald’s NCI criterion which are unaffected by model complexity, sample size and overall fit measures, were administered on the collected data. The model showed acceptable overall CFA fit. Configural invariance was supported across gender. Metric invariance was rejected by LRT, ΔCFI, and Gamma Hat but supported only by McDonald’s NCI by a small margin. The same result was confirmed by sensitivity analysis too. Therefore, the model should be interpreted as configurally invariant but not robustly metrically invariant across gender. Implications of the study with respect to engineering education and psychometric practices are discussed.
Downloads
References
All India Survey on Higher Education AISHE 2018-19, (2019). Ministry of Human Resource Development, Department of Higher Education, Government of India.
Bembenutty, H., & Karabenick, S. A. (1998). Academic delay of gratification. Learning and Individual Differences, 10(4), 329-346. doi:10.1016/S1041-6080(99)80126-5.
Bembenutty, H., & Karabenick, S. A. (2004). Inherent association between academic delay of gratification, future time perspective, and self-regulated learning, Educational Psychology Review, 16(1), pp.35-37.
Buss, A. R., & Royce, J. R. (1975). Detecting cross-cultural commonalties and differences: Intergroup factor analysis. Psychological Bulletin, 82, 128–136.
Brannick, M. T. (1995). Critical comments on applying covariance structure modeling. Journal of Organizational Behavior, 16, 201–213.3
Berry, D.A. & Lindgren, B.W. 1990. Statistics. Brooks/Cole Publishing Co., Pacific Grove, Calif.
Chasmar, J.M., Melloy, B.J. & Benson, L.B. (2015).Use of Self-Regulated Learning Strategies by Second-Year Industrial Engineering Students, Paper presented at the 122nd ASEE Annual Conference and Exposition, Seattle, WA.
Chakraborty, R. & Chechi, V.K. (2020). Network Psychometrics Based Validation of Volitional Component of Self Regulated Learning and Estimation of its Polychoric Ordinal Omega Relaibility, Mukt Shabd (UGC Care Journal), 9(6), ISSN: 2347-3150, pp: 5890-5909, doi: 09.0014.MSJ.2020.V9I6.0086781.105094.
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233–255. doi:10.1207/S15328007SEM0902_5
Corno, L., & Kanfer, R. (1993). Chapter 7: The Role of Volition in Learning and Performance. Review of Research in Education, 19(1), 301–341. doi:10.3102/0091732x019001301.
Churchill, G.A. (1979), ―A paradigm for developing better measures of marketing constructs‖, Journal of Marketing Research, 16(1), pp. 64–73.
Cheung, G. W., & Rensvold, R. B. (1999). Testing Factorial Invariance across Groups: A Reconceptualization and Proposed New Method. Journal of Management, 25(1), 1–27. https://doi.org/10.1177/014920639902500101
Davidov, E., Dulmer, H., Schluter, E., Schmidt, P., & Meuleman, B. (2012). Using a multilevel structural equation modeling approach to explain cross-cultural measurement noninvariance. Journal of Cross-Cultural Psychology, 43, 558-575. doi:10.1177/0022022112438397
Dorrenbacher,.L. & Perels, .F. (2015). Volition completes the puzzle: Development and evaluation of an integrative trait model of self regulated learning, Frontline Learning Research, 3(4), pp:14-36, DOI: http://dx.doi.org/10.14786 /flr.v3i4.179
Deng, L. & Chan, W. (2017). Testing the difference between reliability coefficients alpha and omega, Educational and Psychological Measurements, 77(2), pp: 185-203.
Fan, X., & Sivo, S. A. (2007). Sensitivity of fit indices to model misspecification and model types. Multivariate Behavioral Research, 42(3), 509–529.
Frohlich, M.T. (2002), ―E-integration in the supply chain: barriers and performance‖, Decision Sciences, 33(4), pp. 537–556.
Gardner, J. N., Pattengale, J., & Schreiner, L. A. (2000). The sophomore year: Summary and recommendations. In L. A. Schreiner and J. Pattengale (Eds.), Visible solutions for invisible students: Helping sophomores succeed (Monograph No. 31). Columbia, SC: University of South Carolina, National Resource Center for The First-Year Experience & Students in Transition.
Gadermann, A. M., Guhn, M. & Zumbo, B.D. (2012). Estimating ordinal reliability for Likert-type and ordinal item response data: A conceptual, empirical, and practical guide. Practical Assessment, Research & Evaluation,17(3). Available online: http://pareonline.net/getvn.asp?v=17&n=3
Green, S. B., & Yang, Y., (2009a). Commentary on coefficient alpha: A cautionary tale. Psychometrika, 74(1), pp:121-135.
Green, S. B., & Yang, Y., (2009b). Reliability of summed item scores using structural equation modeling: An alternative to coefficient alpha. Psychometrika, 74(1), pp: 155-167.
Goodboy, A.K. & Martin, M.M. (2020). Omega over alpha for reliability estimation of unidimensional communication measures, Annals of the International Communication Association, 44:4, 422-439, DOI: 10.1080/23808985.2020.1846135
Hu, L. T., & Bentler, P. M. (1995). Evaluating model fit. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues and application (pp. 77-99). Thousand Oaks, CA: Sage.
Hu, L., & Bentler, P.M. (1998). Fit indices in covariance structure modeling: Sensitivity to under parameterized model misspecification, Psychological Methods, 3(4), 424-453.
Hayes, A.F. & Coutts, J.J. (2020). Use Omega Rather than Cronbach’s Alpha for Estimating Reliability. But…, Communication Methods and Measures, 14:1, 1-24, DOI: 10.1080/19312458.2020.1718629
Horn, J. L.,& McArdle, J. J. (1992). A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research, 18, 117–144.
Hall, R.J., Snell, A.F., & Foust, M.S. (1999). Item parceling strategies in SEM: Investigating the subtle effects of unmodeled secondary constructs. Organizational Research Methods, 2, pp:233-256.
Irvine, S. H. (1969). Contributions of ability and attainment testing in Africa to a general theory of intellect. Journal of Biosocial Science, 1, 91–102.
Jimola, A.M. & Jimola, F.E. (2024). Female students and the field of engineering: Stemming the tide of gender underrepresentation for sustainable development. International Journal of Research in Education and Science (IJRES), 10(3), 641-652. https://doi.org/10.46328/ijres.3459
Kelloway, E. K. (1995). Structural equation modelling in perspective. Journal of Organizational Behavior, 16, 215–224.
Kenny, D. A., & McCoach, D. B. (2003). Effect of the number of variables on measures of fit in structural equation modeling. Structural Equation Modeling, 10(3), 333–351.
Kuhl,.J.,(1985): “Volitional mediators of cognition-behavior consistency: Self-regulatory processes and action versus state orientation in Action control: From cognition to behavior”, J. Kuhl & J. Beckmann (Ed), Handbook of physiology, (New York: Springer-Verlag) 101-128.
Little, T. D. (1997). Mean and covariance structures (MACS) analyses of cross-cultural data: Practical and theoretical issues. Multivariate Behavioral Research, 32, 53–76.
Levine, J. & Wyckoff, J. (1990). Identification of Student Characteristics that Predict Persistence and Success in an Engineering College.
Meredith, W. (1964). Notes on factorial invariance. Psychometrika 29, 177–185. https://doi.org/10.1007/BF02289699
Meredith,W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58, 525–543.
Mullen, M. (1995). Diagnosing measurement equivalence in cross-national research. Journal of International Business Studies, 3, 573–596.
McDonald, R. P. (1989). An index of goodness-of-fit based on noncentrality. Journal of Classification, 6, 97-103.
McBurnie, J. E., Campbell, M., & West, J. M. (2012). Avoiding the second year slump: A transition framework for students progressing through university. International Journal of Innovation in Science and Mathematics Education, 20(2), 14-24.
Nackerud, S. (2013). Collaborative Statistics Using Spreadsheets - snackeru. OpenStax CNX.
Orosz, G., Dombi, E., Toth-Kiraly, I. & Roland-Levy, C. (2017). The Less is More: The 17-item Zimbardo Time Perspective Inventory, Current Psychology, 36(1), pp: 39-47, doi: 10.1007/s12144-015-9382-2
Osman, A., & Yusof, Z.M. (2026). A hybrid factor analysis framework for ordinal data integrating CFA, ML, and DL methods. Front. Appl. Math. Stat. 12:1830237.doi: 10.3389/fams.2026.1830237
Patrick A, Riegle-Crumb C, Borrego M. (2021). Examining the Gender Gap in Engineering Professional Identification. Journal of Women Minor Science Engineeing.27(1):31-55. doi: 10.1615/jwomenminorscieneng.2020030909.
Putnick, D.L., & Bornstein, M.H. (2016). Measurement Invariance Conventions and Reporting: The State of the Art and Future Directions for Psychological Research. Dev Rev. 41:71-90. doi: 10.1016/j.dr.2016.06.004.
Ruch, W., Heintz, S., Platt, T., Wagner, L., & Proyer, R.T. (2018). Broadening humor: Comic styles differentially tap into temperment, character and ability, Frontiers in Psychology, 9(6), doi: 10.3389/ fpsyg.2018.00006.
Revelle, W. (2019). Psych Package Manual, Ver. 1.9.12, Procedures for Psychological, Psychometric, and Personality Research, CRAN repository, Date of Publication: 2019-12-20, 16:40:02, UTC.
Riordan, C. M., & Vandenberg, R. J. (1994). A central question in cross-cultural research: Do employees of different cultures interpret work-related measures in an equivalent manner? Journal of Management, 20, 643–671.
Rincon, R., & Korn, R. M., & Williams, J. C. (2019, June), Examining Gender Bias in Engineering in India Paper presented at 2019 ASEE Annual Conference & Exposition , Tampa, Florida.
Ramachandran, B., Ramanathan, C. & Khabou, M. (2020). Advancement of women in engineering: past, present and future. American Society for Engineering Education. advancement-of-women-in-engineering-pastpresent-and-future.pdf
Steiger, J. H. (1989) EzPATH: Causal Modeling. Evanston, IL:SYSTAT.
Steenkamp, J. E. M., & Baumgartner, H. (1998). Assessing measurement invariance in cross- national consumer research. Journal of Consumer Research, 25, 78–90.
Struening, E.L., Cohen, J. (1963). Factorial Invariance and Other Psychometric Characteristics of Five Opinions About Mental Illness Factors. Educational and Psychological Measurement. 1963;23(2):289-298. doi:10.1177/001316446302300206
Suzuki, S., & Rancer, A. S. (1994). Argumentativeness and verbal aggressiveness: Testing for conceptual and measurement equivalence across cultures. Communication Monographs, 6, 256–279.
Singh, J. (1995). Measurement issues in cross-national research. Journal of International Business Studies, 26, 597–619.
Tay, L., Meade, A.W., &Cao, M. (2015). An overview and practical guide to IRT measurement equivalence analysis. Organizational Research Methods, 18:3–46. DOI: 10.1177/1094428114553062.
Thompson, B. (2000). Ten commandments of structural equation modeling. In US Dept of Education, Office of Special Education Programs (OSEP) Project Directors' Conference, 1998, Washington, DC, US; A previous version of this chapter was presented at the aforementioned conference and at the same annual conference held in 1999. American Psychological Association.
Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4–69.
Widaman, KF.. & Reise, SP. (1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In: Bryant, KJ.; Windle, ME.; West, SG., editors. The Science of Prevention: Methodological Advances from Alcohol and Substance Abuse Research. Washington, DC: American Psychological Association;. p. 281-324.
Yockey, R.D., (2016). Validation of Short Form of Academic Procrastination Scale, Psychological Reports, 118(1), pp:171-179.
Zhu, J. (2004). Understanding volition, Philosophical Psychology, 17(2), pp:247-273, DOI: 10.1080/0951508042000239066
Zumbo, B. D., Gadermann, A. M., & Zeisser, C. (2007). Ordinal versions of coefficients alpha and theta for Likert rating scales. Journal of Modern Applied Statistical Methods, 6, 21-29.
Zinbarg, R.E., Revelle, W., Yovel, I., & Li. W. (2005). Cronbach's Alpha, Revelle's Beta, McDonald's Omega: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika. 70, 123-133. doi: 10.1007/s11336-003-0974-7.
Downloads
Published
How to Cite
Issue
Section
Access to login into the old portal (Manuscript Communicator) for Peer Review-
