Reliability Generalization of the Fear of COVID-19 Scale: A Meta-Analysis
Abstract
Since the first quarter of 2019, significant research efforts have been dedicated to curtail the spread of COVID-19. This study aimed to complement existing research by synthesizing Cronbach’s alpha coefficients and generalizing the Fear of COVID-19 Scale, with the goal of reducing the pandemic’s impact. Through a systematic literature search in electronic databases from 2019 to February 2021, we identified 2,753 works published in various sources, including journals, conference proceedings, books, and book chapters. Out of these, only 26 studies provided Cronbach’s alpha coefficients and were included in the meta-analysis. Employing a random-effects model, we analyzed the data and found that the Fear of COVID-19 Scale exhibited excellent internal consistency [α = 0.87 (95% CI 0.86–0.88)]. However, there was significant heterogeneity among the included studies. Despite this, the Fear of COVID-19 Scale demonstrated good internal consistency reliability. As the fight against COVID-19 continues, we encourage future psychometric studies of the Fear of COVID-19 Scale to report important characteristics of participants and details of item scores
Keywords: Cronbach’s alpha, fear of Covid-19 scale, meta-analysis, reliability generalization, random effect
References
[1] Ahorsu DK, Lin CY, Imani V, Saffari M, Griffiths MD, Pakpour AH. The Fear of COVID- 19 Scale: Development and Initial Validation. International Journal of Mental Health and Addiction. 2020;1: https://doi.org/10.1007/s11469-02000270-8
[2] Alyami M, Henning M, Krägeloh CU, Alyami H. Psychometric Evaluation of the Arabic Version of the Fear of COVID-19 Scale International Journal of Mental Health and Addiction. 2020; https://doi.org/10.1007/s11469-020-00316-x
[3] Appelbaum M, Cooper H, Kline RB, Mayo-Wilson E, Nezu AM, Rao SM. Journal article reporting standards for quantitative research in psychology: the APA Publications and Communications Board task force report. American Psychologist. 2018 Jan;73(1):3–25.
[4] Barai MK, Dhar S. COVID-19 Pandemic: Inflicted Costs and Some Emerging Global Issues. Global Business Review. 2021;0972150921991499: https://doi.org/10.1177/0972150921991499.
[5] Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics. 1994 Dec;50(4):1088–101.
[6] Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Publication Bias. Introduction to Meta-Analysis. John Wiley & Sons, Ltd.; 2009. https://doi.org/10.1002/9780470743386.ch30.
[7] Bosco FA, Aguinis H, Singh K, Field JG, Pierce CA. Correlational effect size benchmarks. J Appl Psychol. 2015 Mar;100(2):431–49.
[8] Broche-Pérez Y, Fernández-Fleites Z, Jiménez-Puig E, Fernández-Castillo E, Rodríguez-Martin BC. Gender and Fear of COVID-19 in a Cuban Population Sample. Int J Ment Health Addict. 2020; https://doi.org/10.1007/s11469-020-00343-8.
[9] Chang KC, Hou WL, Pakpour AH, Lin CY, Griffiths MD. Psychometric Testing of Three COVID-19-Related Scales Among People with Mental Illness. Int J Ment Health Addict. 2020; https://doi.org/10.1007/s11469-020-00361-6.
[10] Charter RA. A breakdown of reliability coefficients by test type and reliability method, and the clinical implications of low reliability. Journal of General Psychology. 2003 Jul;130(3):290–304.
[11] Cochran WG. The Combination of Estimates from Different Experiments. Biometrics. 1954;10(1):101–129.
[12] Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997 Sep;315(7109):629–634.
[13] Flynn BB, Schroederb RG, Sakakibara S. A framework for quality management research and an associated measurement instrument. Journal of Operations Management. 1994;11(4):339–366.
[14] Fornell C, Larcker DF. Evaluating Structural Equation Models with unobservable variables and measurement error. Journal of Marketing Research, 18, 39–50. http://eserv.uum.edu.my/login?url=http://search.ebscohost.com/login.aspx?direct=tr ue&db=bth&AN=5015357&site=ehost-live&scope=site García-Reyna, B., Castillo- García, G. D., Barbosa-Camacho, F. J., Cervantes-Cardona, G. 1981.
[15] Cervantes-Pérez A, Torres-Mendoza E, Fuentes-Orozco BM, Pintor C, Belmontes KJ, Guzmán-Ramírez BG, et al. Fear of COVID-19 Scale for Hospital Staff in Regional Hospitals in Mexico: a Brief Report. Int J Ment Health Addict. 2020;19: https://doi.org/10.1007/s11469-020-00413-x
[16] Ghebreyesus TA. (2020). WHO Director-General’s opening remarks at the media briefing on COVID-19 - 11 March 2020. https://www.who.int/directorgeneral/speeches/detail/who-director-general-sopening- remarks-at-the-mediabriefing-on-covid-19—11-march-2020
[17] Giordani RC, Ruiz Giolo, S Muhl C, Zanoni Da Silva M. Psychometric evaluation of the Portuguese version of the FCV-19 scale and assessment of fear of COVID-19 in a Southern Brazilian population. 2020. https://doi.org/10.1080/10911359.2020.1854142.
[18] Golafshani N. Understanding Reliability and Validity in Qualitative Research. Qual Rep. 2015; https://doi.org/10.46743/2160-3715/2003.1870.
[19] Hair JF, Black WC, Babin BJ, Anderson RE, Tatham RL, Black WC, et al. Multivariate Data Analysis. 7th ed. Prentice Hall; 2010.
[20] Haktanir A, Seki T, Dilmaç B. Adaptation and evaluation of Turkish version of the fear of COVID-19 Scale. Death Stud. 2020;:1–9.
[21] Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a metaanalysis. https://doi.org/10.1002/sim.1186.
[22] Statistics in Medicine, 21(11), 1539–1558. https://doi.org/https://doi.org/10.1002/sim. 1186
[23] Hunter JE, Schmidt FL. Methods of meta-analysis: Correcting error and bias in research findings. Methods of meta-analysis: Correcting error and bias in research findings. Sage Publications, Inc.; 1990.
[24] Isabelle F. OECD Interim Economic Assessment Coronavirus: The world economy at risk. 2020.
[25] Jacobson RK, Viswesvaran C. A Reliability Generalization Study of the Political Skill Inventory. SAGE Open. 2017;7(2): https://doi.org/10.1177/2158244017706714
[26] Kuder GF. Presenting a new journal. Educational and Psychological Measurement. 1941;1(1):3–4.
[27] Lakens D, Hilgard J, Staaks J. On the reproducibility of meta-analyses: six practical recommendations. BMC Psychology. 2016 May;4(1):24.
[28] Liang LH. (2020, March 9). How Covid-19 led to a nationwide work-from-home experiment - BBC Worklife. https://www.bbc.com/worklife/article/20200309coronaviruscovid- 19-advice-chinas-work-at-home-experiment
[29] Lord FM, Novick MR, Birnbaum A. Statistical theories of mental test scores. Statistical theories of mental test scores. Addison-Wesley; 1968.
[30] Lupia T, Scabini S, Mornese Pinna S, Di Perri G, De Rosa FG, Corcione S. 2019 novel coronavirus (2019-nCoV) outbreak: A new challenge. Journal of Global Antimicrobial Resistance. 2020 Jun;21:22–27.
[31] Moody’s Investors Service. Banks’ asset quality, profits impacted by Covid-19, Moody’s says | The Star. 2020 Feb 12. https://www.thestar.com.my/business/businessnews/2020/02/12/banks039-assetquality- profits-impacted-by-covid-19-moody039ssays
[32] Nunnally JC, Bernstein IH. No Title. 1991.
[33] Pang NTP, Kamu A, Hambali NLB, Mun HC, Kassim MA, Mohamed NH, Ayu F, Rahim SSSA, Omar A, Jeffree MS. Malay Version of the Fear of COVID-19 Scale: Validity and Reliability. International Journal of Mental Health and Addiction. 2020:1–10. https://doi.org/10.1007/s11469-020-00355-4
[34] Pearce B.. Economics Updated impact* assessment of the novel Coronavirus. 2020.
[35] Pedhazur EJ, Schmelkin LP. (1991). Measurement, design, and analysis: An integrated approach, Student ed. In Measurement, design, and analysis: An integrated approach, Student ed. Lawrence Erlbaum Associates, Inc.
[36] Perz CA, Lang BA, Harrington R. Validation of the Fear of COVID-19 Scale in a US College Sample. International Journal of Mental Health and Addiction. 2020:1–11.
[37] Prinsen CA, Mokkink LB, Bouter LM, Alonso J, Patrick DL, de Vet HC, et al. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018 May;27(5):1147–1157.
[38] R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing. 2018. Retrieved from. https://www.r-project.org/
[39] Riley RD, Higgins JP, Deeks JJ. Interpretation of random effects meta-analyses. BMJ. 2011 Feb;342 feb10 2:d549.
[40] Rodriguez MC, Maeda Y. Meta-analysis of coefficient alpha. Psychol Methods. 2006 Sep;11(3):306–322.
[41] Sakib N, Bhuiyan AK, Hossain S, Al Mamun F, Hosen I, Abdullah AH, et al. Psychometric Validation of the Bangla Fear of COVID-19 Scale: Confirmatory Factor Analysis and Rasch Analysis. International Journal of Mental Health and Addiction. 2020;1: https://doi.org/10.1007/s11469-020-00289-x
[42] Sánchez-Meca, J., López-Pina, J. A., López-López, J. A., Marín-Martínez, F., RosaAlcázar, A. I., & Gómez-Conesa, A. The Maudsley Obsessive-Compulsive Inventory: A reliability generalization meta-analysis. International Journal of Clinical and Health Psychology. 2011;11(3):473–493.
[43] Schmidt FL, Hunter JE. Development of a general solution to the problem of validity generalization. J Appl Psychol. 1977;62(5):529–540.
[44] Shereen MA, Khan S, Kazmi A, Bashir N, Siddique R. (2020). COVID-19 infection: Origin, transmission, and characteristics of human coronaviruses. In Journal of Advanced Research (Vol. 24, pp. 91–98). Elsevier BV. https://doi.org/10.1016/j.jare.2020.03.005.
[45] Singh B, Dhall R, Narang S, Rawat S. The Outbreak of COVID-19 and Stock Market Responses: An Event Study and Panel Data Analysis for G-20 Countries. Global Business Review. 2020;0972150920957274: https://doi.org/10.1177/0972150920957274
[46] Sohrabi C, Alsafi Z, O’Neill N, Khan M, Kerwan A, Al-Jabir A, et al. (2020). World Health Organization declares global emergency: A review of the 2019 novel coronavirus (COVID-19). In International Journal of Surgery (Vol. 76, pp. 71–76). Elsevier Ltd. https://doi.org/10.1016/j.ijsu.2020.02.034
[47] Sterne JA, Harbord RM. Funnel plots in meta-analysis. Stata J. 2004;4(2):127–141.
[48] Sterne JAC, Egger M. Regression methods to detect publication and other bias in meta-analysis. In: Rothstein HR, Sutton AJ, Borenstein M, editors. Publication bias in meta-analysis: Prevention, assessment and adjustment. Wiley; 2005. pp. 99–110. https://doi.org/https://doi.org/10.1002/0470870168.ch6
[49] Sterne Jonathan AC, Sutton AJ, Ioannidis JPA, Terrin N, Jones DR, Lau J, Carpenter J, Rücker G, Harbord RM, Schmid CH, Tetzlaff J, Deeks JJ, Peters J, Macaskill P, Schwarzer G, Duval S. Altman DG, Moher D, Higgins JPT. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomized controlled trials. BMJ (Online). 2011;343(7818). https://doi.org/10.1136/bmj.d4002
[50] Thompson B. Score reliability: Contemporary thinking on reliability issues. Sage; 2003. https://doi.org/10.4135/9781412985789
[51] Thompson B, Vacha-Haase T. Psychometrics is datametrics: the test is not reliable. Educational and Psychological Measurement. 2000;60(2):174–195.
[52] Vacha-Haase T. Reliability Generalization: Exploring Variance in Measurement Error Affecting Score Reliability Across Studies. Educational and Psychological Measurement. 1998;58(1):6–20.
[53] Viechtbauer W. Bias and efficiency of meta-analytic variance estimators in the random-effects model. Journal of Educational and Behavioral Statistics. 2005;30(3):261–293.
[54] Viechtbauer W. Conducting meta-analyses in R with the metafor package. Journal of Statistical Software. 2010;36(3):1–48. Available from: https://www.jstatsoft.org/v36/i03/
[55] Viechtbauer, Wolfgang, & Cheung, M. W.-L. (2010). Outlier and influence diagnostics for meta-analysis. Research Synthesis Methods, 1(2), 112–125. https://doi.org/https://doi.org/10.1002/jrsm.11
[56] Viswesvaran C, Ones DS. Perspectives on models of job performance. International Journal of Selection and Assessment 2000;8(4):216–226.
[57] Wakashima IdK, Asai K, Kobayashi D, Koiwa K, Kamoshida S, Sakuraba M. The Japanese version of the Fear of COVID-19 scale: Reliability, validity, and relation to coping behavior. 2020. https://doi.org/10.1371/journal.pone.0241958.
[58] Warne RT. An Investigation of Measurement Invariance Across Genders on the Overexcitability Questionnaire–Two. Journal of Advanced Academics. 2011;22(4):578–593.