Hassan Bello, Nor Athiyah Abdullah
pp. 277 – 298, download
(https://doi.org/10.55612/s-5002-054-011)
Abstract
User satisfaction with most goods and services in the information system (IS) has been well studied. However, there appears to be little research on examinees’ satisfaction with facilities and services provided at computer-based test (CBT) centres, particularly when the centres are equipped and manned by outsourced vendors. In Nigeria, the CBT centres are public and private examination centres equipped to conduct the annual matriculation examination computer-based test. The study aims to investigate and empirically test the role of quality factors in predicting CBT satisfaction. The study also intends to test a mediator to explore further how some quality factors influence the examinee’s satisfaction. The research framework is based on the updated information system success model (ISSM) and is validated using a survey incorporating system quality, service quality, question content, the cost-effectiveness of CBT services, and CBT centre service satisfaction. A total of 455 surveys using structural equation modelling were analysed.
Keywords: Computer-based test, E-assessment, E-exam, CBT satisfaction, E-assessment quality factors.
References
1. Fagbola T. M., Adigun A. A., Oke A. O.: Computer-Based Test (Cbt) System For University Academic Enterprise Examination, International Journal of Scientific and Technology Research, vol. 2, no. 8, pp. 336-342, (2013)
2. Salah W., Ramadan M., Ahmed H.: Effectiveness of E-Assessment in Quantitative Modules, COVID-19 Consequences: A Case Study by the British University in Egyp, International Conference on Interactive Collaborative and Blended Learning, pp. 466-477, (2021)
https://doi.org/10.1007/978-3-030-67209-6_50
3. Elsalem L., Al-azzam N., Jum A. A., Obeidat N., Mahmoud A., Kheirallah K. A.: Stress and behavioral changes with remote E-exams during the Covid-19 pandemic: A cross-sectional study among undergraduates of medical sciences, Annals of Medicine and Surgery, vol. 60, pp, 271-279, (2020). https://doi.org/10.1016/j.amsu.2020.10.058
4. Nwankwo S. N., Nwosu N. N., Oputa R. E.: Administration of Computer-Based Testing (CBT) For University Admissions in Imo State: Problems and Prospects, Journal of Resourcefulness and Distinction, vol. 13, no. 1, pp. 24- 36, (2016)
5. Chen R.S., Hsieh K. K., Tsai C.: The implementation of face recognition technology and its effect on eâquiz credibility, Asian Journal on Quality, vol. 11, no. 2, pp. 125-136, (2010)
https://doi.org/10.1108/15982681011075934
6. Vairamuthu S., Anouncia S. M.: Reconnoitering Students’ Satisfaction of an Online Based Assessment System to Improve Usability using PSO: An Examination into a Problem Solving and Programming Course, Engineering, Technology and Applied Science Research, vol. 6, no. 5, pp. 1207-1211, (2016)
https://doi.org/10.48084/etasr.800
7. Dobre I.: Students’ Satisfaction Analysis Related To An E-Assessment System That Uses Natural Language Processing, The 11th International Scientific Conference eLearning and Software for Education, pp. 21-28 (2015) https://doi.org/10.12753/2066-026X-13-131
8. Bahati B., Fors U., Hansen P., Nouri J., Mukama E.: Measuring learner satisfaction with formative e-assessment strategies, International Journal of Emerging Technologies in Learning, vol. 14, no. 7, pp. 61-79, (2019) https://doi.org/10.3991/ijet.v14i07.9120
9. DeLone W. H., McLean E. R.: The DeLone and McLean model of information systems success: A ten-year update, Journal of Management Information Systems, vol. 19, no. 4, pp. 9-30, (2003).
https://doi.org/10.1080/07421222.2003.11045748
10. Lin J. W., Lai Y. C.: User acceptance model of computer-based assessment: Moderating effect of self-regulation, Australasian Journal of Educational Technology, vol. 35, no. 1, pp. 163-176, (2019), https://doi.org/10.14742/ajet.4684
11. Wang Y., So K. K. F., Sparks B. A.: Technology Readiness and Customer Satisfaction with Travel Technologies: A Cross-Country Investigation, Journal of Travel Research, vol. 56, no. 5, 563-577 (2017), https://doi.org/10.1177/0047287516657891
12. Ramayah T., Nejati M., Shafaei A.: Modeling predictors of blog stickiness and retention by university students, Malaysian Journal of Library and Information Science, vol. 20, no. 3, pp. 15-27, (2015)
13. Cheng Y. M.: Effects of quality antecedents on e-learning acceptance, Internet Research, vol. 22, no. 3, pp. 361-390, (2012)
https://doi.org/10.1108/10662241211235699
14. Shin D. H.: Effect of the customer experience on satisfaction with smartphones: Assessing smart satisfaction index with partial least squares, Telecommunications Policy, vol. 39, no. 8, pp. 627-641, (2015), https://doi.org/10.1016/j.telpol.2014.10.001
15. Hamilton S., Chervany N. L.: Evaluating information system effectiveness – Part I: Comparing evaluation approaches, MIS Quarterly: Management Information Systems, vol. 5, no. 3, pp. 55-69, (1981), https://doi.org/10.2307/249291
16. Bailey J. E., Pearson S. W.: Development of a Tool for Measuring and Analysing Computer User Satisfaction, Management Science, vol. 29, no. 5, pp. 530-545, (1983) https://doi.org/10.1287/mnsc.29.5.530
17. Fluck A., Adebayo O. S., Abdulhamid S. M.: Secure E-Examination Systems Compared: Case Studies from Two Countries, Journal of Information Technology Education: Innovations in Practice, vol. 16, pp. 107-125, (2017), https://doi.org/10.28945/3705
18. Sanni A. A., Mohammad M. F.: Computer-Based Testing (CBT): An Assessment of Student Perception of JAMB UTME in Nigeria, Computing, Information Systems, Development Informatics and Allied Research Journal, vol. 6, no. 2, pp. 1-16, (2015)
19. Mohammadi H.: Investigating users’ perspectives on e-learning: An integration of TAM and IS success model, Telematics and Informatics, vol. 32, no. 4, pp. 701-719, (2015) https://doi.org/10.1016/j.tele.2015.03.002
20. Seta H. Wati B., Muliawati T., A., Hidayanto A. N.: E-learning success model: An extension of delone and mclean IS’ success model, Indonesian Journal of Electrical Engineering and Informatics, vol. 6, no. 3, pp. 281~291, (2018) https://doi.org/10.11591/ijeei.v6i3.505
21. Hassanzadeh A., Kanaani F., Elahi S.: A model for measuring e-learning systems success in universities, Expert Systems with Applications, vol. 39, no. 12, pp. 10959-10966, (2012) https://doi.org/10.1016/j.eswa.2012.03.028
22. Kim M., Chang Y., Park M. C., Lee J.: The effects of service interactivity on the satisfaction and the loyalty of smartphone users, Telematics and Informatics, 32(4), 949-960, (2015) https://doi.org/10.1016/j.tele.2015.05.003
23. Economides A. A.: Computer adaptive testing quality requirements, World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, 288-295, (2005)
24. Kuikka M., Kitola M., Laakso M. J.: Challenges when introducing electronic exams, Research in Learning Technology, vol. 22, (2014) https://doi.org/10.3402/rlt.v22.22817
25. Loewenberger P., Bull J.: Cost-effectiveness analysis of computer-based assessment, Research in Learning Technology, vol. 11, no. 2, pp. 23-45, (2016) https://doi.org/10.1080/0968776030110204
26. Economides A. A., Roupas C.: Evaluation of Computer Adaptive Testing Systems, International Journal of Web-Based Learning and Teaching Technologies (IJWLTT), vol. 2, no. 1, pp. 70-87, (2007) https://doi.org/10.4018/jwltt.2007010104
27. Fluck A.: Implementation of on-campus digital examination practices, In Transforming Assessment in a Digital Era Revolutionising assessment approaches across Australian universities, pp. 1-8, (2013)
28. Kim S. C., Yoon D., Han E. K.: Antecedents of mobile app usage among smartphone users, Journal of Marketing Communications, vol. 22, no. 6, pp. 653-670, (2016) https://doi.org/10.1080/13527266.2014.951065
29. Adepoju O. A.: An Evaluation of the Challenges Encountered in the First General Computer Based Test in Nigeria, Pakistan Journal of Educational Research and Evaluation, vol. 1, no. 1, pp. 1-13, (2016)
30. Au N., Ngai E. W. T., Cheng T. C. E.: A critical review of end-user information system satisfaction research and a new research framework, Omega, vol. 30, no. 6, pp. 451-478, (2002) https://doi.org/10.1016/S0305-0483(02)00054-3
31. Petter S., DeLone W., McLean E.: Measuring information systems success: Models, dimensions, measures, and interrelationships, European Journal of Information Systems, vol. 17, no. 3, pp. 236-263, (2008) https://doi.org/10.1057/ejis.2008.15
32. Eiamkanchanalai S., Assarut N.: Service Quality and Satisfaction of Traditional and Technology-Enhanced Services, In Academy of Marketing Science Annual Conference, pp. 303-315, (2017) https://doi.org/10.1007/978-3-319-66023-3
33. Alzabi T., Rind M. M., Alenzi K., Solangi Z.: An Empirical Investigation Of Factors Influencing User Satisfaction With Online Services Of Mobile Communication Providers: Study of Kuwait, Sindh University Research Journal, vol. 48, no. 4D, pp. 103-109, (2016)
34. Noh Y., and Chang R.: A Study on the Factors Influencing the Satisfaction of Online Service System on the International Research Cooperation, International Journal of Knowledge Content Development and Technology, vol. 10, no. 2, pp. 71-90 (2020)
35. Subrahmanyam A.: Relationship between service quality, satisfaction, motivation, and loyalty, Quality Assurance in Education, vol. 25, no. 2, pp. 171-188 (2017) .https://doi.org/10.1108/QAE-04-2013-0016
36. Lee F. H., Wu W. Y.: Moderating effects of technology acceptance perspectives on e-service quality formation: Evidence from airline websites in Taiwan, Expert Systems with Applications, vol. 38, no. 6, pp. 7766-7773 (2011) https://doi.org/10.1016/j.eswa.2010.12.131
37. Lin J. S. C., Hsieh P. L.: The role of technology readiness in customers’ perception and adoption of self-service technologies, International Journal of Service Industry Management, vol. 17, no. 5, pp. 497-517 (2006) https://doi.org/10.1108/09564230610689795
38. Fang Y. H., Chiu C. M., Wang E. T. G.: Understanding customers’ satisfaction and repurchase intentions: An integration of IS success model, trust, and justice, Internet Research, vol. 21, no. 4, pp. 479-503 (2011), https://doi.org/10.1108/10662241111158335
39. Ramayah T., Lee J. W. C.: System characteristics, satisfaction, and e-learning usage: A structural equation model (SEM), Turkish Online Journal of Educational Technology, vol. 11, no. 2, pp. 196-206 (2012)
40. Al-Mamary Y. H., Shamsuddin A., Hamid N. A. A., Al-Maamari M. H., Adoption of management information systems in the context of Yemeni organisations: A structural equation modeling approach, Journal of Digital Information Management, vol. 13, no. 6, pp. 429-444 (2015)
41. Almarashdeh I.: Sharing instructors’ experience of learning management system: A technology perspective of user satisfaction in a distance learning course, Computers in Human Behavior, vol. 63, pp. 249-255 (2016) https://doi.org/10.1016/j.chb.2016.05.013
42. Kumbhar V.: Factors affecting on customers’ satisfaction an empirical investigation of ATM service, International Journal of Business Economics and Management Research, vol. 2, no. 3, pp. 144-156 (2011)
43. Atmojo J. T., Sudaryanto W. T., Widiyanto A., Ernawati A. D., Arradini D.: Telemedicine, Cost Effectiveness, and Patients Satisfactionâ¯: A Systematic Review. Journal of Health Policy and Management, Vol. 5, no. 2, pp, 103-107, (2020)
https://doi.org/10.26911/thejhpm.2020.05.02.02
44. Kumbhar V. M.: Factors affecting the customer satisfaction in e-banking: some evidence from Indian banks, Management Research and Practice, vol. 3, no. 4, pp. 1-14, (2011b)
45. Agarwal R., Prasad J.: Are individual differences germane to the acceptance of new information technologies?, Decis. Sci., vol. 30, no. 2, pp. 361-391 (1999) htstp://doi.org/10.1111/j.1540-5915.1999.tb01614.x
46. Venkatesh V., Morris M. G.: Why don’t men stop asking for directions? Gender , Social Influence and their role in society, MIS Q., vol. 24, no. 1, pp. 115-139 (2000) https://doi.org/10.2307/3250981
47. Sibona C., Choi J. H.: Factors Affecting End-User Satisfaction on Facebook, Artif. Intell., no. 1989, pp. 575-578 (2012)
48. Nunnally J.: Psychometric methods. McGraw- Hill, New York, NY. (1978)
49. Podsakoff P. M., MacKenzie S. B., Podsakoff N. P.: Sources of Method Bias in Social Science Research and Recommendations on How to Control It, Annual Review of Psychology, vol. 63, no. 1, pp. 539-569 (2012) https://doi.org/10.1146/annurev-psych-120710-100452
50. Nikou S. A., Economides A. A.: Mobile-based assessment: Investigating the factors that influence behavioral intention to use, Computers and Education, vol. 109, pp. 56-73, (2017) https://doi.org/10.1016/j.compedu.2017.02.005
51. Holsapple C. W., Lee-Post A.: Defining, Assessing, and Promoting E-Learning Success: An Information Systems Perspective, Decision Sciences Journal of Innovative Education, vol. 4, no. 1, pp. 67-85 (2006) https://doi.org/10.1111/j.1540-4609.2006.00102.x
52. Xu F., Du J. T.: Factors influencing users’ satisfaction and loyalty to digital libraries in Chinese universities, Computers in Human Behavior, vol. 83, pp. 64-72 (2018), https://doi.org/10.1016/j.chb.2018.01.029
53. Dang Y., Zhang Y., Ravindran S., Osmonbekov T.: Examining student satisfaction and gender differences in technology-supported, blended learning, Journal of Information Systems Education, vol. 27, no. 2, pp, 119-130 (2016)
54. Okocha F., Toluwani T. E., Owolabi S.: Student Perception and Acceptance of Computer-Based Testingâ¯: A Case Study of Landmark University Students, Journal of Digital Innovations and Contemp Res. In Sc., Eng and Tech, vol. 5, no. 1,pp. 25-32 (2017)
55. Hair J. F., Risher J. J., Sarstedt M., Ringle C. M.: When to use and how to report the results of PLS-SEM, In European Business Review, Vol. 31, no. 1, pp. 2-24 (2019), https://doi.org/10.1108/EBR-11-2018-0203
56. Hair J. F., Hult G. T. M., Ringle C. M., Sarstedt M.: A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM), SAGE. Los Angeles, CA, (2017)
57. Hair J. F., Ringle C. M., Sarstedt M.: PLS-SEM: Indeed, a silver bullet, J. Mark. Theory Pract., vol. 19, no. 2, pp. 139-151 (2011) https://doi.org/10.2753/MTP1069-6679190202
58. Hair J. F., Matthews L. M., . Matthews R. L, Sarstedt M.: PLS-SEM or CB-SEM: updated guidelines on which method to use, Int. J. Multivar. Data Anal., vol. 1, no. 2, p. 107 (2017) https://doi.org/10.1504/IJMDA.2017.087624
59. Fornell C., Larcker F. D.: Evaluating Structural Equation Models with Unobservable Variables and Measurement Error, Journal of Marketing Research, vol. 18, no. 1, pp. 39-50 (1981) https://doi.org/10.1177/002224378101800104
60. Henseler J., Ringle C. M., Sarstedt M.: A new criterion for assessing discriminant validity in variance-based structural equation modelling, Journal of the academy of marketing science, vol. 43, no. 1 pp. 115-135 (2015) https://doi.org/10.1007/s11747-014-0403-8
61. Kock N.: Common Method Bias in PLS-SEM: A Full Collinearity Assessment Approach, International Journal of E-Collaboration, vol. 11, no. 4, pp. 1-10 (2015) https://doi.org/10.4018/ijec.2015100101
62. Hoyer W. D., Herrmann A., Huber F.: When Buyers Also Sellâ¯: The Implications of Pricing Policies for Customer Satisfaction, Psychology and Marketing, vol. 19, no. 4, pp. 329-355 (2002) https://doi.org/10.1002/mar.10013
63. Zhou T.: An empirical examination of continuance intention of mobile payment services, Decision Support Systems, vol. 54, no. 2, pp. 1085-1091 (2013) https://doi.org/10.1016/j.dss.2012.10.034
64. Ramayah T., Ahmad N. H., Hong T. S.: An assessment of e-training effectiveness in multinational companies in Malaysia, Educational Technology and Society, vol. 15, no. 2, pp. 125-137 (2012)