Fábio Silva, João Ramos, Cesar Analide
pp. 191 – 212, download
(https://doi.org/10.55612/s-5002-053-010)
Abstract
Virtual reality and augmented reality have the potential to enhance and widespread practical learning environments in professional courses efficiently in a cost-efficient manner by limiting the costs of real resources substituting them with fixed costs from.VR/AR applications with virtual resources. There are advantages in the learning process, as practical, active and visual learning methods are more efficient and virtual and augmented reality can digitalize these procedures and replicate them at scale with different degrees of virtualization. In this work we aim to provide a framework that allows the creation of VR and AR experiences for learning or training proposes in a serious environment adding gamification elements to keep user engaged in the learning/training process. In the process gamification adaptation to VR/AR environment is demonstrated in real applications. The learning tasks in this approach are not necessarily changed or take advantage of new forms of interactions and guidance but aim to be replicated in a blend between virtual and real environments. In this regard, we hope to advance gamification application to account for more elements, such as VR/AR interaction, digital twins and digital aids in a learning application. In this article we detail possible scenarios for the application of virtual reality and augmented reality combined with machine learning in serious games and learning scenarios.
Keywords: Gamification, Virtual Reality, Augmented Reality.
References
1. M. Prensky, “Digital Natives, Digital Immigrants Part 2: Do They Really Think Differently?,” Horiz., vol. 9, no. 6, pp. 1–6, Nov. 2001, https://doi.org/10.1108/10748120110424843.
2. Z. Genc, “Parents’ Perceptions about the Mobile Technology Use of Preschool Aged Children,” Procedia – Soc. Behav. Sci., vol. 146, no. 2, pp. 55–60, 2014, https://doi.org/10.1016/j.sbspro.2014.08.086.
3. V. Sehnalová, “Using ICT in Education of Preschool Children,” J. Technol. Inf. Educ., vol. 6, no. 1, pp. 4–18, 2014, https://doi.org/10.5507/jtie.2014.001.
4. J. Lepicnik-Vodopivec and P. Samec, “Advantages and disadvantages of information-communication technology usage for four-year-old children, and the consequences of its usage for the children’s development,” Int. J. Humanit. Soc. Sci., 2012.
5. G. Brewer and J. Kerslake, “Cyberbullying, self-esteem, empathy and loneliness,” Comput. Human Behav., vol. 48, pp. 255–260, 2015, https://doi.org/10.1016/j.chb.2015.01.073.
6. A. Lieberoth, “Shallow Gamification: Testing Psychological Effects of Framing an Activity as a Game,” Games Cult., vol. 10, no. 3, pp. 229–248, May 2015, https://doi.org/10.1177/1555412014559978.
7. F. Silva and C. Analide, “Computational sustainability and the PHESS platform: Using affective computing as social indicators,” Futur. Gener. Comput. Syst., vol. 92, pp. 329–341, Mar. 2019, https://doi.org/10.1016/j.future.2018.10.006.
8. I. Blohm and J. M. Leimeister, “Gamification – Design of IT-Based Enhancing Services for Motivational Support and Behavioral Change,” Bus. Inf. Syst. Eng., vol. 5, no. 4, pp. 275–278, Aug. 2013, https://doi.org/10.1007/s12599-013-0273-5.
9. J. Hamari, J. Koivisto, and H. Sarsa, “Does Gamification Work? — A Literature Review of Empirical Studies on Gamification,” in 2014 47th Hawaii International Conference on System Sciences, 2014, pp. 3025–3034, https://doi.org/10.1109/HICSS.2014.377.
10. A. Vanolo, “Cities and the politics of gamification,” Cities, vol. 74, pp. 320–326, 2017, https://doi.org/10.1016/j.cities.2017.12.021.
11. R. Spitz, C. P. Junior, F. Queiroz, L. C. Leite, P. Dam, and A. C. Rezende, “Gamification, citizen science, and civic technologies: In search of the common good,” Strateg. Des. Res. J., vol. 11, no. 3, pp. 263–273, 2018, https://doi.org/10.4013/sdrj.2018.113.11.
12. A. M. Toda et al., “How to gamify learning systems? An experience report using the design sprint method and a taxonomy for gamification elements in education,” Educ. Technol. Soc., vol. 22, no. 3, pp. 47–60, 2019.
13. J. Majuri, J. Koivisto, and J. Hamari, “Gamification of education and learning: A review of empirical literature,” 2018.
14. K. Robson, K. Plangger, J. H. Kietzmann, I. McCarthy, and L. Pitt, “Is it all a game? Understanding the principles of gamification,” Bus. Horiz., vol. 58, no. 4, pp. 411–420, 2015, https://doi.org/10.1016/j.bushor.2015.03.006.
15. E. L. Deci and R. M. Ryan, “Self-Determination Theory,” in Handbook of Theories of Social Psychology: Volume 1, 1 Oliver’s Yard, 55 City Road, London EC1Y 1SP United Kingdom: SAGE Publications Ltd, 2012, pp. 416–437.
16. G. F. Tondello, R. R. Wehbe, L. Diamond, M. Busch, A. Marczewski, and L. E. Nacke, “The Gamification User Types Hexad Scale,” in Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play, Oct. 2016, pp. 229–243, https://doi.org/10.1145/2967934.2968082.
17. Bartle Richard, “Hearts, Clubs, Diamonds, Spades: Players Who Suit MUDS,” J. MUD Res., vol. 1, no. 1, p. 19, 1996.
18. C. J. Costa, M. Aparicio, S. Aparicio, and J. T. Aparicio, “Gamification usage ecology,” SIGDOC 2017 – 35th ACM Int. Conf. Des. Commun., 2017, https://doi.org/10.1145/3121113.3121205.
19. Y. Chow, “The Octalysis Framework for Gamification & Behavioral Design,” 2021. https://yukaichou.com/gamification-examples/octalysis-complete-gamification-framework/.
20. R. S. Kalawsky, The science of virtual reality and virtual environments. 1993.
21. M. Slater and S. Wilbur, “A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments,” Presence Teleoperators Virtual Environ., vol. 6, no. 6, pp. 603–616, Dec. 1997, https://doi.org/10.1162/pres.1997.6.6.603.
22. P. Milgram, H. Takemura, A. Utsumi, and F. Kishino, “Augmented reality: a class of displays on the reality-virtuality continuum,” in Telemanipulator and Telepresence Technologies, Dec. 1995, vol. 2351, pp. 282–292, https://doi.org/10.1117/12.197321.
23. R. T. Azuma, “A Survey of Augmented Reality,” Presence Teleoperators Virtual Environ., vol. 6, no. 4, pp. 355–385, Aug. 1997, doi: 10.1162/pres.1997.6.4.355.
24. R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier, and B. MacIntyre, “Recent advances in augmented reality,” IEEE Comput. Graph. Appl., vol. 21, no. 6, pp. 34–47, 2001.
25. A. Javornik, “Augmented reality: Research agenda for studying the impact of its media characteristics on consumer behaviour,” J. Retail. Consum. Serv., vol. 30, pp. 252–261, May 2016, https://doi.org/10.1016/j.jretconser.2016.02.004.
26. M. Billinghurst, A. Clark, and G. Lee, “A Survey of Augmented Reality,” Found. Trends® Human–Computer Interact., vol. 8, no. 2–3, pp. 73–272, 2015, https://doi.org/10.1561/1100000049.
27. E. Klopfer, Augmented Learning: Research and Design of Mobile Educational Games. 2008.
28. E. Klopfer and K. Squire, “Environmental Detectives—the development of an augmented reality platform for environmental simulations,” Educ. Technol. Res. Dev., vol. 56, no. 2, pp. 203–228, 2008, https://doi.org/10.1007/s11423-007-9037-6.
29. C. Portalés, J. L. Lerma, and S. Navarro, “Augmented reality and photogrammetry: A synergy to visualize physical and virtual city environments,” ISPRS J. Photogramm. Remote Sens., vol. 65, no. 1, pp. 134–142, Jan. 2010, https://doi.org/10.1016/j.isprsjprs.2009.10.001.
30. R. Brunelli, Template Matching Techniques in Computer Vision: Theory and Practice. Wiley, 2009.
31. R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation,” in 2014 IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2014, pp. 580–587, https://doi.org/10.1109/CVPR.2014.81.
32. R. Girshick, “Fast R-CNN,” in 2015 IEEE International Conference on Computer Vision (ICCV), Dec. 2015, pp. 1440–1448, https://doi.org/10.1109/ICCV.2015.169.
33. S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 6, pp. 1137–1149, Jun. 2017, https://doi.org/10.1109/TPAMI.2016.2577031.
34. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2016, pp. 779–788, https://doi.org/10.1109/CVPR.2016.91.
35. J. Redmon and A. Farhadi, “YOLO9000: Better, Faster, Stronger,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul. 2017, pp. 6517–6525, https://doi.org/10.1109/CVPR.2017.690.
36. J. Redmon and A. Farhadi, “YOLOv3: An Incremental Improvement,” Apr. 2018, https://doi.org/10.48550/arXiv.1804.02767.
37. A. B. Nassif, I. Shahin, I. Attili, M. Azzeh, and K. Shaalan, “Speech Recognition Using Deep Neural Networks: A Systematic Review,” IEEE Access, vol. 7, pp. 19143–19165, 2019, https://doi.org/10.1109/ACCESS.2019.2896880.
38. J. Y. Chang, G. Moon, and K. M. Lee, “V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Map,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 2018, pp. 5079–5088, https://doi.org/10.1109/CVPR.2018.00533.
39. X. Du et al., “SpineNet: Learning Scale-Permuted Backbone for Recognition and Localization,” Dec. 2019, https://doi.org/10.48550/arXiv.1912.05027.
40. J. Talukdar, S. Gupta, P. S. Rajpura, and R. S. Hegde, “Transfer Learning for Object Detection using State-of-the-Art Deep Neural Networks,” in 2018 5th International Conference on Signal Processing and Integrated Networks (SPIN), Feb. 2018, pp. 78–83, https://doi.org/10.1109/SPIN.2018.8474198.
41. X. Bu, J. Peng, J. Yan, T. Tan, and Z. Zhang, “GAIA: A Transfer Learning System of Object Detection that Fits Your Needs,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2021, pp. 274–283, https://doi.org/10.1109/CVPR46437.2021.00034.
42. B. G. Witmer and M. J. Singer, “Measuring Presence in Virtual Environments: A Presence Questionnaire,” Presence Teleoperators Virtual Environ., vol. 7, no. 3, pp. 225–240, Jun. 1998, https://doi.org/10.1162/105474698565686.
43. S. Stumpp, T. Knopf, and D. Michelis, “User experience design with augmented reality (AR),” in Proceedings of the European Conference on Innovation and Entrepreneurship, ECIE, 2019, vol. 2, pp. 1032–1040, https://doi.org/10.34190/ECIE.19.019.
44. K. Tenório et al., “An Evaluation of the GamAnalytics Tool: Is the Gamification Analytics Model Ready for Teachers?,” in Anais do XXXI Simpósio Brasileiro de Informática na Educação (SBIE 2020), Nov. 2020, no. Cbie, pp. 562–571, https://doi.org/10.5753/cbie.sbie.2020.562.
45. D. A. Bowman and L. F. Hodges, “Evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments,” in Proceedings of the Symposium on Interactive 3D Graphics, 1997, pp. 35–38, https://doi.org/10.1145/253284.253301.
46. W. Liu et al., “SSD: Single Shot MultiBox Detector,” Lect. Notes Comput. Sci., pp. 21–37, 2016, https://doi.org/10.1007/978-3-319-46448-0_2.
47. F. Silva, R. Ferreira, A. Castro, P. Pinto, and J. Ramos, “Experiments on Gamification with Virtual and Augmented Reality for Practical Application Learning,” in Lecture Notes in Networks and Systems, 2022, vol. 326, pp. 175–184.
48. B. G. Witmer and M. J. Singer, “Measuring Presence in Virtual Environments: A Presence Questionnaire,” Presence Teleoperators Virtual Environ., vol. 7, no. 3, pp. 225–240, 1998, https://doi.org/10.1162/105474698565686.