Barbi Svetec and Blaženka Divjak
pp. 63 – 79, download
(https://doi.org/10.55612/s-5002-064-002sp)
Abstract
Even though the benefits of learning analytics (LA) have been recognized in research, there are still challenges to its more widespread adoption, which could contribute to the development of smart learning ecosystems beyond 2030. Trustworthiness of LA is an aspect which could play an important role in the adoption of LA, especially in the artificial intelligence (AI) era. Different dimensions of trustworthiness of LA have been researched, but the framework is still fuzzy. We conducted a scoping literature review to provide more clarity, and in this paper, we presented an overview of theoretical considerations and dimensions of trustworthiness of LA. We grouped the identified dimensions into social and technological aspects, and pointed to horizontal dimensions related to both. These dimensions were used as the basis for a comprehensive definition of the trustworthiness of LA. Finally, we identified the challenges and open questions related to using LA to support smart learning ecosystems.
Keywords: learning analytics, trustworthiness, trust, data, ethics, data security, algorithms, fairness .
References
1. ASLERD et al., Timisoara declaration: ‘BETTER LEARNING FOR A BETTER WORLD Through People Centered Smart Learning Ecosystems,’ 2016. Accessed: April 10, 2024. [Online]. Available: https://www.aslerd.org/summary-of-timisoara-declaration/ https://doi.org/10.55612/s-5002-064-001d)
2. Tsai Y. S., Whitelock-Wainwright A., and Gašević D.: More Than Figures on Your Laptop: (Dis)trustful Implementation of Learning Analytics, Journal of Learning Analytics, vol. 8, no. 3, pp. 81–100 (2021), doi: 10.18608/jla.2021.7379.
3. de Sousa E. B. G., Alexandre B., Ferreira Mello R., Pontual Falcão T., Vesin B., and Gašević D.: Applications of Learning Analytics in High Schools: A Systematic Literature Review, Frontiers in Artificial Intelligence, vol. 4 (2021) doi: 10.3389/frai.2021.737891.
4. Tsai Y.-S. and Gasevic D.: Learning analytics in higher education — challenges and policies, in Proceedings of the Seventh International Learning Analytics & Knowledge Conference, New York, NY, USA: ACM, pp. 233–242 (2017) doi: 10.1145/3027385.3027400.
5. Veljanova H., Barreiros C., Gosch N., Staudegger E., Ebner M., Lindstaedt S.: Towards Trustworthy Learning Analytics Applications: An Interdisciplinary Approach Using the Example of Learning Diaries, pp. 138–145 (2022) doi: 10.1007/978-3-031-06391-6_19.
6. Alzahrani A. S., Tsai Y. S., Aljohani N., Whitelock-wainwright E., Gasevic D.: Do teaching staff trust stakeholders and tools in learning analytics? A mixed methods study, vol. 71, no. 4. Springer US (2023) doi: 10.1007/s11423-023-10229-w.
7. de Fine Licht K. and Brülde B.: On Defining ‘Reliance’ and ‘Trust’: Purposes, Conditions of Adequacy, and New Definitions, Philosophia, vol. 49, no. 5, pp. 1981–2001 (2021), doi: 10.1007/s11406-021-00339-1.
8. Slade S., Prinsloo P., and Khalil M.: ‘Trust us,’ they said. Mapping the contours of trustworthiness in learning analytics, Information and Learning Science, vol. 124, no. 9–10, pp. 306–325 (2023), doi: 10.1108/ILS-04-2023-0042.
9. Rienties B., Køhler Simonsen H., and Herodotou C.: Defining the Boundaries Between Artificial Intelligence in Education, Computer-Supported Collaborative Learning, Educational Data Mining, and Learning Analytics: A Need for Coherence, Frontiers in Education, vol. 5, (2020), doi: 10.3389/feduc.2020.00128.
10. Chatti M. A., Dyckhoff A. L., Schroeder U., Thüs H.: A reference model for learning analytics, International Journal of Technology Enhanced Learning, vol. 4, no. 5/6, p. 318 (2012), doi: 10.1504/IJTEL.2012.051815.
11. ANP et al.: Troyes Declaration (Timisoara 2.0): Better Learning for a Better World 2.0: People Centred Smart Learning Ecosystems beyond 2030 (2024). Accessed: Oct. 1, 2024. [Online]. Available: https://www.aslerd.org/troyes-declaration/ https://doi.org/10.55612/s-5002-064-002d
12. E. Commission: Ethical Guidelines for Trustworthy AI (2019).
13. E. Commission: Ethical guidelines on the use of artificial intelligence (AI) and data in teaching and learning for educators, Brussels (2022) doi: 10.2766/153756.
14. Drachsler H. and Greller W.: Privacy and analytics – it’s a DELICATE issue a checklist for trusted learning analytics, ACM International Conference Proceeding Series, vol. 25-29-Apri, pp. 89–98 (2016), doi: 10.1145/2883851.2883893.
15. Sclater N.: Developing a Code of Practice for Learning Analytics, Journal of Learning Analytics, vol. 3, no. 1 (2016), doi: 10.18608/jla.2016.31.3.
16. Slade S., Prinsloo P., and Khalil M.: Learning analytics at the intersections of student trust, disclosure and benefit, ACM International Conference Proceeding Series, pp. 235–244 (2019), doi: 10.1145/3303772.3303796.
17. Tsai Y.-S. S. et al.: The SHEILA Framework: Informing Institutional Strategies and Policy Processes of Learning Analytics, Journal of Learning Analytics, vol. 5, no. 3, pp. 31–47 (2018), doi: 10.18608/jla.2018.53.2.
18. Slibar B., Gusic Mundjar J., Rako S., and Simic D.: Co-occurrence patterns of issues and guidelines related to ethics and privacy of learning analytics in higher education—literature review, in LAK22: 12th International Learning Analytics and Knowledge Conference, New York, NY, USA: ACM, pp. 577–582 (2022), doi: 10.1145/3506860.3506974.
19. Ladjal D., Joksimović S., Rakotoarivelo T., and Zhan C.: Technological frameworks on ethical and trustworthy learning analytics, British Journal of Educational Technology, vol. 53, no. 4, pp. 733–736 (2022), doi: 10.1111/bjet.13236.
20. Peters M. D. J., Godfrey C. M., Khalil H., McInerney P., Parker D., Soares C. B.: Guidance for conducting systematic scoping reviews, International Journal of Evidence-Based Healthcare, vol. 13, no. 3, pp. 141–146 (2015), doi: 10.1097/XEB.0000000000000050.
21. Munn Z., Peters M. D. J., Stern C., Tufanaru C., McArthur A., Aromataris E.: Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach, BMC Medical Research Methodology, vol. 18, no. 1, p. 143 (2018), doi: 10.1186/s12874-018-0611-x.
22. Wohlin C.: Guidelines for snowballing in systematic literature studies and a replication in software engineering, in Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering – EASE ’14, New York, New York, USA: ACM Press (2014) pp. 1–10. doi: 10.1145/2601248.2601268.
23. Slade S. and Prinsloo P.: Learning Analytics: Ethical Issues and Dilemmas, American Behavioral Scientist, vol. 57, no. 10, pp. 1510–1529 (2013), doi: 10.1177/0002764213479366.
24. Cerratto Pargman T. and McGrath C.: Mapping the Ethics of Learning Analytics in Higher Education: A Systematic Literature Review of Empirical Research, Journal of Learning Analytics, vol. 8, no. 2, pp. 123–139 (2021), doi: 10.18608/jla.2021.1.
25. E. Union: Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Da) (2016) Official Journal of the European Union.
26. E. Union: Artificial Intelligence Act (2024), doi: https://artificialintelligenceact.eu/the-act/.
27. Pardo A. and Siemens G.: Ethical and privacy principles for learning analytics, British Journal of Educational Technology, vol. 45, no. 3, pp. 438–450 (2014), doi: 10.1111/bjet.12152.
28. Prinsloo P., Slade S., and Khalil M.:The answer is (not only) technological: Considering student data privacy in learning analytics, British Journal of Educational Technology, vol. 53, no. 4, pp. 876–893 (2022), doi: 10.1111/bjet.13216.
29. Liu Q. and Khalil M.: Understanding privacy and data protection issues in learning analytics using a systematic review, British Journal of Educational Technology, vol. 54, no. 6, pp. 1715–1747 (2023), doi: 10.1111/bjet.13388.
30. Ifenthaler D. and Schumacher C.: Student perceptions of privacy principles for learning analytics, Educational Technology Research and Development, vol. 64, no. 5, pp. 923–938 (2016) doi: 10.1007/s11423-016-9477-y.
31. Jones K. M. L. et al.: Transparency and Consent: Student Perspectives on Educational Data Analytics Scenarios, portal: Libraries and the Academy, vol. 23, no. 3, pp. 485–515 (2023), doi: 10.1353/pla.2023.a901565.
32. Jones K. M. L., Rubel A., and LeClere E.: A matter of trust: Higher education institutions as information fiduciaries in an age of educational data mining and learning analytics, Journal of the Association for Information Science and Technology, vol. 71, no. 10, pp. 1227–1241 (2020), doi: 10.1002/asi.24327.
33. Adejo O. and Connolly T.: Learning Analytics in a Shared-Network Educational Environment: Ethical Issues and Countermeasures, International Journal of Advanced Computer Science and Applications, vol. 8, no. 4 (2017) doi: 10.14569/IJACSA.2017.080404.
34. Mutimukwe C., Viberg O., Oberg L., and Cerratto‐Pargman T.: Students’ privacy concerns in learning analytics: Model development, British Journal of Educational Technology, vol. 53, no. 4, pp. 932–951 (2022), doi: 10.1111/bjet.13234.
35. Rodríguez-Triana M. J., Prieto L. P., Martínez-Monés A., Asensio-Pérez J. I., Dimitriadis Y.: The teacher in the loop: Customizing multimodal learning analytics for blended learning, ACM International Conference Proceeding Series, pp. 417–426 (2018), doi: 10.1145/3170358.3170364.
36. Jones K. M. L. et al.: ‘We’re being tracked at all times’: Student perspectives of their privacy in relation to learning analytics in higher education, Journal of the Association for Information Science and Technology, vol. 71, no. 9, pp. 1044–1059 (2020), doi: 10.1002/asi.24358.
37. Li W., Sun K., Schaub F., Brooks C.: Disparities in Students’ Propensity to Consent to Learning Analytics, International Journal of Artificial Intelligence in Education, vol. 32, no. 3, pp. 564–608 (2022), doi: 10.1007/s40593-021-00254-2.
38. Prinsloo P. and Slade S.: Ethics and Learning Analytics: Charting the (Un)Charted, in Handbook of Learning Analytics, Society for Learning Analytics Research (SoLAR) pp. 49–57 (2017), doi: 10.18608/hla17.004.
39. Prinsloo P. and Slade S., Student privacy self-management, in Proceedings of the Fifth International Conference on Learning Analytics And Knowledge, New York, NY, USA: ACM, pp. 83–92 (2015), doi: 10.1145/2723576.2723585.
40. Rubel A. and Jones K. M. L.: Student privacy in learning analytics: An information ethics perspective, The Information Society, vol. 32, no. 2, pp. 143–159 (2016), doi: 10.1080/01972243.2016.1130502.
41. Tsai Y.-S., Perrotta C., and Gašević D.: Empowering learners with personalised learning approaches? Agency, equity and transparency in the context of learning analytics, Assessment & Evaluation in Higher Education, vol. 45, no. 4, pp. 554–567 (2020), doi: 10.1080/02602938.2019.1676396.
42. Buckingham Shum S., Ferguson R., and Martinez-Maldonado R.: Human-Centred Learning Analytics, Journal of Learning Analytics, vol. 6, no. 2 (2019), doi: 10.18608/jla.2019.62.1.
43. Selwyn N. and Gašević D., The datafication of higher education: discussing the promises and problems, Teaching in Higher Education, vol. 25, no. 4, pp. 527–540 (2020), doi: 10.1080/13562517.2019.1689388.
44. Council of Europe: Artificial Intelligence and Education (2022).
45. Alexandron G., Ruipérez-Valiente J. A., Lee S., Pritchard D. E.: Evaluating the Robustness of Learning Analytics Results Against Fake Learners, pp. 74–87 (2018), doi: 10.1007/978-3-319-98572-5_6.
46. Alexandron G., Yoo L. Y., Ruipérez-Valiente J. A., Lee S., Pritchard D. E.: Are MOOC Learning Analytics Results Trustworthy? With Fake Learners, They Might Not Be!, International Journal of Artificial Intelligence in Education, vol. 29, no. 4, pp. 484–506, (2019), doi: 10.1007/s40593-019-00183-1.
47. Deho O. B., Zhan C., Li J., Liu J., Liu L., Duy Le T.: How do the existing fairness metrics and unfairness mitigation algorithms contribute to ethical learning analytics?, British Journal of Educational Technology, vol. 53, no. 4, pp. 822–843 (2022), doi: 10.1111/bjet.13217.
48. Duan X., Pei B., Ambrose G. A., Hershkovitz A., Cheng Y., Wang C.: Towards transparent and trustworthy prediction of student learning achievement by including instructors as co-designers: a case study, Education and Information Technologies (2023)=, doi: 10.1007/s10639-023-11954-8.
49. Brdnik S., Podgorelec V., and Šumak B., Assessing Perceived Trust and Satisfaction with Multiple Explanation Techniques in XAI-Enhanced Learning Analytics, Electronics (Switzerland), vol. 12, no. 12 (2023), doi: 10.3390/electronics12122594.
50. Ciordas-Hertel G. P., Schneider J., Ternier S., Drachsler H.: Adopting trust in learning analytics infrastructure: A structured literature review Journal of Universal Computer Science, vol. 25, no. 13, pp. 1668–1686 (2019).
51. Taylor S. and Munguia P.: Towards a data archiving solution for learning analytics, in Proceedings of the 8th International Conference on Learning Analytics and Knowledge, New York, NY, USA: ACM, pp. 260–264 (2018), doi: 10.1145/3170358.3170415.
52. Hoel T., Griffiths D., and Chen W.: The influence of data protection and privacy frameworks on the design of learning analytics systems, in Proceedings of the Seventh International Learning Analytics & Knowledge Conference, New York, NY, USA: ACM, pp. 243–252 (2017), doi: 10.1145/3027385.3027414.
53. Chen W.: Learning analytics for inclusive higher education, in Proceedings 28th International Conference on Computers in Education , 2020, Asia-Pacific Society for Computers in Education, pp. 113–118 (2020).
54. Khalil M., Slade S., and Prinsloo P.: Learning analytics in support of inclusiveness and disabled students: a systematic review,” Journal of Computing in Higher Education (2023), doi: 10.1007/s12528-023-09363-4.
55. Kloos C. D. et al., H2O Learn – Hybrid and Human-Oriented Learning: Trustworthy and Human-Centered Learning Analytics (TaHCLA) for Hybrid Education, in 2022 IEEE Global Engineering Education Conference (EDUCON), IEEE, pp. 94–101 (2022), doi: 10.1109/EDUCON52537.2022.9766770.
56. Nazaretsky T., Ariely M., Cukurova M., and Alexandron G.: Teachers’ trust in AI‐powered educational technology and a professional development program to improve it,” British Journal of Educational Technology, vol. 53, no. 4, pp. 914–931 (2022), doi: 10.1111/bjet.13232.
57. Cruse D. A.: Lexical Semantics. Cambridge University Press (1986).
58. Savage N.: Synthetic data could be better than real data, Nature (2023), doi: 10.1038/d41586-023-01445-8.
59. Divjak B., Svetec B., and Horvat D.: Learning analytics dashboards: What do students actually ask for?, in LAK23: 13th International Learning Analytics and Knowledge Conference, New York, NY, USA: ACM, pp. 44–56 (2023), doi: 10.1145/3576050.3576141.