Carlo Giovannella, Licia Cianfriglia, Antonello Giannelli
pp. 130 – 154, download
(https://doi.org/10.55612/s-5002-067-006)
Abstract
This contribution builds upon our previous work, in which we tested an integrated evaluation framework designed to assess smartness, well-being, and e-maturity in two high schools located in Rome. In the present study, we have extended the application of this framework to the entire Italian school ecosystem, involving a sample of principals, teachers, and parents in the evaluation process. The most significant outcome of this work is the development of a benchmark for learning ecosystems. This benchmark can serve as a reference for schools that wish to adopt the integrated framework in the future, enabling them to conduct self-assessments through a participatory evaluation process. The study also compared the perspectives of principals, teachers, and parents. The findings show relatively similar evaluations from parents and teachers, while principals tended to provide significantly more positive assessments. Using causal network analysis, we were able to identify the importance each respondent group assigned to the indicators used in evaluating the learning ecosystems. When combined with the insights gathered during the earlier high school evaluations—which also included student participation—these findings underscore the importance of involving all stakeholders in the educational process. Only through such inclusive, participatory evaluation can a truly meaningful assessment of a learning ecosystem be achieved, offering valuable guidance for its improvement.
Keywords: smart learning ecosystems evaluation, participatory evaluation, smartness, well-being, e-maturity, Italian schools’ ecosystem
References
1. https://www.miur.gov.it/documents/20182/2467413/Le+linee+guida.pdf/4e4bb411-1f90-
9502-f01e-d8841a949429?version=1.0&t =1593201965918 , retrieved on 07 April 2025 (in
Italian)
2. Key Competences for Lifelong Learning in the European Schools.
https://www.eursc.eu/BasicTexts/2018-09-D-69-en-2.pdf retrieved on 07 April 2025
3. https://invalsi-areaprove.cineca.it/index.php?get=static&pag=rapporti_invalsi retrieved on 07 April 2025 (in Italian)
4. https://invalsi-areaprove.cineca.it/index.php?get=static&pag=home_ocse retrieved on 07April 2025 (in Italian)
5. Warr P., Bird M., Rackham, N.: Evaluation of management training. London: Gower Press
(1970)
6. Kirkpatrick D.L.: Evaluation of training. New York: McGraw-Hill (1967)
7. Phillips J.: Accountability in Human Resource Management, Butterworth- Heinemann,
Oxford (1996)
8. Stufflebeam D.L., Zhang G.: The CIPP Evaluation Model: How to Evaluate for Im-provement
and Accountability, New York, Guilford Publications (2017)
9. Sergis S., Zervas P., Sampson D. G.: A Holistic Approach for Managing School ICT Com-
petence Profiles towards Supporting School ICT Uptake, International Journal of Digi-tal
Literacy and Digital Competence 5(4) pp. 33-46 (2014)
https://doi.org/10.4018/ijdldc.2014100103
10. Promoting Effective Digital-Age Learning: A European Framework for Digitally-Competent
Educational Organisations, Commissione Europea, Joint Research Centre © European Union
(2015) DOI:10.2791/54070 ISBN 978-92-79-54005-9
https://publications.jrc.ec.europa.eu/repository/bitstream/JRC98209/jrc98209_r_digcomporg_final.pdf
11. https:/publications.jrc.ec.europa.eu/repository/search?query=selfie, retrieved on 07 April 2025
12. Giovannella C., Cianfriglia L., Giannelli A.: A due anni dal lockdown: la percezione di insegnanti e dirigenti scolastici circa gli effetti provocati dallo shock digitale sul si-stema, Bricks N.5, pp. 109-131 (2022)
13. Giovannella C.: An integrated evaluation framework – smartness, well-being, e-maturity – for participatory evaluation of learning ecosystems: first application to two Italian high schools., Interaction Design & Architecture(s) – IxD&A Journal, N.62, 2024, pp. 98–122, https://doi.org/10.55612/s-5002-062-007
14. Giovannella C.: The ASLERD Pyramid of Smartness: A Study on the Stability of Indi-ces and Indicators in Schools in Project and Design Literacy as Cornerstones of Smart Education, Springer, pp. 81-91 (2020) https://doi.org/10.1007/978-981-13-9652-6
15. Giovannella C.: From simplex to complex: designing for wellbeing at scale, Interaction Design & Architecture(s) – IxD&A Journal, N.55, 2022, pp. 123–138, DOI: https://doi.org/10.55612/s-5002-055-006
16. Maslow A.H.: A theory of human motivation, Psychological Review, 50 (4) pp. 370–396.
(1943) https://doi.org/10.1037/h0054346
17. Czisikszentmihalyi M.: Flow – The Psychology of Optimal Experience. Harper & Row (1990)
18. Jolliffe, I.T.: Principal Component Analysis, Springer Series in Statistics, 2nd ed., Springer, NY (2002)
19. Hotelling, H.. Analysis of a complex of statistical variables into principal compo-nents. Journal of Educational Psychology, 24, 417–441, and 498–520 (1933)
https://doi.org/10.1037/h0070888
20. Giovannella C., Autiero M.: The school as a place learning ecosystem – Participatory evaluation of the boundary conditions: the case of the IIS Amaldi., Interaction Design & Architecture(s) – IxD&A Journal, N.60, 2023, pp. 59–84, DOI: https://doi.org/10.55612/s-5002-060-002
21. Giovannella C.: Participatory bottom-up self-evaluation of schools’ smartness: an Italian case study, Interaction Design & Architecture(s) – IxD&A Journal, N.31 pp. 9 – 18 (2016) https://doi.org/10.55612/s-5002-031-001