Carlo Giovannella
pp. 98 – 122, download
(https://doi.org/10.55612/s-5002-062-007)
Abstract
In this contribution, after a brief analysis of the critical issues associated with learning ecosystem evaluation frameworks, a participatory evaluation approach is proposed that aims to: (a) be people-centered and bring out the perceptions of all categories involved in educational processes – with particular reference to students, teachers and parents; this is in order to mitigate the objectivity bias associated with self-evaluations conducted by internal evaluation teams by means of top-down evaluation models; b) base the evaluation on an integrable set of multidimensional constructs such as smartness – capable of capturing both the “external” well-being associated with the quality of the context and the well-being perceived at an individual level – and e-maturity, i.e. the digital maturity of the context that defines its positioning (vision, plans and actions) with respect to the potential and opportunities offered by the digital transition; c) provide reference values with the intention of making evaluations of educational ecosystems both synchronically and diachronically comparable, as well as allowing for the elaboration of an adequate base of knowledge to facilitate the elaboration of improvement plans and to support evidence-based decision-making. The contribution includes also the description of a case study in which the participatory evaluation and the integrated evaluation framework were applied to two secondary schools in the city of Rome.
Keywords: smart learning ecosystems evaluation, participatory evaluation, smartness, well-being, e-maturity, high schools.
References
1. Paproth H., Clinton J.M., Aston R.: The Role of Evaluative Thinking in the Success of Schools as Community Hubs, in Cleveland, B., Backhouse, S., Chandler, P., McShane, I., Clinton, J.M., Newton, C. (eds) Schools as Community Hubs. Springer, Singapore, pp. 309-321 (2023) https://doi.org/10.1007/978-981-19-9972-7_21
2. Clinton J.M., Aston R., Paproth H.: An Evaluation Framework for Schools as Community Hubs, in Cleveland, B., Backhouse, S., Chandler, P., McShane, I., Clinton, J.M., Newton, C. (eds) Schools as Community Hubs. Springer, Singapore, pp. 293-308 (2023)
https://doi.org/10.1007/978-981-19-9972-7_20
3. https://www.miur.gov.it/documents/20182/2467413/Le+linee+guida.pdf/4e4bb411-1f90-9502-f01e-d8841a949429?version=1.0&t =1593201965918 , retrieved on 31 March 2024 (in Italian)
4. https://snv.pubblica.istruzione.it/snv-portale-web/public/scuole retrieved on 31 March 2024 (in Italian)
5. Key Competences for Lifelong Learning in the European Schools.
https://www.eursc.eu/BasicTexts/2018-09-D-69-en-2.pdf retrieved on 31 March 2024
6. https://invalsi-areaprove.cineca.it/index.php?get=static&pag=rapporti_invalsi retrieved on 31 March 2024 (in Italian)
7. https://invalsi-areaprove.cineca.it/index.php?get=static&pag=home_ocse retrieved on 31 March 2024 (in Italian)
8. Hogan L.R.: The Historical Development of Program Evaluation: Exploring Past and Present, Online Journal for Workforce Education and Development, vol 2, n. 4, art. 5, pp 1-13 (2007) https://opensiuc.lib.siu.edu/ojwed/vol2/iss4/5
9. Madaus G.F. & Kellaghan T.: Trends in standards in Great Britain and Ireland, in G. Austin & H. Garber (Eds.) The rise and fall of national test scores. New York: Academic Press (1982)
10. Stufflebeam D.L., Madaus G.F., Kellaghan T.: Evaluation models: Viewpoints on educational and human services evaluation, Boston, Kluwer Academic Publishers (2000)
11. Tyler R.W.: Educational benchmarks in retrospect: Educational change since 1915. Viewpoints, 510(1), pp. 11-31 (1975)
12. Phillips J.: Accountability in Human Resource Management, Butterworth- Heinemann, Oxford (1996)
13. Warr P., Bird M., Rackham, N.: Evaluation of management training. London: Gower Press (1970)
14. Kirkpatrick D.L.: Evaluation of training. New York: McGraw-Hill (1967)
15. Stufflebeam D.L., Zhang G.: The CIPP Evaluation Model: How to Evaluate for Improvement and Accountability, New York, Guilford Publications (2017)
16. Giovannella C., Cianfriglia L., Giannelli A.: A due anni dal lockdown: la percezione di insegnanti e dirigenti scolastici circa gli effetti provocati dallo shock digitale sul sistema, Bricks N.5, pp. 109-131 (2022)
17. Sergis S., Zervas P., Sampson D. G.: A Holistic Approach for Managing School ICT Competence Profiles towards Supporting School ICT Uptake, International Journal of Digital Literacy and Digital Competence 5(4) pp. 33-46 (2014)
18. Promoting Effective Digital-Age Learning: A European Framework for Digitally-Competent Educational Organisations, Commissione Europea, Joint Research Centre © European Union (2015) DOI:10.2791/54070 ISBN 978-92-79-54005-9
https://publications.jrc.ec.europa.eu/repository/bitstream/JRC98209/jrc98209_r_digcomporg_final.pdf
19. https:/publications.jrc.ec.europa.eu/repository/search?query=selfie, verificato il 31 marzo 2024
20. Desmet P. MA. Pohlmeyer A. E.: Positive design: An introduction to design for subjective well-being. International journal of design 7, 3 (2013) available on-line:
https:/repository.tudelft.nl/islandora/object/ uuid%3A06ec60ac 0363-43ea-9ccd-8426ef0d6b64
21: Czisikszentmihalyi M.: Flow – The Psychology of Optimal Experience. Harper & Row (1990)
22. Giovannella C.: The ASLERD Pyramid of Smartness: A Study on the Stability of Indices and Indicators in Schools in Project and Design Literacy as Cornerstones of Smart Education, Springer, pp. 81-91 (2020) https://doi.org/10.1007/978-981-13-9652-6
23. Jolliffe, I.T.: Principal Component Analysis, Springer Series in Statistics, 2nd ed., Springer, NY (2002)
24. Hotelling, H.. Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24, 417–441, and 498–520 (1933)
25. Giovannella C.: Participatory bottom-up self-evaluation of schools’ smartness: an Italian case study, Interaction Design & Architecture(s) Journal – IxD&A, 31, pp. 9–18 (2016) https://doi.org/10.55612/s-5002-031-001
26. Galego D., Giovannella C., Mealha O.: An investigation of actors’ differences in the perception of learning ecosystems’ smartness: the case of the Aveiro University,Interaction Design & Architecture(s) Journal – IxD&A, N. 31, pp. 19-31 (2016)
https://doi.org/10.55612/s-5002-031-002
27. Giovannella C., Cianfriglia L. and Giannelli A.: The Italian School Ecosystems two years after the lockdown: an overview on the “digital shock” triggered by the pandemic in the perceptions of schools’ principals and teachers, in “Polyphonic construction of Smart Learning Ecosystems”, Springer, pp. 47-76 (2022)
28. Giovannella C., Smartness as complex emergent property of a process. The case of learning eco-systems. ICWOAL 2014, IEEE publisher, pp. 1-5 (2014)
29. Giovannella C., Andone D., Dascalu M., Popescu E., Rehm M., Roccasalva G.: Smart-ness of Learning Ecosystems and its bottom-up emergence in six European Campuses, Interaction Design and Architecture(s) Journal – IxD&A, 27, pp. 79-92 (2015) https://doi.org/10.55612/s-5002-027-005
30. Giovannella C., Passarelli M., Persico D.: The Effects of the Covid-19 Pandemic on Italian Learning Ecosystems: the School Teachers’ Perspective at the steady state, Interaction Design and Architecture(s) Journal – IxD&A, N. 45, pp. 264 – 286 (2020) https://doi.org/10.55612/s-5002-045-012
31. Giovannella C. and Passarelli M.: The effects of the Covid-19 pandemic seen through the lens of the Italian university teachers and the comparison with school teachers’ perspective, Interaction Design and Architecture(s) Journal – IxD&A, 46, pp. 120 – 136 (2020) https://doi.org/10.55612/s-5002-046-006
32. Giovannella C., Passarelli M., Alkhafaji A., Peña Pérez Negrón A.: A Model for the Attitude to get Engaged in Technological Innovation (MAETI) derived from a comparative study on the effects of the SARS-CoV2 pandemic seen through the lens of the university teachers of three different national learning ecosystems: Iraq, Italy and Mexico, Interaction Design and Architecture(s) Journal – IxD&A, 47, pp. 167 – 190 (2021) https://doi.org/10.55612/s-5002-047-008
33. Alkhafaji A., Mshali H., Passarelli M. Giovannella C.: Reaction of the university ecosystem to the pandemics in a mid-east country: the case of IRAQ – A compared analysis of students’ and teachers’ perceptions, in “Polyphonic construction of Smart Learning Ecosystems”, Springer, pp. 205-225 (2022)
34. Kalisch M., Maechler M., Colombo D., Maathuis M.H., Buehlmann P.: Causal
Inference Using Graphical Models with the R Package pcalg. Journal of Statistical
Software 47(11), pp. 1–26 (2012) http://www.jstatsoft.org/v47/i11/
35. Giovannella C.: Participatory evaluation as starting point to design for smarter learning ecosystems: the UTOV case history, in “Citizen, Territory and Technologies: Smart Learning Contexts and Practices”, Springer publisher, pp. 64-74 (2017)
36. Meahla O., Giovannella C., Delgado F.: School Smartness augmented by educational community members: a pilot contribution from K9 students, in D’Andrea Fabio and Baldi Vania (eds.) “Codice e luoghi. Abitare le relazioni nel reale/digitale”, Roma, Meltemi Editore, pp. 143-164 (2019)