David S. Bowers and Marian Petre
pp. 97 – 114, download
(https://doi.org/10.55612/s-5002-057-006)
Abstract
As part of the UK Institute of Coding (IoC), established to address the perceived skills gap between computing graduate capabilities and employer needs, a degree accreditation standard was developed which focused on the demonstration of real-world computing competence rather than on traditional academic knowledge. A graduate’s competence was to be assessed by using a portfolio recording their achievements in a real-world setting to map those achievements to the skills descriptors of a global skills framework for IT professionals. This paper explores the decisions made as the accreditation standard was developed, and the range of tests and checks that were required to ensure that the standard would both confirm graduates’ specific competence and meet national regulatory frameworks for degrees. Although the specific accreditation standard was for computing degrees, the design approach and subsequent validations would apply also to competence-based standards for other subjects.
Keywords: Graduate competence, professional skills framework, portfolio assessment, degree accreditation standard, SFIA.
References
1 ISO. Software and systems engineering — Certification of software and systems engineering professionals — Part 1: General requirements. Standard. International Organization for Standardization, Geneva, Switzerland. ISO/IEC 24773-1 https://www.iso.org/standard/69724.html (2019)
2 Wentz, D.K. and Ford, C.V.: A brief history of the internship. JAMA, Dec 28, 252 (24), pp.3390-4. PMID: 6389916 https://doi.org/10.1001/jama.252.24.3390 (1984)
3 Bowers, D. S and Sabin, M.: Demonstrating the use of a professional skills framework to support the assessment of dispositions in IT education. Education and Information Technologies DOI: https://doi.org/10.1007/s10639-023-11933-z (2023)
4 Encyclopedia Britannica: https://www.britannica.com/topic/apprenticeship (2021)
5 UK Department of Education: Teachers’ standards: How should they be used? https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/283567/Teachers_standards_how_should_they_be_used.pdf (2014)
6 Flexner, A.: Medical Education in the United States and Canada: A Report to the Carnegie Foundation for the Advancement of Teaching. http://archive.carnegiefoundation.org/publications/pdfs/elibrary/Carnegie_Flexner_Report.pdf (1910)
7 Gerkman, A., and Cornett, L.: Foundations for practice: The whole lawyer and the character quotient. AccessLex Institute Research Paper No. 16-04, http://dx.doi.org/10.2139/ssrn.2823835 (2016)
8 Joynes, C., Rossignoli, S., and Amonoo-Kuofi, E.F.: 21st century skills: evidence of issues in definition, demand and delivery for development contexts. (K4D Helpdesk Report). Institute of Development Studies, Brighton, UK https://opendocs.ids.ac.uk/opendocs/handle/20.500.12413/14674 (2019)
9 Chalkiadaki, A.: A systematic literature review of 21st century skills and competencies in primary education. International Journal of Instruction 11(3), pp. 1–16 https://doi.org/10.12973/iji.2018.1131a (2018)
10 Voogt, J., and Roblin, N.P.: A comparative analysis of international frameworks for 21st century competences: Implications for national curriculum policies. Journal of Curriculum Studies, 4444 (3), pp. 299–321 https://doi.org/10.1080/00220272.2012.668938 (2012)
11 Unesco: Key competencies 2030, https://www.humantific.com/post/unesco-education-report (2020)
12 EU Council: Recommendation on Key Competencies for Lifelong Learning, https://education.ec.europa.eu/focus-topics/improving-quality/key-competences (2018)
13 Bloom, B.S.: Taxonomy of educational objectives: The classification of educational goals. Longman, New York (1956)
14 Krathwohl, D.R.: A Revision of Bloom’s Taxonomy: An Overview, Theory Into Practice, 41:4, 212-218, https://doi.org/10.1207/s15430421tip4104_2 (2002)
15 Bowers, D., Petre, M., and Howson, O.: Aligning Competence Hierarchies with Bloom’s Taxonomies: Changing the focus for computing education. Koli Calling’19. ACM, 21-24 https://doi.org/10.1145/3364510.3366147 .(2019)
16 Simpson, E.J.: The classification of educational objectives, psychomotor domain. https://files.eric.ed.gov/fulltext/ED010368.pdf (1966)
17 Miller, G.E.: The assessment of clinical skills/competence/performance. Academic Medicine, 65 (9), pp. S63–S67 https://doi.org/10.1097/00001888-199009000-00045 (1990)
18 BCS The Chartered Institute for IT: Guidelines on Course Accreditation. https://www.bcs.org/media/1209/accreditation-guidelines.pdf (2020)
19 The Quality Assurance Agency for Higher Education: The Frameworks for Higher Education Qualifications of UK Degree-Awarding Bodies. Annex D: Outcome classification descriptions, https://www.qaa.ac.uk/docs/qaa/quality-code/annex-d-outcome-classification-descriptions-for-fheq-level-6-and-fqheis-level-10-degrees.pdf (2014)
20 Sabin, M., Alrumaih, H., Impagliazzo, J., Lunt, B., Zhang, M., Byers, B., Newhouse, W., Paterson, B., Peltsverger, S., Tang, C., van der Veer, G., and Viola, B.: IT2017, ACM/IEEE, New York https://it2017.acm.org/ (2017)
21 Sabin, M., Alrumaih, H., and Impagliazzo, J.: A competency-based approach toward curricular guidelines for information technology education. UbL. 2018 IEEE Global Engineering Education Conference (EDUCON), Santa Cruz de Tenerife, Spain, pp. 1214–1221, https://doi.org/10.1109/EDUCON.2018.8363368 (2018)
22 Perkins, D.N., Jay, E., and Tishman, S.: Beyond abilities: A dispositional theory of thinking. Merrill-Palmer Quarterly, 39 (1), pp. 1–21 (1993)
23 Clear, A., Parrish, A., Ciancarini, P., Frezza, S., Gal-Ezer, J., Impagliazzo, J., Pears, A., Takada, S., Topi, H., van der Veer, G., Vichare, A., Waguespack, L., Wang, P., and Zhang, M.: Computing Curricula 2020 (CC2020): Paradigms for Future Computing Curricula. ACM/IEEE, NewYork, https://www.acm.org/binaries/content/assets/education/curricula-recommendations/cc2020.pdf (2020)
24 Leidig, P., Salmel, H.D., Anderson, G., Babb, J., de Villiers, C., Gardner, L., Nunamaker, J.F. Jr, Scholtz, B., Shahkararaman, V., Sooriamurthi, R., and Thouin, M.: IS2020. A Competency Model for Undergraduate Programs in Information Systems. ACM, New York, https://www.acm.org/binaries/content/assets/education/curricula-recommendations/is2020.pdf (2020)
25 Danyluk, A., and Leidig, P.: Computing competencies for undergraduate data science curricula: ACM Data Science Task Force. ACM, New York, ISBN: 978-1-4503-9060-6, https://doi.org/10.1145/3453538 , https://dl.acm.org/citation.cfm?id=3453538 (2021)
26 Raj, R.K., Sabin, M., Impagliazzo, J., Bowers, D.S., Daniels, M., Hermans, F., Kiesler, N., Kumar, A.N., MacKellar, B., McCauley, R., Nabi, S.W., and Oudshoorn, M.: Professional competencies in computing education: Pedagogies and assessment. In: ITiCSE WGR ’21, pp. 133–161. ACM, New York, https://doi.org/10.1145/3502870.3506570 (2021)
27 Brauer, S.: Towards competence-oriented higher education: a systematic literature review of the different perspectives on successful exit profiles, Education and Training 63(9), 1376-1390, https://doi.org/10.1108/ET-07-2020-0216 (2021)
28 Bergsmann, E., Schultes, M., Winter, P., Schober, B., Spiel, C.: Evaluation of competence-based teaching in higher education: From theory to practice, Evaluation and Program Planning, (52) 1-9, https://doi.org/10.1016/j.evalprogplan.2015.03.001 (2015)
29 National Research Council: Education for Life and Work: Developing Transferable Knowledge and Skills in the 21st Century. The National Academies Press, Washington, DC https://nap.nationalacademies.org/catalog/13398/ (2012)
30 Fink, L.D: Taxonomy of Significant Learning. Jossey-Bass, San Francisco, CA (2003)
31 Hossein Abad, Z.S., Bano, M., and Zowghi, D.: How much authenticity can be achieved in software engineering project based courses? ICSE-SEET ’19, pp. 208-219. IEEE Press, https://doi.org/10.1109/ICSE-SEET.2019.00030 (2019)
32 McDermott, R., Zarb, M. Daniels, M., and Isomöttönen, V.: First year computing students’ perceptions of authenticity in assessment. ITiCSE ’17, pp. 10-15. ACM, New York, https://doi.org/10.1145/3059009.3059062 (2017)
33 Barber, W., King, S., and Buchanan, S.: Problem based learning and authentic assessment in digital pedagogy: Embracing the role of collaborative communities. The Electronic Journal of e-Learning, 13 (2), pp. 59-67 https://eric.ed.gov/?id=EJ1060176 (2015)
34 Varjonen, H.M.S., Hellas, A., Tarkoma, S., and Männistä, T.: Assessing IOT projects in university education: A framework for problem-based learning. ICSE-SEET ’17, pp. 37-46. IEEE Press, DOI: https://doi.org/10.1109/ICSE-SEET.2017.6 (2017)
35 Silva, P., Lopes, B., Costa, M. et al. Stairway to employment? Internships in higher education. High Educ 72, 703–721 https://doi.org/10.1007/s10734-015-9903-9 (2016).
36 Jackson, D., Collings, D. The influence of Work-Integrated Learning and paid work during studies on graduate employment and underemployment. High Educ 76, 403–425 https://doi.org/10.1007/s10734-017-0216-z (2018).
37 Scholtz, D.: Assessing workplace-based learning, Int. J. Work-Integrated Learning 21(1), 25-35 https://files.eric.ed.gov/fulltext/EJ1241256.pdf (2020)
38 Wright, J.C.: Continuing development of continuing professional development. Part 3: Recording and support of CPD. Engineering Science & Education Journal, 2 (2), pp. 70–76, https://doi.org/10.1049/esej:19930023 (1993)
39 Lorenzo, G., and Ittelson, J.: An overview of e-portfolios Educause learning initiative. https://library.educause.edu/resources/2005/1/an-overview-of-eportfolios (2005)
40 McKenna, G., Baxter, G., and Hainey, T.: E-portfolios and personal development: a higher educational perspective. Journal of Applied Research in Higher Education, 9 (1), pp. 141–171, https://doi.org/10.1108/JARHE-05-2016-0035 (2017)
41 López-Crespo, G., Blanco-Gandía, M.C., Valdivia-Salas, S. et al. The educational e-portfolio: preliminary evidence of its relationship with student’s self-efficacy and engagement. Educ Inf Technol 27, 5233–5248 DOI:10.1007/s10639-021-10827-2 (2022).
42 Salib, E.: CPD: Monitoring attendance or measuring performance? Psychiatric Bulletin, 23 (2), pp. 114–115 https://doi.org/10.1192/pb.23.2.114-b (1999)
43 Brigley, S., Young, Y., and Littlejohns, P., and McEwen, J.: Continuing education for medical professionals: A reflective model, Postgraduate Medical Journal, Jan., 73 (855), pp. 23–26 https://doi.org/10.1136/pgmj.73.855.23 (1997)
44 The SFIA Foundation: SFIA – Skills Framework for the Information Age. https://sfia-online.org/en/ (2021)
45 IT Professionalism Europe: Concepts and Principles included in the e-CF. https://itprofessionalism.org/about-it-professionalism/competences/the-e-competence-framework/ (2022)
46 IPA: SFIA vs iCD Mapping Research Project. Phase 1: Compare the Underlying Principles and Generics https://sfia-online.org/en/assets/documents/japan-ipa/jipa-sfia-icd-document.pdf (2017)
47 The SFIA Foundation: SFIA – Skills Framework for the Information Age. https://sfia-online.org/en/about-sfia/sfia-guiding-principles (2021)
48 The SFIA Foundation: SFIA 8 Levels of Responsibility. https://sfia-online.org/en/sfia-8/responsibilities (2021)
49 Davenport, J. H., Crick, T., Hayes, A., and Hourizi, R.: The Institute of Coding: Addressing the UK Digital Skills Crisis. CEP’19. ACM. DOI:10.1145/3294016.3298736 (2019)
50 Smith, M. F.: Software Prototyping: Adoption, Practice and Management. McGraw-Hill, Inc. (1991)
51 Bowers, D.S, Hayes, A., Prickett, T., Crick, T., Streater, K. and Sharp C.: The Institute of Coding Accreditation Standard: Exploring the Use of a Professional Skills Framework to Address the UK Skills Gap. UKICER23, ACM, DOI:10.1145/3610969.3611121 (2023)
52 Bowers, D.S., Sabin, M., Raj, R. K., and Impagliazzo, J.:. Computing Competencies: Mapping CC2020 Dispositions to SFIA Responsibility Characteristics. EDUCON’22. IEEE, 428–437. https://doi.org/10.1109/EDUCON52537.2022.9766565 (2022)
53 Zegwaard, K., Coll, R.K. and Hodges, D.: Assessment of workplace learning: A framework, Asia-pacific journal of cooperative education, 4(1), 9–18 . https://api.semanticscholar.org/CorpusID:16412611 (2003)
54 Welland, E. and Shaw, S.: E-portfolios in teaching, learning and assessment: tensions in theory and praxis, Technology, Pedagogy and Education 31(3) 363-379 https://doi.org/10.1080/1475939X.2022.2074087 (2022)
55 Yeo, M. A.: Academic integrity in the age of Artificial Intelligence (AI) authoring apps. TESOL Journal, 00, e716. https://doi.org/10.1002/tesj.716 (2023).