Autumn 2023

Table of contents  –  Authors index – Authors’ short bio

Daniel Rosenberg Munoz, Peter Scupelli, Dina El-Zanfaly, Daragh Byrne
(https://doi.org/10.55612/s-5002-058-001psi) download

Over the past few decades, interactive technologies have become increasingly embedded in and blended with our physical world. These technologies—mobile computing, AR and VR, interactive IoTs, tangible interfaces, and responsive architectures—are increasingly shaping our daily experiences in our homes, workplaces, neighborhoods, and cities [1,3–6,8]. We call these physical-digital spaces hybrid environments [2]. As these technologies continue transforming our lived experiences, many new opportunities emerge for design education and practice.
More and more programs in higher education aim to cross-train designers, architects, creative technologists, engineers, and artists to operate in these emerging hybrid digital terrains [7]. This is a complex pedagogical practice. We noticed in our own teaching that designing for hybrid spaces requires knowledge and training in designing both physical environments and interactive technologies. Hybrid design solutions cut across and often operate at multiple scales: from the object to the room, from the neighborhood to the city. Consequently, the design of good physical-digital spaces must often bridge fundamental knowledge in design, architecture, art, interactivity, and computation. How are we meeting this emergent need? How do we educate cross-trained students to creatively respond and adapt to these complex design spaces?
In this special issue, we invited design educators to share their pedagogies, frameworks, and tools for crafting next-generation physical-digital spaces. When we set out the request for papers, we imagined authors would focus on different strategies and approaches to integrate physical materials and interactive technology within environments. However, the papers we received for this special issue were much broader than our initial notion of hybrid environments. We present six papers where their approaches, pedagogies, and frameworks propose unique strategies to address and leverage the concept of hybridity within educational settings.
Nalli et al. [9] examine the challenges and opportunities of integrating biometric sensors into physical learning environments to measure and adapt experiences to individual learners’ stress levels. Through this case, they highlight how this real-time data can be a resource for learners and instructors by providing feedback that can adapt the learning experience and help foster engagement and self-direction. In their work, hybrid environments are understood as the integration of wearable sensing within physical learning spaces.
By introducing the concepts of remote and asynchronous interactions, Baykal’s paper [10] expands hybridity beyond Nalli et al.’s integration of sensing within environments. This paper investigates a pedagogical framework with concrete in-class activities for teaching how to design for collaborative interactions. Baykal’s work is based on the time-space matrix with face-to-face or remote interactions occurring synchronously or asynchronously [11]. Using this framework, the students explore a range of physical-digital approaches and designs, including mobile collaboration, synchronous interaction in social extended reality (XR) environments, and co-located collaborative games.
For Kluge and Hasle [12], Augmented Reality (AR) constitutes an archetypical hybrid technology, as digital information can overlay objects in the real world. Using this approach, they investigate how students can leverage AR to integrate knowledge from in-person learning sessions within classroom settings. In their study, a fifth-grade class is observed as they use a 3D animation of the human heart overlaid floating on top of their physical textbooks. This hybrid pedagogical setting supported the integration of knowledge. Using AR, the students could expand issues, make relations between them, and put them into a larger context.
Moving from AR to screen-based experiences, Usman et al. [13] introduce a game for teaching land surveying to expand students’ access and prepare them before going to the field. The game simulates fieldwork experiences for land surveying by integrating 3D models of buildings, spaces, and required instrumentation for students to experience from a first-person perspective within a web-based environment. Their hybrid educational pedagogy integrates remote instruction with in-person field activities for students to test their skills in the actual physical environment. Real-world use cases conducted in the lab demonstrate that their hybrid approach supported practical training.
While Usman et al. combined remote virtual games with in-situ practices, Clark and Fernaeus [14] explored the plasticity of design education by going fully remote. This author explores the shift in design education from traditional hands-on practices to software-based approaches. Through a set of online activities, students and instructors engage in creative and reflective practice to investigate what is at stake when designing for distributed work. Although fully remote and asynchronous instruction, this paper unveils the embodied experience of the students and teachers in their physical settings– in their homes and offices as they connect with one another remotely through their cameras and digital screens. This paper expands the notion of hybridity by situating remote learning experiences in context, showing the devices, bodies, and surroundings where they occur.
Finally, Papanikolaou [15] investigates how to design environments that connect people, objects, and places. In his pedagogical approach, hybridity refers to the design solutions, i.e., objects and spaces integrate computational capabilities–analog and digital–for interaction, communication, and intelligence. The paper introduces a theoretical framework for connective environments, supported by a series of design assignments and corresponding samples of students’ work. The assignments and projects range from mechanical computing objects to interfaces for co-creation, from analog artifacts that register human traces to hardware interfaces for physical telepresence. The design of these hybrid projects is possible through a physical-digital learning environment–the design studio. In the classroom, not only are the designs hybrid, but the environment is instrumented with physical and digital tools for designing through making, such as computer-aided design (CAD), physical computing (Arduinos, IoTs), and digital fabrication (3D printers, laser-cutters).
As we expanded our initial understanding of physical-digital spaces, we began generating a framework to organize and correlate the diverse papers’ approaches to hybridity. We call this framework the hybrid chords diagram (Fig. 1). The diagram is structured in three layers or dimensions: medium, presence, and time.

Fig. 1. Illustration of hybrid diversity in three dimensions: medium, presence, and time. Medium ranges from physical to VR. Presence includes the same-location and remote. Time includes synchronous and asynchronous.

The 1st dimension explains our initial understanding of hybridity as the material medium integrating physical and digital spaces (Fig. 2). Using Milgram-Kishino’s virtuality continuum [16] as a starting point, we defined a hybridity spectrum with five sections: Physical (no tech integrated), Physical Computing (IoTs, tangible interfaces, responsive architectures), AR (on screens, headsets, or projections), screen-based technology (GUI in mobile and PC), and VR (headsets with total immersion). The six papers in this special issue are initially mapped to this first dimension, the medium, from Usman’s pure physical in-person field activities to Baykal’s fully immersive collaborative VR games.

Fig. 2. The hybrid chords’ 1st dimension, or base layer. This continuum goes from Physical, Physical Computing, Augmented Reality (AR), Screen-based, and Virtual Reality (VR).

This first dimension, however, only shows the medium or physical-digital material used in an educational setting. Baykal, Usman, et al., and Clark and Fernaeus showed us two additional hybrid dimensions: presence and time. By presence, we mean same-location or remote locations of participants. By time, we mean synchronous or asynchronous interactions. The space-time matrix inspired us to add these two dimensions separately [9]. Our initial understanding of hybrid environments is expanded by adding these two dimensions.
We map the six papers of this special issue using our hybrid chords diagram (Fig. 3). The three dimensions–medium, presence, and time–forced us to reimagine what an extended version of hybrid environments might include. We explored many different ways to visualize these three dimensions. We opted for a musical string and chords metaphor. The chords represent combinations, where different strings could be played with different finger holds. The circles and connected lines show the different kinds of hybrid chords. The papers describe where one would place their fingers to play such a note. We chose the stringed instrument as a metaphor because it allows us to illustrate multi-dimensional spaces without the visualization challenges of drawing 3-D visualizations.

Fig. 3. Illustration of the six papers in the special issues as they relate to the diversity of hybrid on three dimensions: medium, presence, and time. The colored circles and lines show the connections between the dimensions of each paper.

By using IoTs within physical learning spaces, Nalli et al. exemplify our initial understanding of hybrid environments within same-location and synchronous settings. Although Baykal introduces the time-space matrix, the concepts, exercises, and students’ work primarily focus on same-location synchronous interactions using AR, Screen-based, and VR technologies. Kluge and Hasle’s chords touch similar strings for presence and time but use AR as a medium. Usman et al. introduce a double chord with a screen-based virtual game played remotely integrated with in-person field activities. Although Clark and Fernaeus’s chord touches screen-based remote and asynchronous experiences, their approach to embodied and situated interactions expands to the physical and non-tech section within the medium string. Finally, Papanikolaou plays a complex chord, combining physical computing and screen-based technologies with synchronous and remote experiences in same-location settings.
In this editorial, we elicited six papers covering design education for hybrid environments. The papers we received for this special issue expanded our definition of hybridity from physical and digital along the virtuality continuum to include three dimensions: medium, presence, and time. Medium currently includes five types: physical, physical computing, Augmented Reality (AR), screen-based interactions, and Virtual Reality (VR). Presence includes two dimensions: same-location and remote-locations. Time contains two dimensions: synchronous and asynchronous interactions.
The hybrid chords diagram emerged from reading the six papers together, seeking to identify correlations between them. We recognize that our approach to hybridity can be expanded to include N-different dimensions as needed. For example, a four-stringed diagram emerges if we add another dimension, such as duration (e.g., brief, continuous, intermittent, and long-term). Future work could explore that dimension.
Besides the capability to visually map multiple dimensions at once, we also noticed that our diagram exposed empty areas that could be explored in future work. For example, none of the six papers in this special issue explored asynchronous time. A new breed of hybrid environments could emerge by exploring asynchronous interactions ranging from same-location to remote presence across the medium spectrum, from entirely physical to fully virtual. Just as modern harps have 47 strings, we are confident that this kind of diagram can accommodate many more ways to map diverse dimensions of hybridity as our understanding of physical-digital spaces evolves.

References:

1.   Benabdallah G., Bourgault S., Peek N., and Jacobs J.: Remote Learners, Home Makers: How Digital Fabrication Was Taught Online During a Pandemic. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI ’21), Association for Computing Machinery, New York, NY, USA, pp. 1–14 (2021)
DOI:https://doi.org/10.1145/3411764.3445450
2. Buechley L., Mellis D., Perner-Wilson H., Lovell E., and Kaufmann B.: Living wall: programmable wallpaper for interactive spaces. In Proceedings of the 18th ACM international conference on Multimedia (MM ’10), Association for Computing Machinery, New York, NY, USA, pp. 1401–1402 (2010)
DOI:https://doi.org/10.1145/1873951.1874226
3. van Dijk J.: Post-human Interaction Design, Yes, but Cautiously. In Companion Publication of the 2020 ACM Designing Interactive Systems Conference (DIS’ 20 Companion), Association for Computing Machinery, New York, NY, USA, pp. 257–261 (2020)
DOI:https://doi.org/10.1145/3393914.3395886
4. DiSalvo C., Lukens J., Lodato T., Jenkins T., and Kim T.: Making public things: how HCI design can express matters of concern. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’14), Association for Computing Machinery, New York, NY, USA, pp. 2397–2406 (2014)
DOI:https://doi.org/10.1145/2556288.2557359
5.   Dourish P., Finlay J., Sengers P., and Wright P.: Reflective HCI: towards a critical technical practice. In CHI ’04 Extended Abstracts on Human Factors in Computing Systems (CHI EA ’04), Association for Computing Machinery, New York, NY, USA, pp. 1727–1728 (2004)
DOI:https://doi.org/10.1145/985921.986203
6. Frauenberger C.: Entanglement HCI The Next Wave? ACM Trans. Comput.-Hum. Interact. 27, 1 (November 2019), 2:1-2:27 (2019)
DOI:https://doi.org/10.1145/3364998
7. Garcia Estrada J. and Prasolova-Førland E.: Running an XR lab in the context of COVID-19 pandemic: Lessons learned from a Norwegian university. Educ Inf Technol (Dordr) (February 2021), pp. 1–17 (2021)
DOI:https://doi.org/10.1007/s10639-021-10446-x
8. Garg R. and Cui H.: Social Contexts, Agency, and Conflicts: Exploring Critical Aspects of Design for Future Smart Home Technologies. ACM Trans. Comput.-Hum. Interact. 29, 2 (January 2022), 11:1-11:30 (2022)
DOI:https://doi.org/10.1145/3485058
9. Nalli G., Dafoulas G., Tsiakara A., Langari B., Mistry K., Tahmasebi Aria F.: Hybrid Educational Environments – Using IoT to detect emotion changes during student interactions, Interaction Design & Architecture(s) – IxD&A Journal, N.58, pp. 39–52 (2023)
DOI: https://doi.org/10.55612/s-5002-058-001
10. Baykal G., Torgersson O., Ruijten-Dodoiu P., Eriksson E.: Teaching Design of Technologies for Collaborative Interaction in Physical-Digital Environments, Interaction Design & Architecture(s) – IxD&A Journal, N.58, pp. 53–71 (2023)
DOI: https://doi.org/10.55612/s-5002-058-002
11. Johansen, R.: Groupware: Computer support for business teams, New York: The Free Press, (1988)
12. Kluge A., Hasle M.: Use of Augmented Reality to Integrate Knowledge, Interaction Design & Architecture(s) – IxD&A Journal, N.58, pp. 90–109 (2023)
DOI: https://doi.org/10.55612/s-5002-058-004
13. Usman M., Usmani A., Jadidi M.: Re-envisioning Field Surveying as an Immersive Gamification Experience, Interaction Design & Architecture(s) – IxD&A Journal, N.58, pp. 72–89 (2023)
DOI: https://doi.org/10.55612/s-5002-058-003
14. Clark B. and Fernaeus Y.: Playing with the elasticity of hybrid design education, Interaction Design & Architecture(s) – IxD&A Journal, N.58, pp. 110–131 (2023)
DOI: https://doi.org/10.55612/s-5002-058-005
15. Papanikolaou D.: Connective Environments: Towards Interdisciplinary Pedagogical Models for Design of Networked, Interactive, Participatory Systems, Interaction Design & Architecture(s) – IxD&A Journal, N.58, pp. 132–156 (2023)
DOI: https://doi.org/10.55612/s-5002-058-006
16. Milgram, P., Kishino, F.: A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems, 77(12), pp. 1321–1329 (1994)