Arminda Guerra
pp. 78-95, download
(https://doi.org/10.55612/s-5002-069-004)
Abstract
Multimodal interaction has become a central concern in contemporary Human–Computer Interaction (HCI), particularly in contexts where bodily engagement and sensory integration are fundamental to user experience. Interactive art installations constitute a specific domain within HCI in which interaction is exploratory, situated, and often resistant to conventional usability evaluation. Although principles from User-Centered Design (UCD), Interaction Design, and User Experience (UX) are frequently invoked to describe such systems, their empirical application to interactive art remains methodologically underspecified. This paper examines multimodal dialogue as an analytical lens for understanding interaction in artistic installations. Three interactive installations, SandBox, TIDE (IDEArt), and Threshold, are analyzed as qualitative case studies using a unified research design. Data were collected through systematic observation, semi-structured interviews, and audiovisual documentation, and analyzed using thematic analysis. The results identify recurrent patterns of embodied interaction, sensory exploration, and interpretative meaning-making across the three installations. In preference to proposing a new theoretical framework or quantitative evaluation model, this study contributes a methodologically transparent approach to investigating multimodal dialogue in interactive art, demonstrating, through evidence gathered across three case studies, how embodied and interpretative forms of interaction support sustained engagement, methodological reuse, and the long-term relevance of interactive installations beyond technological novelty.
keywords: Multimodal interaction, Interactive art installations, Embodied interaction, Dialogue, Qualitative HCI, Sustainable creative practices.
References
1. Endter, Cordula. “Design for Elderly – A Meeting Point for Ethnography and Usability” i-com, vol. 15, no. 1, 2016, pp. 17-26. https://doi.org/10.1515/icom-2016-0005
2. K. Höök, S. Bardzell, S. Bowen, P. Dalsgaard, S. Reeves, and A. Waern, “Framing interaction design through design theory,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2015), Seoul, South Korea, 2015. https://doi.org/10.1145/2702123.2702523
3. A. Lopes, Design as Dialogue: Encouraging and Facilitating Interdisciplinary Collaboration. Saarbrücken, Germany: VDM Verlag Dr. Müller, 2009.
4. S. Andrews, A. Lopes, J. MacDermott, and C. Newell, “TIDE (IDEArt),” in Proceedings of the Culture, Creativity and Interaction Design (CCID) Symposium, York, UK, 2006.
5. N. Mounajjed, C. Peng, S. Walker, N. Bryan-Kinns, and J. Sheridan, “Threshold,” in Proceedings of the Culture, Creativity and Interaction Design (CCID) Symposium, York, UK, 2006.
6. A. Moreno, A. Lopes, and M. Mendes, “Grains of Memory: User experience in the SandBox interactive installation,” in Advances in Intelligent Systems and Computing, vol. 954. Cham, Switzerland: Springer, 2019, pp. 261–270. https://doi.org/10.1007/978-3-030-20444-0_28
7. S. Oviatt, “Multimodal interfaces,” in The Human–Computer Interaction Handbook, 2nd ed., A. Sears and J. A. Jacko, Eds. Mahwah, NJ, USA: Lawrence Erlbaum, 2008, pp. 413–432.
8. S. Reeves, S. Benford, C. O’Malley, and M. Fraser, “Designing the spectator experience,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2004), Vienna, Austria, 2004, pp. 741–750. https://doi.org/10.1145/985692.985786
9. L. Candy and E. Edmonds, Explorations in Art and Technology. London, UK: Springer, 2002.
10. Gaver, B. (1999). Design: Cultural probes. Interactions.
https://doi.org/10.1145/291224.291235
11. M. J. Muller, E. Edmonds, and J. Connell, “Living laboratories: A new form of participatory design research,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2010), Atlanta, GA, USA, 2010, pp. 195–204. https://doi.org/10.1145/1753326.1753351
12. K. Höök, M. Jonsson, A. Ståhl, and J. Mercurio, “Somaesthetic design,” Interactions, vol. 23, no. 4, pp. 26–33, 2016. https://doi.org/10.1145/2940156
13. K. Höök, Designing with the Body: Somaesthetic Interaction Design. Cambridge, MA, USA: MIT Press, 2018.
14. P. Dalsgaard and L. K. Hansen, “Performing perception—Staging aesthetics of interaction,” ACM Transactions on Computer-Human Interaction, vol. 27, no. 2, Article 11, 2020. https://doi.org/10.1145/3381005
15. S. Benford, G. Giannachi, B. Koleva, and T. Rodden, Uncomfortable Interactions. Cambridge, MA, USA: MIT Press, 2023.
16. E. D. Mekler and K. Hornbæk, “A framework for the experience of meaning in human–computer interaction,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, UK, 2019. https://doi.org/10.1145/3290605.3300455
17. T. Schiphorst, “Felt experiences: Designing for movement awareness,” in Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI 2020), Sydney, Australia, 2020.
https://doi.org/10.1145/3374920.3374936
18. R. K. Yin, Case Study Research and Applications: Design and Methods, 6th ed. Thousand Oaks, CA, USA: Sage, 2018.
19. V. Braun and V. Clarke, “Using thematic analysis in psychology,” Qualitative Research in Psychology, vol. 3, no. 2, pp. 77–101, 2006. https://doi.org/10.1191/1478088706qp063oa
20. J. McCarthy and P. Wright, Technology as Experience. Cambridge, MA, USA: MIT Press, 2004.
21. Dourish, P., Where the Action Is: The Foundations of Embodied Interaction. Cambridge, MA, USA: MIT Press, 2001.