health – Simon.BuckinghamShum.net

CHI’19: Towards Collaboration Translucence

For several years in CIC, we’ve been prototyping multimodal learning analytics in partnership with our colleagues in the UTS Faculty of Health, with the ambition to generate instant feedback for debriefing after simulation exercises with mannikin patients. I’m looking forward to presenting this new work at CHI in May (in Glasgow, no less, where I grew up!). Congratulations to Vanessa on her great PhD work, and to Roberto for leading the multimodal learning analytics research program.

Echeverria, V., Martinez-Maldonado, R. and Buckingham Shum, S. (2019). Towards Collaboration Translucence: Giving Meaning to Multimodal Group Data. In Proceedings of ACM CHI Conference (CHI’19). ACM, New York, NY, USA, Paper 39, 16 pages. https://doi.org/10.1145/3290605.3300269 [PDF Reprint]

ABSTRACT: Collocated, face-to-face teamwork remains a pervasive mode of working, which is hard to replicate online. Team members’ embodied, multimodal interaction with each other and artefacts has been studied by researchers, but due to its complexity, has remained opaque to automated analysis. However, the ready availability of sensors makes it increasingly affordable to instrument work spaces to study teamwork and groupwork. The possibility of visualising key aspects of a collaboration has huge potential for both academic and professional learning, but a frontline challenge is the enrichment of quantitative data streams with the qualitative insights needed to make sense of them. In response, we introduce the concept of collaboration translucence, an approach to make visible selected features of group activity. This is grounded both theoretically (in the physical, epistemic, social and affective dimensions of group activity), and contextually (using domain-specific concepts). We illustrate the approach from the automated analysis of healthcare simulations to train nurses, generating four visual proxies that fuse multimodal data into higher order patterns.

Here’s an extended version of the CHI talk, presented at U. Sydney CHAI research group [PDF slides]:

Narrative annotation scheme: design rationale & evaluation

Last year PhD student Joanna Kwiat released the detailed literature review from her Storymaking Project as a self-contained KMi Technical Report: “From Aristotle to Gabriel: A Summary of the Narratology Literature for Story Technologies.”

This inspired the design and detailed evaluation of a prototype “storybase” (story database) for healthcare professionals to share their knowledge and experiences as annotatable stories. The advance that this work makes is to go beyond tagging, by using an annotation scheme grounded in narrative theory, in which peers declare the different “points” of the story as they see them, plus other optional facets such as the protagonist, antagonist, genre of story, etc.

I’m delighted to say that the full report of this work is now published as her dissertation:

Kwiat, J. (2009). Multi-Perspective Annotation of Digital Stories for Professional Knowledge Sharing within Health Care. Doctoral Dissertation, available as: Technical Report KMI-09-02, Knowledge Media Institute, The Open University, UK. http://kmi.open.ac.uk/publications/pdf/kmi-09-02.pdf

Abstract

[We] dream in narrative, daydream in narrative, remember, anticipate, hope, despair, believe, doubt, plan, revise, criticize, construct, gossip, learn, hate, and love by narrative. Barbara Hardy, 1977.

This thesis investigates the potential of narrative theory to inform the design of tools for sharing and annotating stories, in the context of professional knowledge sharing. We begin with a detailed review of the literature on modelling narrative, to establish the theoretical foundations for a narratologically-grounded annotation schema. Medicine is then selected for a tri-part study, since narrative-based approaches in healthcare education and practice are seen by many as significant.

The first part seeks evidence of narrative among medical professionals communicating spontaneously and informally online. The frequency and range of stories identified shows that this appears to be a common and valued mode of communication.

The second part envisions a Web story database (“storybase”) supporting flexible annotation grounded in a narratological metadata scheme. The model draws on various narrative structure theories, and in particular, point-structure. A story can be annotated via a graphical user interface on various dimensions, enabling multiple interpretations.

The third part analyses users annotating representative samples of the stories abstracted from the corpus in part 1. Data is analysed quantitatively (annotation value clustering, questionnaire responses and task phase durations) coupled with a qualitative account of participant behaviour based on grounded theory video analysis. While this study has limitations, it validates both the expressiveness and usability of the story annotation schema, and shows that participants found the experience to be enjoyable and stimulating. Interaction analysis demonstrates the centrality of interface design in shaping annotation behaviour. This work motivates further storybase research, informing the design of future studies and storybase technologies.

An additional report provides data and analysis appendices:

Kwiat, J. (2009). Multi-Perspective Annotation of Digital Stories for Professional Knowledge Sharing within Health Care: Appendices. Technical Report KMI-09-03, Knowledge Media Institute, The Open University, UK. http://kmi.open.ac.uk/publications/pdf/kmi-09-03.pdf

Very well done Joanna! A significant piece of work in an emerging field at the intersection of narrative and digital media, tuned to support collective sensemaking in a way that engages both head and heart.