Here’s the symposium paper and session replay from the International Conference on Quantitative Ethnography, where a group of us reflected on the prospects for moving forward the concept of Participatory Quantitative Ethnography.
[Update! which has led to the creation of a PQE SIG]
Abstract: This symposium proposes that Participatory Quantitative Ethnography (PQE) is an important new strand of research for the QE community to develop. This paper introduces the participatory research values motivating PQE, outlines contributions from symposium speakers explaining the importance of PQE from different perspectives, before closing with a set of research questions that motivate a research agenda. It is hoped that this symposium may spark fruitful conversations and collaborations that advance PQE concepts, methodologies and tools.
We just wrapped up the 2nd International Conference on Quantitative Ethnography (open access proceedings from Springer), postponed from October due to the pandemic, in the hope that we might all yet meet up in February — alas it was not to be. However, the organisers did a really great job designing the program with a lot of informal interaction time, and the delegates threw themselves into it with a fantastic spirit, with everyone out to help everyone, as the crew figures out how to sail this recently launched ship!
If QE is new to you, it springs from the foundational work at the University of Wisconsin-Madison’s Epistemic Analytics Lab, led by the inspirational David Williamson Shaffer. The team’s publications are the source point, specifically, David’s QE book, which impressed me so much and drew me into this vision of how quant+qual could come together. In fact, I first met David in 2013 after I received a very hot tip that he was doing amazing work, relevant to a discourse analytics workshop I was chairing. His keynote was a revelation to me of his team’s long term research program, but it’s taken a while to figure out if and how to bring it into my own work.
Well, the first conference (ICQE 2019 / proceedings) was a great success, and so it was a real honour to be asked to give one of the keynote talks at this year’s conference. However, following the brilliant 2019 keynotes by Jim Gee, Dragan Gašević and Gol Arastoopour Irgens, I accepted with some trepidation to be honest, since I am far from a QE expert compared to those blazing this new trail. At the time of being invited, my team had not done any work with the main QE analysis approach, Epistemic Network Analysis, though we had drawn inspiration from its data modelling methodology in our multimodal learning analytics work.
So I thought long and hard about what I might bring to the party, with several false starts, which might have gone deeper into QE and Learning Analytics, or QE and Algorithmic Accountability. I decided in the end to go back to my roots — all the way back to my PhD in fact, focusing on the cognitive affordances of semiformal graphical representations, and what I’ve learnt since about what it takes to wield such tools in participatory design with fluency, developing open source visual hypermedia software for 20 years, and more recent work on data storytelling. What was particularly fun was bringing that into dialogue with what I was seeing in the QE webinars last year, specifically, how the community is telling its stories with visualizations. It was really enjoyable thinking what my journey might have to say to the QE community, the talk seemed to go down well, and I’m looking forward to seeing if/how these ideas take deeper root.
I am indebted to so many colleagues who have shaped those ideas, noting in particular, the Knowledge Art research and practice of my PhD student, colleague and friend, Al Selvin, tragically taken from us, far too early.
Quantitative Ethnography Visualizations as Tools for Thinking [pdf slides]
Abstract: All research must give form to data and insights. Visualizations serve as cognitive extensions that assist researchers not only in exploring their data, but in communicating findings to colleagues and broader audiences. Especially in data-intensive fields, widely used software tools define, and are defined by, research communities; you can’t fully participate in a community until you can wield its tools responsibly. In an emerging field like Quantitative Ethnography (QE), inventing its own tools, how we model and map the world are therefore defining characteristics, and merit critical reflection.
QE’s principles currently find fullest expression in Epistemic Network Analysis (ENA). It’s fair to say that the interest in ENA is attributable not only to the power of its data modelling and analysis, but also to the engaging, interactive visualizations it generates. Inspired by the ways I see ENA used, in this talk I bring my background in Human-Computer Interaction and the design of tools for working with conceptual structures, as a lens on ENA and other QEgenerated visuals. When we consider in detail how external representations serve as personal and shared cognitive tools, this illuminates current and future techniques for presenting QE analyses. A data-storytelling lens asks how the audience will engage with our insights, while participatory methods ask whether we cast them as passive recipients or active agents in validating those narratives. Moreover, as QE analyses begin to underpin new tools designed for people other than QE researchers, human-centred design should give voice to non-technical stakeholders. These lenses could point to a future in which visualization tools evolve to scaffold more participatory forms of sensemaking as an important hallmark of how QE models and narrates the world.
2020 will be remembered for many things… but amidst the disruption, it’s been a year of consolidation for the exciting, emerging field of Quantitative Ethnography (the book by David Williamson Shaffer; my review for Jnl. Learning Analytics).
The newly launched International Society for QE has been coordinating virtual events to strengthen professional ties across the globe, upskill researchers in the new tools and techniques, and the 2nd international conference is in Feb 2021. ISQE are to be congratulated on this progress, and in particular, the Epistemic Analytics Lab at U. Wisconsin-Madison are doing an awesome job in generously sharing their expertise, and making their work available through free analytical tools.
I was honoured to be asked to help design and chair the monthly webinar series which is building a library of examples how QE methods can be applied in diverse contexts. That’s proven to be a fascinating experience, and our own work (based on Vanessa Echeverria’s PhD) wrapped this up earlier this month (more coming in 2021!).
Abstract: Collocated, face-to-face teamwork remains a pervasive mode of working and learning, which is hard to replicate online. In team-based situations, learners’ embodied, multimodal interaction with each other and with digital and material resources has been studied by researchers, but due to its complexity, has remained opaque to automated analysis. The ready availability of sensors makes it increasingly affordable to instrument work spaces to automatically capture activity traces to study teamwork and groupwork. Yet, a key challenge is the enrichment of these multiple and intertwined quantitative data streams with the qualitative insights needed to make sense of them. In this seminar, we will discuss our inroads into giving meaning to multimodal group data. We have followed a human-centred approach to design meaningful end-user interfaces that convert multimodal data into data stories. Based on Quantitative Ethnography principles, we developed a modelling technique, termed the Multimodal Matrix, to grounding quantitative data in the semantics derived from a qualitative interpretation of the context from which it arises. We will present practical examples in the context of high-fidelity clinical simulations in which multimodal data (physiological, positioning, and logged actions) have been transformed into learning analytics interfaces that support teachers’ and learners’ reflection.
I am indebted to so many colleagues who have shaped those ideas, noting in particular, the Knowledge Art research and practice of my PhD student, colleague and friend, 
