GenAI for QDA: LLM workflows, interactive tools, and methodological rigour

Here’s a short talk I recently gave at Monash University’s Symposium on Using AI in the Scholarship of Teaching & Learning Research Process: Methods, Tools and Practices, hosted by their AI in Research Reference Group and the Business Education Research Group. My thanks to Jess Co for inviting me.

It’s a snapshot of work underway in various projects with colleagues at UTS and beyond, and our PhD students. References + links in the slides. For more details on ThemeExplorer and GROPROE, see this seminar and the associated papers.

And here’s the ThemeExplorer video I showed, designed as part of the Student Voices on AI in HE project, codesigned with Antonette Shibani, Lisa-Angelique Lim, Trish McCluskey and the amazing Aneesha Bakharia (U Queensland) who coded it (paper below):

Bakharia, A., Shibani, A., Lim, L.-A., McCluskey, T., & Buckingham Shum, S. (2025). From Transcripts to Themes: A Trustworthy Workflow for Qualitative Analysis Using Large Language Models. Proceedings of Workshop From Data to Discovery: LLMs for Qualitative Analysis in Education, LAK25: 15th International Conference on Learning Analytics & Knowledge, Dublin, IRE, pp. 1-10. https://ceur-ws.org/Vol-3995/LLMQUAL_paper1.pdf

 

Award! CIC and faculties co-design personalised feedback with OnTask to build student belonging

UTS awards CIC’s multi-year, multidisciplinary faculty collaboration, using the OnTask tool to scale personalised feedback and build student belonging.

Since 2017, CIC has been supporting academics to embed OnTask messaging in learning design, offering workshops and consultations. When Lisa-Angelique Lim joined CIC in 2021, she brought deep expertise from her PhD at UniSA, and took this work to a new level, leading a learning community to implement OnTask across a wider range of subjects, thereby enabling personalised feedback and support for students in multiple faculties. 

So it’s a true mark of recognition to see that CIC, in partnership with UTS academics from multiple faculties, received an award in recognition for our sustained efforts in the Student Experience category, for fostering students’ belonging in multiple disciplines through personalised feedback. This accolade was part of the Vice-Chancellor’s 2024 Learning and Teaching Awards and Citations, and was celebrated at the 2025 Learning and Teaching Awards Ceremony at UTS on 4 April 2025.

Our large, multidisciplinary team was led by Dr Lisa-Angelique Lim (CIC), with Associate Professor Amanda White (Business), Dr Amara Atif (FEIT), Chris Croese (Law), Associate Professor James Wakefield (Business), Dr Keith Heggart (FASS), Associate Professor Nicole Sutton (Business), Ram Ramanathan (CIC PhD student), Dr Rina Dhillon (Business), Dr Simone Faulkner (Business), and Professor Simon Buckingham Shum (CIC).

Addressing the challenges of student belonging with personalised feedback using OnTask

Belonging is a cornerstone of the student experience, critically impacting engagement, retention, and overall success. At UTS, the Student Experience Framework places a strong emphasis on belonging. However, the challenge of nurturing a sense of belonging at the classroom level is significant, given the diverse and large student population, particularly in first-year core subjects. Personalised feedback has emerged as a potent tool to address this challenge, serving as a form of ‘relational pedagogy’ that builds self-efficacy and fosters greater engagement and thriving at university.

To tackle the challenge of belonging, our team leveraged OnTask — a tool designed to support students through personalised communication and feedback across multiple disciplines, based on their data. OnTask uses learning analytics to provide tailored messages that help students stay on track with their studies, understand their progress, and feel supported throughout their academic journey. Check out this short animation on how OnTask works.

Quick teaser video of our academics’ perspectives of their implementation of personalised feedback in their context:

 

What this looks like in practice

Here, we highlight a few examples of the work by our team.

  • The UTS Business First and Further Year Experience (FFYE) team, led by A/Prof James Wakefield and Dr Simone Faulkner, used OnTask to personalize orientation communications for both undergraduate and postgraduate commencing students, leading to a significant increase in orientation registrations.
  • A/Prof Amanda White used OnTask in the Accounting for Business Decision A (ABDA) subject to tailor emails based on students’ progress, encouraging higher engagement and leading to improved exam grades.
  • Dr Keith Heggart created personalised video messages tailored to students’ confidence levels in the fully online Graduate Certificate in Learning Design, significantly enhancing student engagement and perceptions of support.
  • The positive impacts of OnTask are well-documented. For instance, in the large first-year subject, 22208 Accounting, Business and Society (ABS), led by Dr Rina Dhillon and A/Prof Nicole Sutton, personalised messages based on weekly quiz results led to improved pass rates and enhanced feelings of being valued among at-risk students.
  • Similarly, personalised feedback messages sent by Chris Croese to his students in large Law subjects kept students on track and motivated, correlating with better final grades. More stories of how academics at UTS have used OnTask, with research papers, can be found on CIC’s OnTask page.

Towards future partnerships to enhance student belonging

With this award, we celebrate the collaborative efforts and innovative approaches taken by our team to foster a sense of belonging among students through personalised feedback. Our journey over these past three years demonstrates the sustainability of this practice and its potential to influence and enhance teaching and learning widely.

We are honoured to receive this recognition, and look forward to continuing strong partnerships, to make a positive impact on the student experience at UTS.

Compendium Archive & Network 2021

CONTEXT… Those of you who know my R&D will know that I have a long-standing interest in the role that software can play in human sensemaking around wicked problems, a form of “augmenting human intellect” (Doug Engelbart). A powerful example is structured, visual hypermedia that makes tangible the ways that ideas, data and arguments connect with each other and documents.

This is certainly a story about the evolution of an interactive visual tool for thinking—but far more interestingly, it’s about the co-evolution of software with a set of practices to develop human fluency with the tool. Those practices were (in historical order) Dialogue Mapping, Issue & Argument Mapping, and Knowledge Art.

Here’s how I told this story in 2014, tracing my work back to Engelbart’s vision of personal computing and collective IQ, and here’s a quick overview of key books and papers.

COMPENDIUM… provides extremely flexible hypermedia linking between conceptual objects (e.g. questions, ideas, arguments), data and documents (local/online), through a visual user interface. As a hypertext system for thinking with, connections can be made in multiple ways: spatial proximity and visual linking within a view, plus tagging and transclusions across views (i.e. embedding a node in multiple views). Views can contain each other non-hierarchically (A can contain B which can contain A). Search can be refined by node types and tags. This 2014 RAE Impact Case distills the research –> impact narrative.

It is an open source, desktop Java application with a full SQL database, with XML and SQL import/export, and HTML publishing. In the course of its development, it was interoperable with (at the time state of the art) Jabber (open source instant messaging) XML, and semantic web RDF.

HYPERTEXT HISTORY… Compendium is descended from the pioneering hypermedia system gIBIS (graphical Issue-Based Information System) at MCC Labs led by Jeff Conklin and Michael Begeman, which led to the commercial corporate memory product CM1, renamed Questmap. Compendium was then developed over the course of around 20 years R&D starting in the early 90s at Bell Atlantic labs (White Plains NY) led by Al Selvin and Maarten Sierhuis, continued at NASA Ames Research Centre by Maarten on the Mobile Agents project providing a human-agent science team tool, in collaboration with my team at the Open University’s Knowledge Media Institute (KMI) from 1995-2014. The gIBIS/CM1/Questmap/Compendium lineage exemplifies an influential strand of hypertext R&D that preceded the invention of the Web, which like Xerox NoteCards, exemplifies what Frank Halasz called hypertext for idea processing — focusing on visualising and managing the connections between nodes as a form of intellectual work. You can learn a lot more about the intellectual lineage of these ideas in this brief history, another account, on this blog via the compendium tag, and on the memorial archive of Compendium co-inventor, my colleague, PhD student and friend, Al Selvin.

COMPENDIUM INSTITUTE… The Compendium Institute coordinated the international user/developer network, feature requests, code releases, research and training. Software development has been on pause since 2013, and may well not be continued, since much of the world now expects web-based tools (indeed see the great work by DebateGraph, and the KMI team developed quite a few). However, there remains an active user base of people who value the speed and functionality of a Java desktop app which continues to run on current Mac/Win/Linux Java.

I have therefore archived the Compendium Institute website for posterity, since it contains lots of resources:

SOFTWARE… You want the tool! I’m trying to keep links alive, and Java seems to be holding up remarkably since development ended around 2016.

  • If you’re a developer, then you can get the most recent CompendiumNG, from the CompendiumNG (next generation) wiki.
  • I can also offer this Mac installer version of CNG which requires no code knowledge to get running
  • If you’re non-technical on Win/Linux then this version of Compendium comes in an integrated installer:
    • KMi Open University version 2.0beta with advanced experimental features (like Maps supporting video annotation): Downloads page | QuickStart Guide for Mac or Windows *follow the guide*
    • CogNexus Institute also offers download links for a slightly older version

COMMUNITY… Since Yahoo closed down their groups end of last year, I’ve created a new Google Group which you’re warmly invited to join if you want to stay connected with fellow users and some of the original team. Collectively, we will hopefully be able to answer any queries about Compendium’s functionality and design rationale — and who knows, possible futures…