Simon.BuckinghamShum.net

I particularly enjoy writing these blog posts, celebrating the publication of a doctoral thesis. The blood, sweat and tears by doctoral researchers that goes into this moment is always immense, when one thinks about the learning curve they go through, the highs and the lows, and the balancing act of juggling this with the rest of life over 3-4 years. But here we are again, and I’m delighted to add a new one to these team posts!

Well done Carlos Prieto-Alvarez, as today sees the publication of one of the first PhDs in Learning Analytics devoted to the contributions of human-centred design methods (specifically, co-design) to give non-technical stakeholders such as educators and students a voice in shaping Learning Analytics. My thanks to co-supervisor Roberto Martinez-Maldonado, and also to Theresa Anderson who was on the team for the first year or so. Carlos is the third graduate from the CIC Learning Analytics PhD Program launched in 2016!

Prieto-Alvarez, C.G. (2020), Engaging Stakeholders in the Learning Analytics Design Process. Doctoral Dissertation, Connected Intelligence Centre, University of Technology Sydney, AUS. http://hdl.handle.net/10453/142525

The abstract and full dissertation are below, read the CIC news stories over the years from his work, and replay his final year thesis presentation:

Engaging Stakeholders in the Learning Analytics Design Process

ABSTRACT:

Learning Analytics (LA) is a new promising field that is attracting the attention of education providers and a range of stakeholders including teachers, learning designers academic directors and data scientists. Researchers and practitioners are interested in learning analytics as it can provide insights from student data about learning processes, learners who may need more help, and learners’ behaviours and strategies. However, problems such as low educator satisfaction, steep learning curves, misalignment between the analytics and pedagogical approaches, lack of engagement with learning technologies and other barriers to learning analytics development have already been reported. From a human-centred design perspective, these problems can be explained due to the lack of stakeholders’ involvement in the design of the LA tools. In particular, learners and teachers are commonly not considered as active agents of the LA design process. Including teachers, learners, developers and other stakeholders as collaborators in the co-design of LA innovations can bring promising benefits in democratising the LA design process, aligning analytics and pedagogy, and meeting stakeholders’ expectations. Yet, working in collaboration with stakeholders to design LA innovations opens a series of questions that are addressed in this thesis in order to contribute to closing the gap for effective co-design of LA innovations. The questions addressed in this thesis are the following:

1. How can co-design techniques assist in the integration of diverse stakeholders in the LA design process?
2. What are the roles of the co-design practitioner/researcher in the LA design process?
3. What are the challenges in engaging stakeholders in the LA design process?

Based on co-design principles, and following a Design-Based Research process, this thesis explores the critical challenge of engaging educators and students, the non-technical stakeholders who are often neglected, but who should ultimately be the main beneficiaries of LA innovations. In this research work, three case studies have been used to test, analyse and verify various co-design techniques in diverse learning contexts across a university to generate a co-design toolkit and recommendations for other co-design practitioners: i) learners and educators engaged in simulation-based healthcare scenarios, ii) learners, educators and other stakeholders in a Data Science Masters program, and iii) educators interested in providing personalised feedback at scale.

This thesis presents three contributions to knowledge for effectively collaborating with educational stakeholders in the LA co-design process:

1. Inspired by archetypal challenges reported in classic and contemporary co-design literature, and in current LA research, the thesis identifies, exemplifies and reflects on five key challenges for LA co-design: power relationships, surveillance, learning design dependencies, asymmetric teaching/learning expertise, and data literacy.
2. By adopting and adapting well established co-design techniques, across the three case studies, the thesis provides empirical evidence of how these techniques can be used in LA co-design, reflecting on their affordances, and providing guidance on their usage. These detailed findings are distilled into a Learning Analytics Co-design Playbook, published under an open license to assist adoption and improvements.
3. Recognising the importance of the co-design practitioner in ensuring that the design process is participatory, the thesis documents and discusses the key functions and skills that this position requires. The role is further complicated when the practitioner is not only a facilitator serving a project, but also a researcher of co-design. This motivates guidelines on the role of the co-design practitioner/researcher when working with stakeholders, and simultaneously studying the LA co-design process, tools and methods.

See Carlos’ ResearchGate site for full-text papers.

Conference Papers

• Prieto-Alvarez, C.G., Martinez-Maldonado, R. and Buckingham Shum, S. (2020). LA-DECK: A Card-Based Learning Analytics Co-Design Tool. Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK2020), Frankfurt, Germany, March 2020, ACM, New York, NY, USA. 10 pages. DOI: https://doi.org/10.1145/3375462.3375476
• Prieto-Alvarez, C.G., Martinez-Maldonado, R. and Buckingham Shum, S. (2018). Mapping Learner-Data Journeys: Evolution of a Visual Co-design Tool. Proceedings of the 30th Australian Conference on Computer-Human Interaction (OzCHI ’18), Melbourne, Australia, Dec. 2018, ACM, New York, NY, USA, pp. 205–214. DOI: https://doi.org/10.1145/3292147.3292168
• Prieto-Alvarez, C.G, et al. (2018). Collaborative Personas for Crafting Learners Stories for Learning Analytics Design. Workshop Participatory Design for Learning Analytics, International Conference on Learning Analytics and Knowledge LAK’18. Sydney, Australia, ACM: 647-652. ISBN: 978-1-4503-6400-3

Book Chapter

• Prieto-Alvarez, C.G., Martinez-Maldonado, R. Anderson, T. (2018). Co-designing learning analytics tools with learners. Learning Analytics in the Classroom: Translating Learning Analytics Research for Teachers, Taylor & Francis Groups: 93-110.

Workshops

• Carlos G. Prieto-Alvarez et al (2018). Learning Analytics Design Cards (LA-DECK): Unpacking inter stakeholder co-design through strategic cards. Australian Learning Analytics Summer Institute. Melbourne, Australia. Website: http://ladeck.utscic.edu.au/events.html
• Carlos G. Prieto-Alvarez et al (2018). Participatory design of learning analytics. International Conference on Learning Analytics and Knowledge LAK’18. Sydney, Australia, ACM. Website: http://pdlak.utscic.edu.au.

Categories: Team
Tags: learning analytics, participatory-design