Simon.BuckinghamShum.net

The International Conference on Artificial Intelligence in Education has been running online all week. The theme this year is Mind the Gap: AIED for Equity and Inclusion, reflecting justified concerns in society at large, about the potential for data, analytics and AI to exacerbate societal inequities. Given the call for the community to work on “diversity, equity, and inclusion practices”, I was delighted to be asked to host a discussion session on Human-Centred Design.

Here we are in gather.town (worked nicely, and note the Covid-safe seating plan!) — below are the key points and links I posted to the chat during the session, which I’ve touched up slightly to make them intelligible… 

Firstly, from my perspective, designing AIED is a specific instance of the broader challenge of designing interactive systems that people value and use. So we can and should draw on the wealth of knowledge out there on how to do this well.

Secondly, I want to flag that HCD is about far more than nice user interfaces! Depending on the scale of lens you want to use, for me, we’re talking about understanding how socio-technical infrastructures get embedded into daily life.

So Informatics provides us with the broad lens needed to integrate computational artifacts into human ecosystems. Hence I frame my work as “Learning Informatics” http://simon.buckinghamshum.net/2020/09/why-learning-informatics

Human-Computer Interaction (HCI) aka Human-Centred Informatics provide a wealth of methodologies to study how people engage with interactive tools.

• Dan Russell’s opening keynote emphasised the vital importance of an HCI skillset for designing effective AIED, especially when that intelligence is imperfect
  • (you may be interested in this paper on why imperfect machine intelligence may be an educational feature not a bug)
• Great book: “Ways of Knowing in HCI” edited by Judy Olson & Wendy Kellogg https://www.springer.com/gp/book/9781493903771
• A great book on how the meanings and roles of “theory” have evolved in HCI (which might spark thoughts wrt how AIED is evolving) “HCI Theory: Classical, Modern, and Contemporary” by Yvonne Rogers https://doi.org/10.2200/S00418ED1V01Y201205HCI014
  • Note in particular that when HCI started out (the first ACM CHI conference was in the early 80s) we sought theory to get a grip on the dominant computing paradigm: individual in front of a computer. Cognitive psychologists believed they brought the concepts and tools needed to design and evaluate user interfaces, but that things have come a long way since.

The current important interest in FATE of AIED (https://doi.org/10.1007/s40593-021-00239-1) boils down to the trustworthiness of LA/AIED systems. This is about a lot more than opening black boxes. A diverse set of arguments underpins the claim that a system should be considered trustworthy http://simon.buckinghamshum.net/2019/11/black-box-learning-analytics

HCI is now coming into dialogue with Learning Analytics & AIED:

• BJET special issue “Learning Analytics and AI: Politics, Pedagogy and Practices” https://onlinelibrary.wiley.com/toc/14678535/2019/50/6
• Jnl Learning Analytics special issue “Human-Centred Learning Analytics”  https://learning-analytics.info/index.php/JLA/issue/view/463
  • Note: in the editorial we discuss briefly whether there are any features of education that make HCD different from other domains. I mentioned this:
    “In most HCI design contexts, stakeholders are treated as authoritative sources on how their work should be performed. Current work practices are studied to ensure that the envisaged software system does not inadvertently disrupt the human ecology of formal and informal activity. In sharp contrast, for HCLA, while learners are obviously able to speak with authority about their experiences of studying, they are
    • not expert learners whose work practices should necessarily be worked around;
    • not experts in the subject matter;
    • not expert educators whose views —e.g., about the design of a course, what counts as good feedback, or what analytics will help learning — can be treated as authoritative.”

There was some good discussion about whether there is anything distinctive about AIED systems that requires the invention of special HCD techniques to aid, for example, rapid prototyping.

• We noted the use of Wizard of Oz
• Paper prototyping, but ensuring that workshop participants are empowered to co-construct the designs
  • I mentioned a framework we’ve developed called Knowledge Art that clarifies the skills and dispositions that a participatory design facilitator brings to empower participants with visualizations, something that can be taught and practised https://simon.buckinghamshum.net/2019/05/knowledge-art-learning-resources
• The tuning of existing techniques specifically for our domain, examples below

Specific examples of human-centred design for LA/AIED in recent CIC PhDs — Antonette Shibani, Vanessa Echeverria and Carlos Prieto-Alvarez:

• Writing Analytics: https://cic.uts.edu.au/cics-first-phd-pedagogy-writing-analytics (inc this example of co-design http://heta.io/co-designing-automated-feedback-on-reflective-writing-with-the-teacher)
• Collocated Teamwork Analytics: https://cic.uts.edu.au/congratulations-dr-vanessa-echeverria
• Participatory/Co-design for Analytics: https://cic.uts.edu.au/congratulations-dr-carlos-prieto-alvarez-learning-analytics-co-design-phd

We’re translating this into HCD training/resources for researchers and educators:

• Writing Analytics http://wa.utscic.edu.au/events/laln-2020-workshop
• Visual Data Storytelling https://cic.uts.edu.au/events/lak21-workshop-a-tutorial-on-data-storytelling-for-learning-analytics-dashboards
• Learning Analytics DECK (LA-DECK) participatory design ‘playing cards’ http://ladeck.utscic.edu.au
• Learner-data journeys as a version of user/customer journeys  https://dl.acm.org/doi/10.1145/3292147.3292168

Backing out to the bigger picture, as researchers, we’re transitioning education into a new kind of “knowledge infrastructure” — the system of systems that interoperate technically and socially to generate, sanction and maintain knowledge about a field  http://simon.buckinghamshum.net/2018/06/icls2018-keynote

Categories: News-Archives
Tags: AI.edu, ethics, learning analytics, participatory-design