learning dispositions – Simon.BuckinghamShum.net

Framing Professional Learning Analytics as Reframing Oneself

It’s been rewarding working with close colleagues on this, weaving our ideas together over the last year. Here’s the Open Access Preprint (final version has some minor edits). It will appear later this year in what should be a really interesting special issue on “Designing Technologies to Support Professional & Workplace Learning for Situated Practice”.

Buckingham Shum, S., Littlejohn, A., Kitto, K. & Crick, R. (2022). Framing Professional Learning Analytics as Reframing Oneself. IEEE Transactions on Learning Technologies, 15(5), pp.634-649. https://doi.org/10.1109/TLT.2022.3190055

Abstract: Central to imagining the future of technology-enhanced professional learning is the question of how data are gathered, analyzed, and fed back to stakeholders. The field of learning analytics (LA) has emerged over the last decade at the intersection of data science, learning sciences, human-centered and instructional design, and organizational change, and so could in principle inform how data can be gathered and analyzed in ways that support professional learning. However, in contrast to formal education where most research in LA has been conducted, much work-integrated learning is experiential, social, situated, and practice-bound. Supporting such learning exposes a significant weakness in LA research, and to make sense of this gap, this article proposes an adaptation of the Knowledge-Agency Window framework. It draws attention to how different forms of professional learning locate on the dimensions of learner agency and knowledge creation. Specifically, we argue that the concept of “reframing oneself” holds particular relevance for informal, work-integrated learning. To illustrate how this insight translates into LA design for professionals, three examples are provided: first, analyzing personal and team skills profiles (skills analytics); second, making sense of challenging workplace experiences (reflective writing analytics); and third, reflecting on orientation to learning (dispositional analytics). We foreground professional agency as a key requirement for such techniques to be used effectively and ethically.

CfP: Learning Analytics for C21 Competencies

CfP: Learning Analytics for 21st Century Competencies

Special Issue Editors: Simon Buckingham Shum & Ruth Crick (University of Technology Sydney)

Full call for Special Issue submissions: http://bit.ly/jlac21

A strategic educational response to a world of constant change is to focus explicitly on nurturing the skills and dispositions which equip learners to cope with novel, complex situations, assessed under authentic conditions. Thus, even if we do not know what the future holds, we can be better equipped for the only thing we can be sure of — change. The qualities that learners need have thus been dubbed “21st Century” in nature — not because these were of no use before (although they may take forms today which are novel) — but because of their critical importance in jobs involving sensemaking and creativity.

This sets the challenging context for understanding the potential of Learning Analytics approaches for the formative (and possibly summative) assessment of 21st century competencies, which are important precisely because they need to be displayed in interpersonal, societally and culturally valid contexts. By definition, the concept of assessing qualities that are lifelong and spanning the ‘arc of life’ inside and beyond formal learning demands new kinds of evidence. Computational support for tracking, feeding back, and reflecting learning processes holds the promise that these qualities can be evidenced, at scale, in ways that have been impractical until now.

Framed thus, the goal is to forge new links from the body of educational/learning sciences research — which typically clarifies the nature of the phenomena under question using representations and language for researchers — to documenting how data, algorithms, code and user interfaces come together through coherent design in order to automate such analyses — providing actionable insight for the educators, students and other stakeholders who constitute the learning system in question.

Quantifying these deeply personal qualities in order to feed back and strengthen them, without in the process reducing them to meaningless statistics, is the heart of the learning analytics challenge. How does one gather data from a diversity of life contexts, as potential evidence of these new competencies? How do we translate theoretical constructs with integrity into algorithms? How can they be rendered for human interpretation, by whom, and with what training? Should such analytics be used primarily for formative assessment, or should we be aiming for summative grades? Who gets to design the analytics, and who gets to validate them? Do analytics of this sort raise new ethical dilemmas?

Contributions are invited to this special issue to document and advance theory, design methodology, technology implementation or evidence of impact, including but not limited to:

Analytics for higher order competencies such as critical thinking, curiosity, resilience, creativity, collaboration, sensemaking, self-regulation, reflection/meta-cognition, transdisciplinary thinking, or skilful improvisation
Theoretical arguments around the opportunities, or indeed the limits, for analytics in illuminating particular competencies
Principles and methodologies for combining complementary analytical approaches, including reflections on conventional educational assessment instruments, and computational approaches
Methodologies for validating analytics
Analytics for learning dispositions/mindsets/“non-cognitive” factors known to shape readiness to engage in learning
Analytics for different kinds of authentic assessment and inquiry-based learning
Technological challenges and opportunities for lifelong, life-wide learning analytics extending beyond formal educational contexts
Arguments regarding whether analytics could effect a shift in the assessment regimes, and associated pedagogies and epistemologies, promoted by conventional education policy
Analysis of the systemic organisational adoption issues for such analytics
Visualisation design for different user groups, in particular, to promote increasing learner self-awareness and capacity to take responsibility for one’s learning

Assessing learning dispositions/academic mindsets

A few years ago Ruth Deakin Crick and I spent a couple of days with the remarkable Larry Rosenstock at High Tech High, and were blown away by the creativity and passion that he and his team bring to authentic learning. At that point they were just beginning to conceive the idea of a Graduate School of Education (er… run by a high school?!). Yes indeed.

Now they’re flying, running the Deeper Learning conference in a few weeks, and right now, the Deeper Learning MOOC [DLMOOC] is doing a great job of bringing practitioners and researchers together, and that’s just from the perspective of someone on the edge who has only managed to replay the late night (in the UK) Hangouts and post a couple of stories. Huge thanks and congratulations to Larry, Rob Riordan and everyone else at High Tech High Grad School of Education, plus of course the other supporting organisations and funders who are making this happen.

Week 7 coming up is focused on Assessing for Deeper Learning, and I’m very much looking forward to hearing from the panellists. So in this post on Learning Emergence, I share some thoughts about how we go about assessing what some of us call lifelong learning dispositions, while others refer to academic mindsets, and welcome comments from DLMOOCers and others.

Moving: children taking charge of their learning

For me, there’s nothing quite so moving in the world of learning than witnessing teachers give young people the responsibility and safe freedom to show what they are capable of, and then seeing those children talking about how that felt, and showing what they did.

Tragically our national curricula often crush this possibility out of school life, but when it happens, it’s startling. And the network of people sharing the evidence and stories is building. The movie below is an example from my local primary school, with whom I work:

Bushfield Y5 Fashion Project 2013 from Bushfield School on Vimeo.

Full details:

Small, T., Shafi, A. and Huang, S. (2014) Learning Power and Authentic Inquiry in the English Primary Curriculum: A Case Study, Report No. 12, ViTaL Development & Research Programme, University of Bristol. [pdf]

This report documents progress in a two-year action-research programme at Bushfield School, Milton Keynes, with two main purposes: firstly, to build on the School’s success in developing children’s capacity to learn; secondly, to track and measure the impact of its interventions for this purpose. The school combined the Effective Lifelong Learning Inventory (ELLI) with the Authentic Inquiry learning methodology from University of Bristol. Qualitative and quantitative data are combined to examine the impact of the pilots from the perspective of staff and pupils, comparing learning power against a range of demographic and attainment datasets, in the distinctive context of a primary school already experienced in the Building Learning Power approach.

One community who are active right now is the Deeper Learning MOOC (DLmooc), which is doing a fabulous job of catalysing conversations about the future shape of learning, with a good dose of student voice. Check out the YouTube/Vialogue replays of their live Hangouts.

I just added a couple of stories to the set they’re compiling, from the above work with Bushfield School, which has benefited from working first with Guy Claxton on ‘building learning power’, and most recently with Ruth Deakin Crick on assessing learning dispositions and authentic inquiry, both strands of work originating from University of Bristol Grad School of Education (disclaimer — I’m a visiting fellow there: now you know why!):

I have no idea how “massive” this MOOC is, but they’re doing a good job in this course of bringing together reflective practitioners and students.

Dispositional Learning Analytics wkshp

TODAY at LASI13, Stanford University…

Full details including background papers and websites:
learningemergence.net/events/lasi-dla-wkshp

GOALS

This LASI workshop does not assume any prior knowledge of learning dispositions, although background papers and resources are provided in advance for those who wish. The goal is to forge new connections between people already working in the field, and spark new conversations for the rest of LASI and beyond, hopefully, growing into new initiatives.

Goals:

introduce research on how students’ and educators’ dispositions to learning can shape outcomes
describe software tools grounded in that research, which enable the techniques to be deployed at scale
review the impact of such tools on learners and educators show how, as a by-product of web delivery, one can build quality datasets
describe how this data is amenable not only to traditional educational analysis, but explore the prospects for using machine learning and big data approaches
consider user interfaces which enable different stakeholders (eg. learners; educators; researchers) to interact with that data coherently
looking to the future, what ideas can we brainstorm for the tough questions this field faces, e.g. Can we develop dispositional analytics based on learners’ activity traces (rather than self-report)? Can we move from analytics, to recommendation engines able to make timely interventions for educators, or guidance to learners?

SPEAKERS

Organiser: Simon Buckingham Shum – Professor of Learning Informatics, Knowledge Media Institute, The Open University, Milton Keynes, UK

Ruth Deakin Crick – Reader in Systems Learning & Leadership, Centre for Systems Learning and Leadership, Graduate School of Education, University of Bristol, UK

Chris Goldspink – Director and Chief Scientific Officer, Incept Labs, Sydney, AUS

Nelson González – Co-Founder & Chief Strategy Officer, PiersonLabs, San Francisco, USA

Dave Paunesku – Founder, PERTS Lab, & Dept. Psychology, Stanford University, USA

PROGRAMME OVERVIEW

2pm: Welcome — Simon Buckingham Shum

2.15pm: Dispositional Learning Analytics: Learning Power & Complex Systems — Ruth Deakin Crick and Simon Buckingham Shum

2.35pm: Open discussion

2.45pm: Measuring & Changing Student Psychology Online: Lessons from a Scale-up Project — Dave Paunesku

3.05pm: Open discussion

3.15pm: Layers, Loops and Processes: Multi-level Analytics in Learning Systems — Chris Goldspink and Ruth Deakin Crick

3.35pm: Open discussion

3.45pm: Learner Dispositions: Big Data Meets Focused Social Science Research — Nelson Gonzáles, Chris Goldspink and Ruth Deakin Crick

4.05pm: Open discussion

4.15pm: Closing discussion on key issues, and where next?

4.30pm: Close

…sparking many conversations for the rest of the week, and beyond!…