Simon.BuckinghamShum.net

Reflections

EAIR is a new community to me. Like most other academics, I work in a university but I don’t study university policy, management and leadership as an academic. However, while I do lab evaluations, my knowledge media research has always been focused on sensemaking–support systems in authentic contexts — I want real users to choose my tools out of choice.

So, as my research has shifted into learning analytics, within the OU I’m spending increasing time forging the links with key stakeholders (VC Exec; enterprise data warehouse; frontline learning systems/VLE managers; institutional researchers; student experience teams; other predictive modellers). It made sense to connect with the wider community of their counterparts, especially those who are reflective enough to want to research the dynamics of their institutions, in the context of turbulent global higher education sector.

EAIR set out its focus as follows:

The central theme of the 2013 EAIR Forum is “The Impact of Higher Education” and the subtitle is: “Addressing the challenges of the 21st century”. This theme will be analysed and discussed from four different angles: governance, students and staff, performance, and society. From the governance perspective, a key question is how governmental policy can contribute to increase the impact of higher education. Impact not solely refers to research (useful knowledge), but also to teaching and learning (impacts of graduates). Obviously, governments affect the impact by e.g. policy measures regarding access to higher education and measure with respect to research funding. Second, from the student and staff perspective, there is still a lot to gain regarding the improvement of quality of education and the preparation of students for an increasingly faster economy and society. For academic staff, the question is how to be able to meet the various challenges: publish or perish, the acquisition of research grants, but also regarding the changing demands of contemporary students. The second perspective ties in with the third perspective: performance. There is increasing attention to measurement of performances of students, staff and higher education institutions. Student ratings of teaching and learning, national and global rankings and journal impact scores are only a few examples of this. A salient question is whether we should and, if so, how we can measure impact. The final perspective links higher education to its societal stakeholders and a central question will be which relationships with its stakeholders will have the most effective impact.        

These are focused by a set of tracks:

Track 1: Governance: impact follows strategy?

Track 2: Student learning and the student experience

Track 3: Institutional research: how to measure impact?

Track 4: Quality management in higher education

Track 5: Ranking: excellence and reputation management

Track 6: Access to higher education

Track 7: The impact of higher education on the economy

Track 8: Staff development and staff experience

Arriving at the conference with just Thurs evening and Friday to tune in, I was very much on a sensemaking mission, trying to hear the ways in which they articulate challenges about which I might have something to say. I met a handful of technologically sophisticated delegates, but most were not in that space, bringing expertise around governance, policy and the myriad other disciplines that underpin a university, or a national/global higher education sector.

Learning analytics clearly has some things to say about:

• Student experience
• Methods, metrics and methodologies
• Measuring impact
• Performance indicators
• Data collection and validity

There is much critical discourse (indeed a whole Track 5) around the impact (indeed for many here it seems, the distorting effects) of national and international league tables. This concern with the ways in which one classifies and counts intellectual work at faculty and institutional level, has a direct parallel with the critical eye needed on learning analytics at the student level.

I talked about this in terms of EAIR’s focus currently being on the macro- and meso-levels of data and analytics, with learning analytics now heralding a data explosion at the micro-level: student interaction with curriculum, resources and peers.

Several papers, and conversations I had, flagged the lack of a university culture that seeks to make evidence-based decisions, or in which information is available but never consulted, or in which the data collection/analysis/feedback cycle is labour-intensive. As we move to digitally instrumented physical spaces, learning platforms and indeed lives (cf. quantified self/lifelogging movement), we will begin to accelerate this loop. It will become increasingly practical for academics and researcher-practitioners to see the impact of their learning designs.

On this front, I flagged the potential contribution of complex systems thinking, and how concepts such as resilience and accelerated feedback loops are central to an organism’s capacity to adapt to a turbulent environment. I was excited to see the flyer for a forthcoming book from Rosalind Pritchard and Jan Erik Karlsen on Resilient Universities, the first to bring to bear organizational resilience thinking on HEIs.

The best newcomer award went for the second year running to Lisa Degn, a doctoral student who uses Karl Weick’s foundational work on organizational sensemaking as a framework for analysing her interviews with senior university leaders such as faculty deans. That’s a pretty impressive track record for a PhD student! The sensemaking connection obviously resonated with my own work, and I emphasised how infused with human judgement the data capture and analysis cycle is. As George Siemens emphasises, analytics is really all about sensemaking, hence the key role of good information visualization.

Even as I reflected on the need for universities to become more like complex adaptive systems, with much faster sensing capability and feedback loops, I emphasised that it’s myopic to worry about preventing the extinction of universities unable to carve an ecological niche for themselves, if in the process we continue to turn out graduates who do not themselves have those qualities. This begs the question whether learning analytics can illuminate these higher order, transferable qualities and skills, to which I argue the answer is yes, with examples such as quantifying proxies for social capital (social network analysis), fluency in the use of language for collaborative knowledge building and critical thinking (discourse analytics), and growth-oriented mindset/readiness to seize ‘arc of life’ learning opportunities (dispositional analytics).

EAIR is a diverse conference, with a huge range of topics and methodologies, some of which will strike learning analytics researchers as rather ‘soft’, although this is in part a reflection of the human processes they study. I suggest that analytics brings the possibility to ‘harden up’ the evidence base that some institutional researchers seek. EAIR’s deep connections into university governance make them a strategically important community for raising the profile of analytics with senior leaders, and studying the processes of embedding analytics into organizational life.