Black Box Learning Analytics? Beyond Algorithmic Transparency

ABSTRACT:

As algorithms pervade societal life, they’re moving from an arcane topic reserved for computer scientists and mathematicians, to the object of far wider academic and mainstream media attention (try a web news search on algorithms, and then add ethics). As agencies delegate machines with increasing powers to make judgements about complex human qualities such as ’employability’, ‘credit worthiness’, or ‘likelihood of committing a crime’, we are confronted by the challenge of “governing algorithms”, lest they turn into Weapons of Math Destruction. But in what senses are they opaque, and to whom? And what is meant by “accountable”?

The education sector is clearly not immune from these questions, and it falls to the Learning Analytics community to convene a vigorous debate, and devise good responses. In this tutorial, I’ll set the scene, and then propose a set of lenses that we can bring to bear on a learning analytics infrastructure, to identify some of the meanings that “accountability” might have. It turns out that algorithmic transparency and accountability may be the wrong focus — or rather, just one piece of the jigsaw. Intriguingly, even if you can look inside the algorithmic ‘black box’, which is imagined to lie in the system’s code, there may be little of use there. I propose that a human-centred informatics approach offers a more wholistic framing, where the aggregate quality we are after might be termed Analytic System Integrity. I’ll work through a couple of examples as a form of ‘audit’, to show where one can identify weaknesses and opportunities, and consider the implications for how we conceive and design learning analytics that are responsive to the questions that society will rightly be asking.

[Compressed PDF slides 3.7Mb] [Powerpoint slides 27.9Mb]

CONTEXT:

In 2016 I started giving briefings on the meaning(s) of algorithmic accountability in education. This evolved into a tutorial that I ran at the 2017 Learning Analytics Summer Institute, a version of which was recorded at U. Michigan MOOC studios, but for various reasons, never edited together. I’m pleased to say that (thanks to our intern Ran Ding!) this is now available as a Creative Commons licensed resource. Reuse, chunk and remix please!

Since 2016, activity around the ethics of Big Data/AI has exploded in an encouraging way, with many accessible resources becoming available (e.g. Data & Society Institute; AI Now Institute), as well as the emergence of the FATE (Fairness, Accountability, Transparency, Ethics) conference and network. However, there remain few resources specifically on the nature of, and responses to, algorithmic transparency and accountability in education, so this talk still seems relevant, and I welcome your feedback on this fast moving challenge.

The next steps would be to develop learning activities around this material to assist deeper engagement, and again, I’d love to hear from you if you want to move this forward.

Leave a Reply

You can use these XHTML tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>