This post was sparked by the international condemnation of George Floyd’s death, and the many others who came before him. Many communities and institutions are now reflecting on how structural racism manifests in their work (e.g. see SoLAR’s BLM statement and resources to help members learn more).

This is a tentative step into issues of race, about which I should declare I have no academic grounding. Nonetheless, it is important to ask what the implications are for a specific form of Learning Analytics, namely the predictive modelling of student outcomes. Should demographic attributes such as ethnicity be explicitly modelled, or should the models be “colour-blind”? While all categories have politics, this struck me as an interesting question, given that such techniques are demonstrating their value specifically in levelling the university playing field for all students. 

With thanks to Madi Whitman, Bart Rienties, Marti Hlosta and Paul Prinsloo for initial fact-checking and feedback. All comments are welcomed via this blog (moderated), the twitter thread or the LA Google Group thread.

---

“Be more white. Be more male. Be wealthier. Those are the biggest correlations with success. It’s terrible, but it’s the truth.”
[12] (p.1)

“Classification systems provide both a warrant and a tool for forgetting […] what to forget and how to forget it […] The argument comes down to asking not only what gets coded in but what gets coded out of a given scheme.”
[13] (pp. 277, 278, 281)

Since the emergence of Learning Analytics (c.2011) as both an intellectual community and commercial marketplace, an influential strand of work in higher education has been the use of predictive analytics, that is, developing computational models to identify students who look statistically likely (i.e. on the evidence of similar past cohorts) to be struggling, at risk of failing, or even dropping out. This is a dominant form of analytics inherited from the business world and machine learning, where it is highly lucrative to be able to predict the likelihood of, for instance, a customer buying a product or switching service provider — and take anticipatory action to change that possible future. So why not do the same for education?

Debate surrounds the ethics of such models in higher education, a particular version of broader concerns around the “datafication” of education through analytics, and now AI. The issues are complex, but examples of constructive dialogue are emerging, in which Learning Analytics and AI in Education engage with such critiques (e.g. these recent edited collections [2-4]).

Predictive modelling intersects with questions around the profiling of students, one attribute being ethnicity, which is what I want to focus on here given the current times we’re in, just a few weeks after the death of George Floyd at the hands of the police.

High profile success stories serve as iconic posters for the use of predictive modelling of student outcomes. Consider the Georgia State University Graduate Progression Success Advising program. It’s not called GPS by accident: the predictive model alerts student support teams when students look like they’ve ‘missed a turning’ (to push the metaphor) and off-course. An example screen from the system is shown below.

Discipline-level, cohort summaries of Low, Medium and High risk levels in the Georgia State University Graduate Progression Success Advising program.

Intriguingly, with regard to the question of racial colour-blindness, there’s a strong social justice angle that challenges head-on the demographically-related achievement gaps that many universities know only too well. Tim Renick, VP (Enrollment) at Georgia State University is unapologetic about GSU’s mission, and the GPS Advise website proclaims the sophistication of the analytics that help to power this:

“We have eliminated achievement gaps. For the last four years, we have been the only national university at which black, Hispanic, first-generation and low-income students graduated at rates at or above the rate of the student body overall. Georgia State is showing, contrary to what experts have said for decades, that demographics are not destiny.

Students from all backgrounds can succeed at comparable rates. Predictive analytics have helped all demographic groups graduate at higher rates from Georgia State, but just as critically, they have helped to level the playing field for all of our students.”

The irony will not be lost on those concerned about the datafication of education. Here we have analytics helping to level what historically has not been a level playing field for all students. When tools such as this are used intelligently, as aids for student support teams who are very much in the intervention loop, producing impressive outcomes for historically minoritized groups such as these (evidence which is not contested to my knowledge) — well, what’s not to like?

Another mature example of the process of embedding a predictive modelling tool into work practices is from The Open University UK (webinar / paper / paper [6, 7]). Working with online distance learning students, most of them mature students returning to academic study long after leaving high school, and including a high proportion of students with accessibility needs, the OU team has shown that compared to staff who did not use OU Analyse to monitor student progress, those who did contacted them more, with higher success rates [5]. Again, here we have analytics helping traditionally disenfranchised cohorts.

A screenshot from the OU Analyse dashboard, showing the risk of each student not submitting an assignment, their predicted grade, and their probability of passing or failing the course. (Figure 2 from [7])

Having set the scene, I want to focus on a specific decision that has to be made in such work, which I’m framing as follows:

Should predictive models of student outcome be “colour-blind”?

Two sides of the debate go something like this:

YES: MODELS SHOULD IGNORE HISTORIC INJUSTICES. Predictive models should ignore demographic attributes, which are well known to be highly predictive of outcomes, but students obviously have no control over their ethnicity, high school, being first-generation-in-family at university, etc. It’s clearly unethical to classify students as higher risk from day 1 for those reasons, immediately placing them in the shadow of inequitable historical patterns. They’ve got to university, possibly demonstrating greater resilience than their more privileged peers, so we wipe the slate clean. What counts is what they do when they walk through the door, some of which can be tracked by analytics through digital activity traces. Such models can therefore be declared to be “colour-blind”: ethnicity is not modelled explicitly, and nor are any other known proxies (e.g. Zip code; High School).

NO: MODELS SHOULD REFLECT BUT NOT PERPETUATE ALL KNOWN FACTORS. Predictive models of student success/risk should include demographic variables, since they greatly improve the model’s performance. It is myopic to ignore this, just as we should not ignore science and social science when they provide solid evidence of other difficult truths about societal inequities. The student’s demographics are not held against them, but rather, used to improve their chances. We should thus model student risk as comprehensively as possible, with our ethical ‘eyes wide open’, forearmed to use this knowledge in the students’ best interests, with strong ethical principles to ensure that competing interests are not allowed to influence decisions (e.g. a student’s need for extra support has resource implications).

Until recently, I thought of these positions as rather polarised. But a third analysis struggles with an unequivocal yes or no. This view problematises the goal of even trying to achieve colour-blindness:

BEING “COLOUR-BLIND” ≠ BEING ETHICAL

I’ll state very clearly that I’m brand new to reading anything academic about racism. As a result of reading sparked by George Floyd’s murder, I only just became aware of the work of people like Eduardo Bonilla-Silva on the nature of white privilege and structural racism, and at this point, have only managed to read various summaries and reviews of his influential book, Racism without racists: Color-blind racism and the persistence of racial inequality in the United States [1]. He argues:

“Whereas Jim Crow racism explained blacks’ social standing as the result of their biological and moral inferiority, color-blind racism avoids such facile arguments. Instead, whites rationalize minorities’ contemporary status as the product of market dynamics, naturally occurring phenomena, and blacks’ imputed cultural limitations” (p.2).

“Much as Jim Crow racism served as the glue for defending a brutal and overt system of racial oppression in the pre-Civil Rights era, color-blind racism serves today as the ideological armor for a covert and institutionalized system in the post-Civil Rights era” (p.3)

Colour-blind racism operates through:

1. liberalism (markets are open to all and do not discriminate)
2. naturalization (people “naturally” segregate themselves from other racial groups)
3. cultural racism (minorities participate in self-defeating behavior) and
4. minimization of racism (racism is no longer prevalent to address, specifically).

I found another article fascinating, introducing critical race theory to reflect on how academia functions, specifically HCI, a sister field to Learning Analytics (which just won CHI’20 Best Paper) [9]. In their summary of critical race theory, the authors also note Bonilla-Silva’s point (1) above:

“Liberalism itself can hinder anti-racist progress [34]. Liberalism’s very aspirations to color-blindness and equality – while admirable – can impede its goals, as they prohibit race-conscious attempts to right historical wrongs. In addition, liberalism’s tendency to focus on high-minded abstractions can lead to neglect of discrimination in practice.” (p.3)

These ideas raised the question in my mind: does making our computational infrastructure “colour-blind” merely perpetuate systemic discrimination in universities? So I was delighted to read the work of Madi Whitman [12], who presents an ethnographic account of how a university made its modelling decisions. There are some interesting quotes from the data science team, which I suspect might be echoed by many others, who are trying to make ethical decisions. First they are aware of the uncomfortable truth, as are many universities:

“Be more white. Be more male. Be wealthier. Those are the biggest correlations with success. It’s terrible, but it’s the truth.”

—Excerpt from interview with Don, a university administrator [12] (p.1)

Since the predictive model drives automated nudges to the students, they try to do the right thing (for the YES camp) — exclude demographic attributes over which students have no control:

“Socioeconomic status things. Demographic markers. But they’re all things that either because it’s too late in the game, we can’t tell a student, “Boy, it would have been great if you would have studied harder in high school.” And we certainly can’t tell a student on a demographic or socioeconomic thing, we can’t say, “Hey, it’d be good if you weren’t so poor.” There’s nothing a student can do with that. Even though it does put ‘em in a higher risk category. So we took those things that were malleable by the students. Things like, how much time they were spending on campus. Whether they were a proxy for whether we believed they were paying attention in class by how much data they were downloading in a class.” (p. 6)

Note the strong argument for student agency, which is a principle valued in much ethical discourse in Learning Analytics, and Human-Centred Design thinking. The student should be in control:

“I guess that we assume that what [students] did in the course of the day, they had control over. Right, so they chose whether they were gonna eat or not . . . they chose the gym or not, being on campus or not . . . They chose living where they chose to live. I think they have some say in that…So it seemed to me that any time that they had an opportunity to make a decision about what they were going to be doing, we called that a behavior.” (p.7)

Whitman helps us understand that while the analytics team sees this as the ethical response, it’s a double-edged sword: do they really have that level of control? She argues that:

“Because attributes are removed from the model and nudging, the reliance on behaviors suggests that students’ choices are at the heart of their success at the institution. Because demographic data are not incorporated into the predictive model at all, success is linked with behaviors and students’ choices. The purposeful presentation of data to students encourages students to internalize those data and act on them. As such, responsibility now rests on the students to take hold of their success.” (p.10)

If you are in the YES camp, this is exactly the goal. Level the playing field, we don’t care what colour you are, everyone is must take responsibility for their study habits, level of engagement, assignment submission, etc.

However, might this not also resonate with items 1, 3 and 4 in Bonilla-Silva’s work introduced above? The university and its learning platforms are framed as “open markets”, with opportunity for all (1); if students do not make wise choices, they only have themselves to blame (3), because we’ve erased racism from the algorithms (4):

1. liberalism (markets are open to all and do not discriminate)
2. naturalization (people “naturally” segregate themselves from other racial groups)
3. cultural racism (minorities participate in self-defeating behavior) and
4. minimization of racism (racism is no longer prevalent to address, specifically).

So Whitman with her modelling case study, and Bonilla-Silva in general, are questioning whether students from historically marginalised groups are really as autonomous and agentic as their more privileged peers. Whitman concludes:

“The visualizations of certain kinds of data—namely data students ought to use to inform their everyday decision-making—and obscuring of demographic data place the burden of responsibility and success on students. By minimizing the role that race, class, and gender play on graduation outcomes, the institution, through the model, can present behaviors as major factors in the likelihood of a student grad- uating within four years. If students do not attend class, a low GPA is a consequence of that decision.

Thus, the constraints around choices become invisible. The university and its existing inequalities start to vanish because success is placed in the hands of students. Social climate problems, structural barriers, issues of belongingness, and resource shortages disappear. A student cannot cite external factors in this model of success dominated by behaviors. The result is a shift in a locus of responsibility, wherein nudging is meant to give students tools to manage themselves and regulate their own behavior based on insights they ought to draw from their data.” (p.10)

WAYS FORWARD?

There seem to be some questions that could be asked, as a way to move this forward.

Does anyone contest the positive outcomes for students from the use of predictive models?

For instance, when GSU reports the startling impact of the GPS Advising initiative, is anybody questioning the figures? Is anyone questioning the claim that the algorithm has a pivotal role to play in this, rather than the impressive level of human support available to students? At the Open University, we knew that simply calling a student increased the chances of a positive outcome.

What is the purpose of the modelling?

If you’re designing automated nudges for students (as in the Whitman case study), clearly, there’s no point nudging them based on their static demographic history, so removing such attributes from the model seems uncontroversial in modelling terms. Whitman, of course, is concerned about this erasure (but see next section as to whether this is justified).

If you’re designing a model to understand the spectrum of challenges students face, in order to understand how to support them, then ignoring demographics becomes problematic. The UK Open University’s Student Probability Model  [7] was developed for financial forecasting, assessing the likelihood of a student still being enrolled as the course unfolded (sometimes over years for part-time students). This took into account deprivation indices, which could of course be a proxy for race in some contexts, but erasing this would simply lead to more erroneous financial forecasts. We should ask (perhaps even more so in these straightened times for universities) if it is in anybody’s interests for universities not to budget as accurately as possible.

The OU Analyse predictive model also takes into consideration a range of demographic variables including socio-economic and ethnic when making the first initial predictions, before a course starts. However, nearly all of the demographic factors quickly lose relevance once actual engagement and behavioural data is gathered when a course begins, in particular once the first assessment deadline has passed. Furthermore, previous credits obtained is mostly more predictive than any demographics. Interestingly, while the OU Analyse team has wanted to remove demographics given the limited additional variance its explains, those teaching on the front line apparently prefer to retain this, since it helps them to ‘colour in’ their picture of a student. Ethical arguments for both the Yes and No camps?

Given this tension between quant and qual drivers, it seems particularly important to understand when and why predictive models fail, through close qualitative analysis (see this recent example from the OU team [8]), as well as to understand in detail the experiences of the student support teams who use – or are expected to use – the outputs predictive models (e.g. [5]).

Is any real harm is caused by colour-blind modelling?

Whitman argues that in principle, an unfair burden is imposed on marginalised students if we assume they have the same capacity as their more privileged peers to respond to nudges and make wise choices. There is plenty of evidence that marginalised groups are not as free to make the same life-choices as more privileged whites, but is there any empirical evidence yet regarding student choices in response to automated nudges? I don’t know any yet.

One size does not fit all: students with the same demographics may still be very diverse

A black student may be working from home, in very poor physical and emotional conditions, poor computing and network access, struggling financially, commuting long hours, with dependents to care for. That student is clearly battling constraints that others are not, which will seriously affect how much “control” they have over their choices, through no fault of their own.

• This is all invisible in the colour-blind model (YES camp). It is visible when we model such metadata (NO camp) and could be taken into account.

Another black student may have a generous scholarship, living on campus, free from carer responsibilities, and able to seize every opportunity that comes their way.

• This seems to be the default assumption behind colour-blind student modelling — and that is precisely the point.

Should we just stop using predictive models in education?

Despite the flagship examples, perhaps the potential for poorly implemented predictive modelling is so high that they’re best steered clear of. It’s complex both technically and ethically. A range of ethical concerns not covered includes:

• One size does not fit all. A body of evidence now demonstrates that a predictive model for one course does not translate smoothly to other courses. Differences in discipline, cohort, pedagogy and learning design introduce myriad variables.
  But within a given course, things are simpler, surely?
• We don’t necessarily want to teach the way we always have. Predictive models assume that historically stable patterns are a reliable predictor of the future. But even within a course, this is not always true, since teaching staff, curriculum and pedagogies change. Indeed, many universities are trying to shift the way their staff teach and assess to more future-focused pedagogies. Innovations by definition break from the past, and so will likely break the predictive model, and the last thing we want is for our analytics to act as a brake on improving teaching. In our pandemic-afflicted world, predictive models based on a blended pedagogy with on-campus students, are unlikely to translate smoothly to 100% online students, working from diverse timezones (but that is ultimately, an empirically testable question).
• Risk of misclassification. As in all areas of society where algorithms are classifying people, there is growing concern over the risk of being misclassified. Who wants a High Risk of Failure flag on their record, even before they start their studies? Is that flag really deleted, or saved to help validate future models? And could that classification be leaked to other entities, who could use it inappropriately?
• University lacks the capacity to act. Prinsloo and Slade argue that a university has at least a moral, if not legal, obligation to act if it believes a student is at risk of failure. Predictive models, when valid, thus place a new burden on universities [11]. A key take-home from mature case studies such as GSU and the OU clarify the investment in people, processes and tools required to deliver on this.

So, there are significant risks that universities could buy predictive modelling products like any other ed-tech, but either use them badly, or if they are tuned well, still cannot act on what the dashboards are telling them, thus opening themselves up to charges of negligence. Perhaps it’s better not to know tens of thousands of students’ risk profiles in such precise terms…

Many universities choose instead to focus on other forms of analytics that make visible student activity in helpful ways, to both educators and students, provide educators with tools to intervene with personalised feedback at scale [10], but make no attempt to build a risk profile. That profile is left implicit, inferred by (hopefully well trained) student support mentors and educators.

What do students think?

I’ll close with this obvious question, but not one with any empirical evidence I know of. Let’s bring diverse students into the conversation and consult with them on these matters. Learning Analytics is beginning to introduce human-centred design methods that give a voice to students, and as with any co-design process, this requires learning, and listening, by all stakeholders. However, I do not know of any that engages students around predictive models in particular, and issues of race specifically.

How do students from diverse backgrounds engage with questions such as these?…

• Do you want to be treated by the university just like any other student? Or should the university be recognising that you come from very different backgrounds, live in very different conditions, facing very different challenges day-to-day?
• This extends into our IT systems: what do you think about analytics that continuously predict your likelihood of success, to maximise the support we can give you? Demographics including ethnicity and postcode can help improve such models, and help us ensure that outcomes are equitable for all students – does that seem reasonable? 
• Are you surprised or shocked, or would you expect no less from a technically advanced university?
• Are you happy to trust that the university will behave ethically, or do you want more transparency? How much do you want to know about the data we have and how we use it, and how much control do you want over this data?

References

[1] Bonilla-Silva, E. Racism without racists: Color-blind racism and the persistence of racial inequality in the United States. Rowman & Littlefield Publishers, 2006.

[2] Buckingham Shum, S. Critical Data Studies, Abstraction & Learning Analytics: Editorial to Selwyn’s LAK keynote and invited commentaries. Journal of Learning Analytics, 6, 3 (2019), 5-10 https://doi.org/10.18608/jla.2019.63.2

[3] Buckingham Shum, S., Ferguson, R. and Martinez-Maldonado, R. Human-Centred Learning Analytics. Journal of Learning Analytics, 6(2), 1–9. . Journal of Learning Analytics, 6, 2 (2019), 1-9 https://doi.org/10.18608/jla.2019.62.1

[4] Buckingham Shum, S. and Luckin, R. Learning analytics and AI: Politics, pedagogy and practices. British Journal of Educational Technology, 50, 6 (2019), 2785-2793 https://doi.org/10.1111/bjet.12880

[5] Herodotou, C., Rienties, B., Boroowa, A. and Zdrahal, Z. A large‑scale implementation of predictive learning analytics in higher education: the teachers’ role and perspective. Educational Technology Research Devevelopment, 67 (2019), 1273–1306 https://doi.org/10.1007/s11423-019-09685-0

[6] Herodotou, C., Rienties, B., Hlosta, M., Boroowa, A., Mangafa, C. and Zdrahal, Z. The scalable implementation of predictive learning analytics at a distance learning university: Insights from a longitudinal case study. The Internet and Higher Education, 45 (2020), 100725 https://doi.org/10.1016/j.iheduc.2020.100725

[7] Herodotou, C., Rienties, B., Verdin, B. and Boroowa, A. Predictive Learning Analytics ’At Scale’: Guidelines to Successful Implementation in Higher Education. Journal of Learning Analytics, 6, 1 (2019), 85-95 https://doi.org/10.18608/jla.2019.61.5

[8] Hlosta, M., Papathoma, T. and Herodotou, C. (2020). Explaining Errors in Predictions of At-Risk Students in Distance Learning Education. Proc. International Conference on Artificial Intelligence in Education (AIED 2020), pp 119-123. https://link.springer.com/chapter/10.1007/978-3-030-52240-7_22

[9] Ogbonnaya-Ogburu, I. F., Smith, A. D. R., To, A. and Toyama, K. Critical Race Theory for HCI. In Proceedings of the Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(Honolulu, HI, USA, 2020). Association for Computing Machinery. https://doi.org/10.1145/3313831.3376392

[10] Pardo, A., Bartimote, K., Buckingham Shum, S., Dawson, S., Gao, J., Gašević, D., Leichtweis, S., Liu, D., Martínez-Maldonado, R., Mirriahi, N., Moskal, A. C. M., Schulte, J., Siemens, G. and Vigentini, L. OnTask: Delivering Data-Informed, Personalized Learning Support Actions. Journal of Learning Analytics, 5, 3 (2018), 235-249 https://doi.org/10.18608/jla.2018.53.15

[11] Prinsloo, P. and Slade, S. An elephant in the learning analytics room: the obligation to act. In Proceedings of the Proceedings of the Seventh International Learning Analytics & Knowledge Conference(Vancouver, British Columbia, Canada, 2017). Association for Computing Machinery. https://doi.org/10.1145/3027385.3027406

[12] Whitman, M. “We called that a behavior”: The making of institutional data. Big Data & Society, 7, 1 (2020), 1-13 https://doi.org/10.1177/2053951720932200

[13] Bowker, G. C. and Star, L. S. (1999). Sorting Things Out: Classification and Its Consequences. MIT Press, Cambridge, MA.

 

Categories: Politics, Software, Theory
Tags: ethics, learning analytics, politics, predictive modelling, race