Team-Based Learning APAC keynote

Last week I had the pleasure of being welcomed into the Team-Based Learning community, at their Asia Pacific Community Symposium. While everyone is doing “team-based learning” of some sort, I was not familiar with TBL as a set of specific learning design patterns which have been refined across multiple contexts, with a dedicated and passionate community advocating and researching it, and dedicated platforms scaffolding the process.

They invited me to share how UTS has responded to GenAI, to which (given the community) I added a few pointers to work from my other work on Collective Intelligence and Teamwork Analytics. Slides below and as PDF.

GenAI for Critical Analysis: Practical Tools, Cognitive Offloading and Human Agency

I’m looking forward to LAK24 in Kyoto next month. In response to the GenAI workshop call for practical examples and experiences of GenAI, this short paper shares some examples from the last tumultuous year, with some brief reflections…

Buckingham Shum, S. (2024). Generative AI for Critical Analysis: Practical Tools, Cognitive Offloading, and Human Agency. In Joint Proceedings of LAK 2024 Workshops, co-located with 14^th International Learning Analytics and Knowledge Conference, March 18-22, 2024, Kyoto, pp. 205-213. https://ceur-ws.org/Vol-3667/GenAILA-paper4.pdf

Abstract: Generative artificial intelligence (GenAI) is now capable of performing tasks that we have considered intellectually demanding. There are justified concerns that this will undermine the agency of both educators and students, if tools are poorly designed, poorly used, or imposed — with consequences for education and the future of work. This short paper contributes practical examples pointing the potential for GenAI to promote critical analysis as part of intellectually demanding tasks, by both students and educators. However, this depends on appropriate usage. The paper then briefly discusses how we may balance the benefits and risks of human cognitive offloading to AI, as a perspective on human agency.

Conversational GenAI for argument analysis

History: making thinking visible

As you can tell from a quick scan of my site, I’ve spent a lot of time fascinated by how computers can make thinking visible (books Visualizing Argumentation and Knowledge Cartography), plus many papers and blog posts (on Argument Mapping and Dialogue Mapping), using experimental open source software we built (such as Compendium and Cohere). A big picture account can be found in this 2007 keynote, Hypermedia Discourse: Contesting Networks of Ideas and Arguments.

So, all that’s to say that making arguments visible so that you and others can — in a very real sense — “see what you’re saying” has been a career-long passion. A key challenge in this long field of research has been that rigorous thinking is hard work. Bad luck, welcome to university! Argument Mapping and its related techniques use the affordances of visual trees/networks as an extended, external memory to augment personal and collective intelligence. Making one’s ideas visible as coherent diagrams is also hard work — but it’s good pain — the cognitive and discursive effort this entails is designed to clarify one’s thinking by revealing visually where the weaknesses are, in ways that writing and reading chunks of prose cannot tell you at a glance.

Enter NLP and rhetorical parsing

In 2012, we were now in the Web 2.0 era, and an exciting collaboration with NLP and linguistics expert Ágnes Sándor (Xerox) led to a new conception of Contested Collective Intelligence. For the first time in my work, machines could identify argumentative moves in sentences, complementing the argumentative moves that our web annotation tools enabled for people — who unlike machines, can of course can ‘read between the lines’ and see connections between ideas that may not even be in the texts.

This was extremely exciting, and the ideas and open source code carried through to our current Academic Writing Analytics project and web apps. I reflected on the impact of encountering NLP colleagues, in the context of The Future of Text book.

Conversational generative AI

And so we arrive at generative AI based on large language models, which advances the state of the art in language processing and generation in so many ways. Moreover, the conversational paradigm, when a chat application is overlaid, opens so many interesting human-computer/personal-collective intelligence possibilities. I’ve been intrigued to play with GPT-4 to see what its argument analysis capabilities are.

Previously, I’ve shared some early experiments on ChatGPT-3.5’s ability to identify implicit premises in prose arguments, and critique a flawed argument by analogy. I’ve now had the chance to experiment a little with the version of GPT-4 that is Bing Chat, accessed via Microsoft Edge browser. I was dying to see how far I could get in generating an Argument Map from a written argument.

The task is a typical analysis workflow, as prep for teaching:

search for relevant sources
select one for analysis
extract key elements of the argument and their relationships (described using a structured markdown notation called ArgDown)
diagram them to show their key relationships (in the ArgDown web app)
discuss (with the AI)
start thinking about student activities to help them learn

I don’t mind admitting that watching a machine do this for the first time was startling! I tell the story here…

U21 2023 Educational Innovation Symposium Keynote from McMaster University (OFFICIAL) on Vimeo.

Deeper dive

Let’s take a closer look at what Bing Chat did, because it wasn’t perfect.

The gold stars signal what in my view are good summaries of what the authors said, correctly linked.
The blue info circles are “commentary” from Bing Chat about the arguments
The red crosses signal that the authors did not say this, it is a false reconstruction by Bing Chat.
The red underline signals classification of a premise using incorrect, or indeed made-up argument schemes. There is to my knowledge no such argument type as Argument from responsibility, or Argument from precaution. Argument from omission seems to be a jumbling of Fallacy of omission and Argument from ignorance.

If we take this node for example, it reads well as a summary:

However, the authors do not talk about researchers at all, they say:

“The letter addresses none of the ongoing harms from these systems, including 1) worker exploitation and massive data theft to create products that profit a handful of entities, 2) the explosion of synthetic media in the world, which both reproduces systems of oppression and endangers our information ecosystem, and 3) the concentration of power in the hands of a few people which exacerbates social inequities.”

As an amusing sidenote, Bing Chat was curiously resistant to recognising this, insisting that it was correct, first “quoting” a fabricated passage from the article to me, and then saying that this implied that the authors meant researchers. I thought that this sort of stubbornness had been ironed out after Bing Chat’s earlier escapades! More seriously, this points to the value of dialogic learning, with a partner who can be conversed with 24/7 — but who must still be treated with some caution, certainly at this stage of maturity.

To summarise:

Bing Chat showed intriguing capability, for a machine, to analyse an argumentative article:
- extracting the key claim and underlying premises, summarising them in own words
- generating markdown (ArgDown) showing supporting/challenging relationships
- (and without being asked to) attempting to classify some nodes using Walton’s Argumentation Schemes.
However it also introduced fallacious nodes (incorrect summaries of the authors, and incorrect commentary nodes), incorrect links, and argument classifications (inventing argument types, and/or misclassifying nodes).

This is an exploratory example, and more systematic evaluations are required, of the sort we see in the growing Argument Mining literature.

Reflections

It does feel to me that we’ve turned a corner in the long, wintry history of AI. Perhaps this is a passing summer, which will fade like the others. But in my own career, punctuated by eureka moments such as seeing my first Apple Mac, my first web page load, and an iPhone — this is up there.

University is to teach you to think. Argument analysis is serious intellectual work, of the sort that we would hope to see from our students. Nor is there always “one map to rule them all’ — a correct map, since like in spatial cartography, design decisions are made about scale and purpose. The point about knowledge cartography is that it provokes productive reflection and discourse. So even if the AI gets the map wrong (and it will), the conversation this should provoke should be useful. With colleagues Kirsty Kitty and Andrew Gibson, I’ve argued that embracing imperfection in tech can be productive if it promotes deeper critical thinking in learners, e.g., learning by correcting the automated output, or reflecting on questions it asks, or why it seems wrong. Students must, however, be scaffolded to engage in such activity.

Informal learning? This is feasible in formal education, but may be less attractive in other informal learning contexts where we want to promote critical deliberation, e.g. citizens engaged in a policy deliberation, many of whom lack the internal or external motivation to think that hard. But assuming future tools give more accurate argument maps/outlines, that require less debugging, perhaps we can see use-cases including:

assisting facilitators/educators to prepare learning resources for civic deliberations
assisting very engaged citizens to dissect complex arguments, and perhaps lowering the entry threshold for others who might otherwise not engage with such structured, critical deliberation
an article is very different to a multi-author conversation, but we can envisage summarising online discussions (NB: Teams is starting to summarise topics and actions in meeting transcripts)

Did we just supplant student cognition? From a learning sciences perspective, an overriding concern with generative AI is that it does too much cognitive work for the learner. Editing an AI-generated draft is not the same as wrestling with the blank page yourself. Ditto for reviewing an AI-generated argument map.

I have just done what many professionals have enjoyed doing in recent months: putting GPT through its paces to test its technical capability. But learners are not professionals: they don’t know what they don’t know. As I argue elsewhere, they may lack the knowledge, skills and dispositions to engage critically with AI output. They will require suitable scaffolding from mentors and teachers to learn what we mean by critical thinking and argument analysis, in order then to be equipped to use a power tool such as an argument mapping tool. Much empirical research awaits to test the affordances of generative AI like this, to establish when they are most useful to use developmentally, with a given age/stage of learner.

But we do know that argument mapping has struggled to gain traction (in formal education and among professionals) because it’s hard intellectual work. It could be that by generating full or intentionally incomplete argument maps, AI provides a step up for many learners to quickly get feedback on their work, or see examples of arguments about topics they are knowledgeable about — and thus better equipped to critique — compared to examples chosen by the teacher or textbook. Generative AI may open new possibilities because it can generate examples tuned to the interests of each learner, activating their curiosity to go deeper.

Your feedback is welcome, which is hosted on LinkedIn…

Compendium Archive & Network 2021

CONTEXT… Those of you who know my R&D will know that I have a long-standing interest in the role that software can play in human sensemaking around wicked problems, a form of “augmenting human intellect” (Doug Engelbart). A powerful example is structured, visual hypermedia that makes tangible the ways that ideas, data and arguments connect with each other and documents.

This is certainly a story about the evolution of an interactive visual tool for thinking—but far more interestingly, it’s about the co-evolution of software with a set of practices to develop human fluency with the tool. Those practices were (in historical order) Dialogue Mapping, Issue & Argument Mapping, and Knowledge Art.

Here’s how I told this story in 2014, tracing my work back to Engelbart’s vision of personal computing and collective IQ, and here’s a quick overview of key books and papers.

COMPENDIUM… provides extremely flexible hypermedia linking between conceptual objects (e.g. questions, ideas, arguments), data and documents (local/online), through a visual user interface. As a hypertext system for thinking with, connections can be made in multiple ways: spatial proximity and visual linking within a view, plus tagging and transclusions across views (i.e. embedding a node in multiple views). Views can contain each other non-hierarchically (A can contain B which can contain A). Search can be refined by node types and tags. This 2014 RAE Impact Case distills the research –> impact narrative.

It is an open source, desktop Java application with a full SQL database, with XML and SQL import/export, and HTML publishing. In the course of its development, it was interoperable with (at the time state of the art) Jabber (open source instant messaging) XML, and semantic web RDF.

HYPERTEXT HISTORY… Compendium is descended from the pioneering hypermedia system gIBIS (graphical Issue-Based Information System) at MCC Labs led by Jeff Conklin and Michael Begeman, which led to the commercial corporate memory product CM1, renamed Questmap. Compendium was then developed over the course of around 20 years R&D starting in the early 90s at Bell Atlantic labs (White Plains NY) led by Al Selvin and Maarten Sierhuis, continued at NASA Ames Research Centre by Maarten on the Mobile Agents project providing a human-agent science team tool, in collaboration with my team at the Open University’s Knowledge Media Institute (KMI) from 1995-2014. The gIBIS/CM1/Questmap/Compendium lineage exemplifies an influential strand of hypertext R&D that preceded the invention of the Web, which like Xerox NoteCards, exemplifies what Frank Halasz called hypertext for idea processing — focusing on visualising and managing the connections between nodes as a form of intellectual work. You can learn a lot more about the intellectual lineage of these ideas in this brief history, another account, on this blog via the compendium tag, and on the memorial archive of Compendium co-inventor, my colleague, PhD student and friend, Al Selvin.

COMPENDIUM INSTITUTE… The Compendium Institute coordinated the international user/developer network, feature requests, code releases, research and training. Software development has been on pause since 2013, and may well not be continued, since much of the world now expects web-based tools (indeed see the great work by DebateGraph, and the KMI team developed quite a few). However, there remains an active user base of people who value the speed and functionality of a Java desktop app which continues to run on current Mac/Win/Linux Java.

I have therefore archived the Compendium Institute website for posterity, since it contains lots of resources:

http://projects.buckinghamshum.net/compendiuminstitute/

SOFTWARE… You want the tool! I’m trying to keep links alive, and Java seems to be holding up remarkably since development ended around 2016.

If you’re a developer, then you can get the most recent CompendiumNG, from the CompendiumNG (next generation) wiki.
I can also offer this Mac installer version of CNG which requires no code knowledge to get running
If you’re non-technical on Win/Linux then this version of Compendium comes in an integrated installer:
- KMi Open University version 2.0beta with advanced experimental features (like Maps supporting video annotation): Downloads page | QuickStart Guide for Mac or Windows *follow the guide*
- CogNexus Institute also offers download links for a slightly older version

COMMUNITY… Since Yahoo closed down their groups end of last year, I’ve created a new Google Group which you’re warmly invited to join if you want to stay connected with fellow users and some of the original team. Collectively, we will hopefully be able to answer any queries about Compendium’s functionality and design rationale — and who knows, possible futures…

https://groups.google.com/g/compendiummapping

“The Future of Text” hits the streets

Congratulations to Frode Heglund, convenor for 9 years of the widely respected Future of Text Symposium, who has just edited and published The Future of Text book! The book is “a letter to the distant future”, in which an astonishingly diverse group of people share perspectives on where humanity has got to with the symbol systems we use to codify and share ideas — “Text” — and where we might be going. I was honoured to be invited to contribute to this: The Future of Text in Three Moves.

From the book’s introduction:

“I have invited some truly phenomenal people to contribute to this work, and against all odds, they said yes. There are eminent representatives from the worlds of art, typography, science, software development, academia and more. Text writes our history, and text guides our future, but text itself is not often reflected on, let alone written about.”

You can replay videos from the launch event, which was introduced by no less than Vint Cerf who has supported this work from its early days:

Well it’s been a hell of a year, so here’s a book to settle down with over Christmas and New Year, accompanied by your favourite brew, to dip into at random and make some space for reflection. Give it to all your friends and family! The pieces are short (some only a page), and a mind-stretching joy to browse.

I’m old enough to still love the weighty-tome gravitas of the printed volume in the above photo, but of course, most of us will read this digitally. It’s freely available as PDF | EPUB, and the PDF takes on greater interactive qualities when viewed in Frode’s Reader hypertext tool. Here’s his demo of this and the other tools he’s been inventing…

Alert to the challenge of digital preservation (“we’re writing in sand” as Frode memorably states about digital texts) — they’re working on a physical edition with the entire text micro-engraved into plastic and metal for longevity.

Frode kindly invited me to contribute a chapter, which gave me a much appreciated pause for thought on the 30 years I’ve spent using hypertext to create interactive visualisations of dialogue and arguments, and in the last decade or so, learning from my NLP and machine learning colleagues. So, here’s my letter to the future: The Future of Text in Three Moves.

Many congratulations also to Niko Grupen, whose piece From Author to Editor: Our Place Alongside the New Life Cycle of Text (p.148) scooped first prize in the Student Competition! His punishment is to read a signed Visualizing Argumentation e-book cover-to-cover over Christmas 😉

Doug Engelbart (RIP) is the fire driving both Frode and me (his influence on my career) — so for old time’s sake, here’s when Doug and Frode Heglund visited me in 2004 🙂

2 tools to help PhD students make their thinking visible

In this post, I share two ways to map your thinking, at different scales (rather like geographical maps)…

CQOCE Diagrams (or “Thesis Maps”)

One of the challenges that most/all PhD students have is to make their thinking visible — to themselves, to their supervisors, and to other researchers. There are so many potential ideas to weave into a narrative, and often different narrative pathways.

In our Learning Analytics PhD Program, we’ve been using the unpronounceable but very useful CQOCE diagram described by Luis Prieto in his Happy PhD blog. I encourage you to learn more about this:

“the diagram is commonly used in the introduction section of a dissertation, and it is meant to introduce, in graphical form, some of its main elements: the research Context, main research Question, Objectives, Contributions of your thesis and their Evaluation. However, many of us have also used it way before starting to write the dissertation book itself, as a “guiding star” when discussing with others and planning the thesis work.”

We have also been using it not just as a writing up device, but as a challenge right in the first year, to get doctoral researchers thinking about their thesis story. I tend to just call it the Thesis Map! As Luis comments, this goes through many revisions as the PhD takes its twists and turns. So in the end it serves two key purposes:

As a mirror for the supervision team to reflect on how we’re doing — “Oh, the contributions I thought I was making don’t align with the Research Questions…” “What kind of evaluation will be needed next year to back up this claimed Contribution 2?…”
As a navigational aid — a map — for the reader of the thesis, or indeed, for the audience if it’s used in a talk (“…in this talk I’ll be covering only this part of the map, but it shows you how this is a stage in a longer journey, in a wider landscape…”)

Here are two recent Learning Analytics theses that use this, from Vanessa Echeverria and Carlos Prieto.

Echeverria, V. (2020). Designing Feedback for Collocated Teams using Multimodal Learning Analytics

Prieto-Alvarez, C.G. (2020), Engaging Stakeholders in the Learning Analytics Design Process

Note that sections of the map can then be introduced in each chapter, to remind the reader where we are on the journey.

Argument Maps

The Thesis Map provides a macro-structure for the thesis argument: once you’ve bashed your map into shape, then your high level claim to have evidenced contributions to advance knowledge that addresses important RQs just drops out naturally. But there will be many micro-level arguments in the thesis that are invisible at this scale.

Zooming in, we’re experimenting with Argument Maps, that make visible more detailed moves. Here’s my briefing during a PhD session which introduces some basics…

Argumentation 101 for Learning Analytics PhDs! from Simon Buckingham Shum

2 weeks later, a couple of researchers shared their maps for feedback, and both commented on how it helps clarify thinking. Thanks to Ben Hicks and Gloria Fernandez-Nieto for jumping in!

Ben used the freely accessible ArgDown website which uses a classic Argument Map notation, enriched with optional colour-coding from #tags:

Gloria used the free Compendium tool that my team developed at KMi Open University, using the IBIS notation (QuickStart Guide to install):

Note: Al Selvin inventor and power user of Compendium, used it to create multimedia maps of his thesis thinking and qualitative data analysis [screen demos], incorporating many kinds of documents (which can be dragged and dropped onto maps).

Far more info on Knowledge Cartography is available if this interests you.

I hope these help you make your own thinking more visible — to yourself, your supervisors, and the world 🙂

The craft + tech of structuring participatory deliberation

As a Fellow of the RSA I’m happy to draw attention to the important new series of webinars just launched, on the critical role that effective, participatory deliberation has to play in resolving complex challenges, even apparently intractable dilemmas — at many different scales, from an organisation, to a local community, city, regional or even international scale.

As happens sometimes, I ended discovering a colleague at my own university doing fantastic work! Check out Nivek Thompson and her Deliberately Engaging portal. In prepping some notes for her, I thought I might as well blog them in case of wider interest to this community.

Hypermedia Discourse

A lot of my work has investigated a particular way in which software can help make thinking visible, the focus of all my work. Such tools seek to “augment human intellect” in Doug Engelbart‘s memorable words (my tribute to his inspiration for my work, and what he thought about this [Visualizing Argumentation]).

Here are some examples of how this works:

Make aspects of the conversational structure visible. Once a phenomenon is visible, rendered in a visual language that provides helpful ways to reflect on what is unfolding, it can be talked about, and is an “improvable object”. The Hypermedia Discourse project prototyped and evaluated the potential of combining models of dialogue and argumentation, with hypertext functionality for connecting issues, ideas, arguments and documents. Since we were interested in discourse about wicked problems, differences in perspective were the default starting point.
- The Delta Dialogues is an outstanding example of our Compendium tool in action (view Meetings), in the hands of skilled facilitators, my longstanding colleagues Jeff Conklin and Eugene Eric Kim.
- DebateGraph is a fine example of facilitated and un-supported deliberation, led by Peter Baldwin and David Price
- There are significant overlaps between educational technology focused on argumentation, and collective intelligence platforms for deliberation.
Support online forum moderators/facilitators assess the health of the conversation. A well designed user interface helps online participants to structure their contributions in ways that can provide the software with new ways to check the state of the debate (not possible with conventional flat chats, or threaded forums), and reflect this back to participants and/or moderators (see the Catalyst project for example).
Help to track ideas. Hypertext systems (more powerful than the Web) provide flexible ways to keep track of ideas (nodes), not just information. Anna De Liddo’s doctoral research is an example of how this can provide new forms of accountability in the participatory process ((in her work, for participatory urban design).

An important strand of our work was examining the facilitator skillset and disposition required to make good use of visualizations in real time, to augment the deliberation. Spearheaded by Al Selvin’s doctoral research, this led to a book that set out the concept of Knowledge Art.

Collaborative Evidence-based Problem-Solving

More recent work led by Tim van Gelder at Melbourne University (an Argument Mapping philosopher and software entrepreneur) has broken new ground in a particular niche of the design space: how do you convene a team of citizens to tackle a complex problem, with the challenge of devising an evidence-based, plausible analysis of the best way forward?

They have just published exciting results demonstrating that some teams of volunteers recruited via Facebook performed as well as, and in some cases better than, teams of professional intelligence analysts. See the paper to appear in the Journal of Cognitive Engineering and Decision Making on the Hunt Lab website, and this CIC webinar.

The emergence of NLP to detect critical, reflective writing

Natural Language Processing (NLP) has in recent years emerged from the AI labs into the mainstream. This has been a recent focus of my work, in the context of giving students instant feedback on their drafts. This has yet to be deployed in the context of participatory deliberation, but here are some preliminary reflections on where the automated detection of shallow and deeper reflection might assist participants posting online to reflect on how they are reacting to challenges — from other people, or the turbulent life events that are threatening dearly held assumptions, and ways of life.

Might the growing potential of NLP to make sense of rich, narrative prose offer the optimal combination in years to come — playing to the respective strengths of machines and humans to make sense of the world?

Power tools (and new literacy?) for deliberation professionals?

I remain excited about the potential of interactive, usable visualizations to help tackle the limitations of individual and collective human cognition. I have also seen first hand how hard it is for people to learn to structure their thinking more carefully than firing off their thoughts in the usual way. The role of the deliberation facilitator can be absolutely critical to modelling and scaffolding stakeholders into more reflective modes of reflective dialogue and rigorous argumentation.

That provides the basis for a good conversation with the growing international networks of deliberation experts who will also be working increasingly in online or hybrid modes.

BCII Visualizing Complexity

Resources from today’s session with students on the UTS Bachelor of Creative Intelligence & Innovation, who are on a 2-week intensive Creativity & Complexity summer school.

BCII 2016 – Visualizing Complexity from Simon Buckingham Shum [PDF]

Readings and multimedia to go deeper…

Human-Centred Informatics (HCI) & Complexity

Making Thinking Visible in Complex Times: This is a video replay of how the 1960s pioneering work of Doug Engelbart foresaw our current need to “augment human intellect” to tackle the “urgent complex problems” facing society, and how this traces through to mapping dialogue and debate.

Collective Intelligence for OER Sustainability: This article considers the implications for Collective Intelligence tools, of some of complex systems concepts introduced in the summer school.

Knowledge Cartography Software Tools and Mapping Techniques [Lib][Pub]

Constructing Knowledge Art: An Experiential Perspective on Crafting Participatory Representations [Lib][Pub][Fbk].

How work and organisations are changing how knowledge flows

Hagel III, J., Seely Brown, J. & Davison, L. (2010). The Power of Pull: How Small Moves, Smartly Made, Can Set Big Things in Motion. Basic Books

Al Selvin – Memories & Tributes

Words can only say a bit of what Al’s loss means, but a version of what I said at his memorial service is on the website.

POSTED TODAY TO THE COMPENDIUM INSTITUTE COMMUNITY FORUM.

Dear Compendium Community and other colleagues,

Al Selvin, one of Compendium’s inventors and leaders, passed away suddenly due to heart failure on 10th Oct while out cycling, enjoying one of the activities he loved most. He was 56. This is a terrible shock for all who knew him, and our thoughts are very much with his wife, children and wider family. Moving tributes to him at his memorial events reflected the deep love and huge esteem in which he was held as a humble, loving, deeply reflective man.

For those of us in the Compendium community, Al was the driving force to refine Compendium as a digital medium. It was the experience of the mapper that particularly fascinated him, since nobody seemed to pay any attention to this particular mix of skills and dispositions. Why did so many people say that it looked like ‘magic’ when he produced a Dialogue/ Issue/ Conversational Map during a meeting? What was a mapper doing when they were combining this with more formal modelling, all seamlessly integrated with navigating and linking the myriad other documents in that team’s work? This passion produced many research articles, much thinking aloud in his blog, and a PhD, recently distilled as a book. See the links at the top of the page for more.

Without Al, this community would not exist.

Please use this page as a place to share your memories: http://bit.ly/alselvin

We welcome all reflections on Al’s contribution — his writing, talks, videos, or first hand experience working with him. Feel free to include links and photos.

To contribute, please email us with your contribution and we’ll add it in.

We thought we’d let Al start the contributions in his own words, acknowledging how many of you supported him in his PhD.

With our thanks in advance to you, for helping his family and friends, as well as ourselves, to understand and celebrate how many people Al’s life touched.

Feel free to forward this to relevant colleagues.

Maarten Sierhuis & Simon Buckingham Shum

msierhuis and sbuckinghamshum at gmail dot com

Complexity Science, Learning Analytics & Collective Intelligence

The European FET Flagships competition has the following mission:

FET Flagships are ambitious large-scale, science-driven, research initiatives that aim to achieve a visionary goal. The scientific advance should provide a strong and broad basis for future technological innovation and economic exploitation in a variety of areas, as well as novel benefits for society.

As part of the FuturICT submission from a year-long pilot, we recently submitted our proposal for the first 30months of a 10 year research programme (PDF).

FuturlCT is a FET Flagship project using collective, participatory research, integrated across the fields of ICT, the social sciences and complexity science, to design socio-‐inspired technology and develop a science of global, socially interactive systems. The project will bring together, on a global level, Big Data, new modelling techniques and new forms of interaction, leading to a new understanding of society and its co-‐ evolution with technology. It will place Europe at the forefront of a major scientific drive to understand, explore and manage our complex, connected world in a more sustainable and resilient manner.

Working with FuturICT is the closest that someone in my field may get to “Big Science” a la Human Genome project or Large Hadron Collider. We don’t tend to have projects of that scale in human-centred computing! But the over-arching theme of Complexity Science as a way of making sense of societal big data provides that scale of vision. The European Physical Journal does not spring to mind as the first place to look for work on ICT for learning analytics or collective intelligence – my particular interests – but it has an explicit focus on advances in Complex Systems, including socio-technical-economic systems, not just physical or biological. So it’s very satisfying to point to a more detailed account of the thinking behind the proposal, which we’ve just published as an open access special issue of EPJST.

Within the special issue, you’ll find a fascinating set of contributions from European scientists who set out a 10 year research agenda within their fields: what are the really tough problems? There are also visionary position papers outlining the kind of socio-technical infrastructure that FuturICT will investigate, and a foregrounding of the ethical dimensions that human Big Data and Analytics always raise.

For those interested in Human-Centred Computing, CSCW, Collective Intelligence and Learning, these run implicitly through many of the articles, but I contributed on these themes to two of them:

Buckingham Shum, S., Aberer, K., Schmidt, A., Bishop, S., Lukowicz, P., Anderson, S., Charalabidis, Y., Domingue, D., de Freitas, S., Dunwell, I., Edmonds, B., Grey, F., Haklay, M., Jelasity, M., Karpištšenko, A., Kohlhammer, J., Lewis, J., Pitt, J., Sumner, R. and Helbing, D. (2012). Towards a Global Participatory Platform: Democratising Open Data, Complexity Science and Collective Intelligence. Eur. Phys. J. Special Topics, 214, pp.109-152. http://dx.doi.org/10.1140/epjst/e2012-01690-3

The FuturICT project seeks to use the power of big data, analytic models grounded in complexity science, and the collective intelligence they yield for societal benefit. Accordingly, this paper argues that these new tools should not remain the preserve of restricted government, scientific or corporate élites, but be opened up for societal engagement and critique. To democratise such assets as a public good, requires a sustainable ecosystem enabling different kinds of stakeholder in society, including but not limited to, citizens and advocacy groups, school and university students, policy analysts, scientists, software developers, journalists and politicians. Our working name for envisioning a sociotechnical infrastructure capable of engaging such a wide constituency is the Global Participatory Platform (GPP). We consider what it means to develop a GPP at the different levels of data, models and deliberation, motivating a framework for different stakeholders to find their ecological niches at different levels within the system, serving the functions of (i) sensing the environment in order to pool data, (ii) mining the resulting data for patterns in order to model the past/present/future, and (iii) sharing and contesting possible interpretations of what those models might mean, and in a policy context, possible decisions. A research objective is also to apply the concepts and tools of complexity science and social science to the project’s own work. We therefore conceive the global participatory platform as a resilient, epistemic ecosystem, whose design will make it capable of self-organization and adaptation to a dynamic environment, and whose structure and contributions are themselves networks of stakeholders, challenges, issues, ideas and arguments whose structure and dynamics can be modelled and analysed.

Johnson, J., Buckingham Shum, S., Bishop, S., Zamenopoulos, T., Swithenby, S., MacKay, R., Merali, Y., Lorincz, A., Costea, C., Bourgine, P., Louçã, J., Kapenieks, A., Kelley, P., Caird, S., Bromley, J., Deakin Crick, R., Goldspink, C., Collet, P., Carbone, A. and Helbing, D. (2012). The FuturICT Education Accelerator. Eur. Phys. J. Special Topics, 214, pp.215-243. http://dx.doi.org/10.1140/epjst/e2012-01693-0

Education is a major force for economic and social wellbeing. Despite high aspirations, education at all levels can be expensive and ineffective. Three Grand Challenges are identified: (1) enable people to learn orders of magnitude more effectively, (2) enable people to learn at orders of magnitude less cost, and (3) demonstrate success by exemplary interdisciplinary education in complex systems science. A ten year ‘man-on-the-moon’ project is proposed in which FuturICT’s unique combination of Complexity, Social and Computing Sciences could provide an urgently needed transdisciplinary language for making sense of educational systems. In close dialogue with educational theory and practice, and grounded in the emerging data science and learning analytics paradigms, this will translate into practical tools (both analytical and computational) for researchers, practitioners and leaders; generative principles for resilient educational ecosystems; and innovation for radically scalable, yet personalised, learner engagement and assessment. The proposed Education Accelerator will serve as a ‘wind tunnel’ for testing these ideas in the context of real educational programmes, with an international virtual campus delivering complex systems education exploiting the new understanding of complex, social, computationally enhanced organisational structure developed within FuturICT.

In the above paper, my collaboration around LearningEmergence.net with Ruth Deakin Crick (Univ. Bristol) and Chris Goldspink (Incept Labs) opens up for us an intriguing landscape around complexity thinking and learning:

“As in many other fields, there is now active interest in the possibility that the concepts and tools of complexity science hold the promse of providing a new, more rigorous language and suites of computational tools for systemic thinking within educational research. These could enable possible futures to be mapped, modelled, simulated, and rendered in appropriate forms to help both researchers and practitioners to understand and, where appropriate, choose to act differently to achieve their intended outcomes. A central claim to be investigated in this research programme is that complexity science provides a language for transdisciplinary learning-centred discourse between system stakeholders, serving as reference points for modelling and, suitably communicated and embodied in tools, for educational leaders, and learners.

For instance, autopoiesis is relevant to the emergence of learner identity in co-constructed domains of meaning, and hence for the way we approach learning as well as school change. Dissonance, defined as conflict between agents and processes, creates a space for deep learning when agents have the capacity to hold conflicting ideas in tension. Emergence focuses attention on the quality of relationships for creative learning and leadership in complex organisations. Resilience has been identified as key to learning to learn, and has been operationalised as a formally modellable quality in individual learners, not just socio-technical collectives [9].

To take another example, the evidence is that efforts to manage educational systems (whether at national or institutional level) which do not take into account complex systems dynamics, do not result in sustained school improvement: standards in schools across the developed world are plateauing, as measured by student outcomes [23]. There is a pressing need for management and self evaluation processes [24] which can account for such complexity in order to facilitate, value and enhance the breadth and range of student outcomes. The evidence emerging from these new approaches is that systemic transformation is indeed possible (e.g. [25–28]).

A research community is now emerging at the intersection of Complexity Science, Educational Theory and Practice [24, 28, 30–33]. Through our visiting scholars programme, and international workshop and webinar series, we anticipate a very productive dialogue with these networks. FuturICT will make available unprecedented computational infrastructure for tracking and modelling complex systems — the question is how does this contribute to current theoretical discourse, and how can intensely practical challenges around the design and management of resilient learning ecosystems be tackled in fresh ways when traditional theory is combined with simulation and visualisation tools that can render complex systems in new ways, for both researchers and practitioners?”

Working with the FuturICT consortium has been a fascinating experience so far, and if we are successful, the start of what should be a remarkable intellectual journey. For great introductions to the field, we’re working with Philip Ball, an award-winning science journalist. He’s written a brilliant historical contextualisation of the notion of a “physics of society”, controversial as this is — check out Critical Mass – and is helping us explain Why Society is a Complex Matter.

OER mapping and knowledge cartography

There’s been a flurry of interest in mapping within the Open Educational Resources community, catalysed by Susan D’Antoni’s facilitated 3-week online discussion to crowdsource requirements for a participatory “OER World Map”. This has sparked many dreams and visions about what could or should be usefully ‘mapped’.

This discussion triggered memories from back in 2008 when I was developing an account of how “cartography” as both a metaphor, art and discipline could be ‘mapped’ (sorry) to the charting of emergent intellectual landscapes (two examples appended).

In 2010-12, Hewlett Foundation funded the OLnet project to research the evidence base around the impact of OER, and with our colleagues in IET, we developed the Open Education Evidence Hub. This generates a filterable, interactive Google Map as one view (of many) onto the OER movement, complemented by conceptual maps of the evidence base (challenges, issues, claims, solutions, resources), plus social networks of the people and organizations:

In the follow-on work which we’ve driven forward in KMI, we’ve subsequently generified this into Evidence-Hub.net — a customizable shell for any community of enquiry seeking to pool their collective intelligence, bridging across academic and practitioner perspectives:

So, some broader perspectives on knowledge cartography…

Knowledge Mapping for Sensemaking

Ontolog webinar, 17 April 2008

Introduction: A “knowledge mapping” approach to managing information, knowledge and decisions places the emphasis on the creation of “cartographic” layers, which like spatial maps, weave different kinds of stories over the “raw data” of documents, deadlines, resources, problems. Just as the map is not the territory, neither is a knowledge map neutral, nor necessarily a consensus worldview, and not to be taken as truth. Continuing the spatial planning metaphor, a good knowledge map is a powerful representation for sensemaking: orientation, shared memory, filtering complexity, and maintaining shared attention for planning and decision making. In software, digital maps of such intellectual landscapes exploit the power of hypermedia, folksonomy, social tools, and (where possible) reasoning over formal ontology. The centrality of social processes in negotiating the meaning of a map is unquestioned. The speakers in today’s session work from these assumptions, and will take you deeper into their particular approaches.

Knowledge Cartography

Preface to Knowledge Cartography: Software Tools and Mapping Techniques. (Eds.) Okada, A., Buckingham Shum, S. and Sherborne, T. Springer: Advanced Information and Knowledge Processing Series. ISBN: 978-1-84800-148-0

The eyes are not responsible when the mind does the seeing.
Publilius Syrus (85-43 BC)

Maps are one of the oldest forms of human communication. Map-making, like painting, pre-dates both number systems and written language. Primitive peoples made maps to orientate themselves in both the living environment and the spiritual worlds. Mapping enabled them to transcend the limitations of private, individual representations of terrain in order to augment group planning, reasoning and memory. Shared, visual representations opened new possibilities for focusing collective attention, re-living the past, envisaging new scenarios, coordinating actions and making decisions.

Maps mediate the inner mental world and outer physical world. They help us make sense of the universe at different scales, from galaxies to DNA, and connect the abstract with the concrete by overlaying meanings onto that world, from astrological deities to signatures for diseases. They help us remember what is important, and explore possible configurations of the unknown. Cartography — the discipline and art of making maps — has of course evolved radically. From stone, wood and animal skins, we now wield software tools that control maps as views generated from live data feeds, with flexible layering and annotation. (Our sister volume in this series, The Geospatial Web, explores the convergence of spatial data, mapping tools and the social web: Scharl and Tochtermann, 2006)

“Foundational concept, fragmented thinking, line of argument, blue skies research, peripheral work”: we spatialise the world of ideas all the time with such expressions. Maps can be used to make such configurations tangible, whether sketched on a napkin or modelled in software. In this book we bring together many of the leading researchers and practitioners who are creating and evaluating such software for mapping intellectual worlds. We see these as new tools for reading and writing in an age of information overload, when we need to extract and construct meaningful configurations, around which we can tell different kinds of narrative.

For a visual generation of children who have never known a world without ubiquitous information networks, we might hypothesise that knowledge maps could have particular attraction as portals into the world of ideas. Moreover, the network is not only dominant when we think about our social and technical infrastructures, but almost an ontological stance in postmodernity, where we hold our viewpoints to be precisely that: always partial and contextualised. Weaving connections between nodes in the network is the most flexible way to bring ideas and information into locally coherent relationships with each other, knowing that there is always another viewpoint on the validity of these patterns. Modelled in software, the vision is that intellectual continents, islands and borders can be invoked and dissolved at different scales, as required.

Knowledge Cartography can be defined as:

the art, craft, science, design and engineering of different genres of map to describe intellectual landscapes — answering the question how can we create knowledge maps?

and the study of cartographic practices in both beginners and experts as they make and use such maps — answering the question how effective are knowledge maps for different kinds of user?

The particular focus of the authors in this volume is on sensemaking: the process by which externalising one’s understanding clarifies one’s own grasp of the situation, as well as communicates it to others — literally, the making of sense (Weick, 1995: p.4). While “sense” can be expressed in many ways (non-verbally in gesture, facial expression and dance, and in prose, speech, statistics, film…), knowledge cartography as construed here places particular emphasis on digital representations of connected ideas, specifically designed to:

I. Clarify the intellectual moves and commitments at different levels.
(e.g. Which concepts are seen as more abstract? What relationships are legitimate? What are the key issues? What evidence is being appealed to?)

II. Incorporate further contributions from others, whether in agreement or not.
The map is not closed, but rather, has affordances designed to make it easy for others to extend and restructure it.

III. Provoke, mediate, capture and improve constructive discourse.
This is central to sensemaking in unfamiliar or contested domains, in which the primary challenge is to construct plausible narratives about how the world was, is, or might be, often in the absence of complete, unambiguous data.

Our intention with this book is to provide a report on the state of the art from leaders in their respective fields, identify the important challenges as they are currently seen in this relatively young field, and inspire readers to test and extend the techniques described — hopefully, to think more critically and creatively. Many of the tools described are not sitting in research labs, but are finding application in diverse walks of life, with active communities of practice. These communities represent the readership we hope for: learners, educators, and researchers in all fields, policy analysts, scenario planners, knowledge managers and team facilitators. We hope that practitioners will find new perspectives and tools to expand their repertoire, while researchers will find rich enough conceptual grounding for further scholarship.

Genres of knowledge map

A range of mapping techniques and support tools has evolved, shaped by the problems being tackled, the skill of mappers, and the sophistication of software available. We briefly characterise below the main genres of map. The appendix summarises at a glance which mapping approaches and software tools are presented in each chapter.

[see rest of preface for examples…]