Al Selvin and I are working on a book to distill the key messages from his long term research programme, culminating in his PhD 18 mnths ago. So, we’re thinking aloud as we try out different ideas… reactions welcomed!
Possible titles…
Making Representations Matter: Catalyzing Collective Intelligence with Visualizations
Collective Intelligence Cartography: Emergence of a New Discipline
The word depict, from the Latin depingere, means to picture; a depiction means a representation in a picture, sculpture, or words. What this book is about is understanding how people help each other understand things by creating a picture of some kind together. By engaging people in making some kind of depiction, we can help each other explore an issue, or comprehend a situation or idea better. While much of this book will describe sophisticated kinds of depictions, like software-assisted knowledge maps, the act of engaging each other in depictions for the purpose of understanding is something that can happen at the most basic of levels.
Young Tom, six years old, had been given a small, round black compass by his parents. He was excited when he got it and ran off from the lunch table to experiment. Not long after he came back into the kitchen, on the verge of tears. “I don’t understand what it means,” he mourned. “The little needle moves all over the place. How does it help me find things?” His mother, Jackie, said, “It’s not that hard, Tom. I’ll show you.” She took some brass badges out of a drawer, and put them on the kitchen table. Tom sat him down in the chair next to her. “Now let’s pretend this one is Sally’s house,” she said, pointing at one of the badges, “and this one is the school. Where would the park be?” He moved one of the badges near one the “school”. “That’s right!” she said. “Now put the compass in the middle of them where our house would be.” Jackie took Tom’s hand, the one clutching the little compass, and moved it to the table between the objects. He put it down. Jackie pointed at the “school” and said, “Which letter on the compass is closest to the school?” “E”, Tom said. “That’s right, E is east, so that means that the school is east of our house. What about the park?” “S”, Tom said. “So which direction is the park from us?” “South,” Tom said, beginning to smile.
She moved one of the badges to the lower left, and asked “Now which direction is that?” “South… west,” said Tom, smiling. “Southwest.” “Southwest, that’s right,” his mum said. “What if it’s… west east?” he said.” “You can’t get west east,” she replied, pointing at the compass so he could see that the two directions were on opposite sides. He laughed. “Where’s Luke’s house?” asked his father. “He’s on Peel Road,” Tom answered. “Can you point? Where do you think Luke’s house is, in relation to our house?” asked his father. “Where’s L “It would be…” he started. She handed him a pen lid, saying “here you go, show us where Luke’s house is.” She pointed at one of the badges. “This is where our house is, show us where Luke’s house is.” ““Mummy can I just have a piece of paper?” Tom asked. “No, do it with the pen lid. Where’s Luke’s house?” He smiled and first pushed the lid farther away on the table. “No, it’s not over there”, she said, while Tom laughed. “Which way do we walk to get to Luke’s house?” He then put it more carefully to the left of the compass. “That’s right, so which way are we going to get to Luke’s house?” “North,” Tom said, smiling confidently. She pointed at her plate from lunch, farther to the left on the table. “Now Mummy’s plate is school, which way do we walk to school?” “North,” he said. “There you go!” she declared.
This story illustrates much of what this book will explore.
There was an important issue – Tom’s sadness and frustration at not understanding how the compass worked. There was someone who wanted to help Tom deal with the issue, learn something, and have fun doing it. There was the instant, unplanned inspiration – an improvisation – that Jackie came up with, to use the little shiny objects and make it into a game. She decided how the objects would play into the representation, making aesthetic choices with simple but clear tactile and visual rules, deciding (on the fly) on a compelling and engaging form that Tom could understand, enter into and be part of. She got him to point and him to move the objects around, or in the language we will use in this book, she got him to engage in the representation and in its evolution. She improvised a “script” for the game that contained a rising set of complexity and demands on Tom’s skills and evolving knowledge, or in the terms we will use in this book, she crafted a narrative with actions that had sequences and meanings. She made decisions – what we will call ethical choices – about what Tom should pay attention to, and what he could do and not do with the representation (for example, she decided that he should stick with the depiction they had created together when it came to saying where Luke’s house was, and not just draw it on paper as he asked. Jackie was sweet to Tom throughout, but also kept him firmly “on task” long enough for Tom to grasp the principles of the compass and understand how it could tell him directions between things he cared about. In doing this, Jackie helped Tom make sense of something that had been frustrating him, turning it into something he could enjoy and take forward into life.
This is what the book covers. We will discuss how we engage others in creating representations to help understand issues, but more to the point what that activity is actually composed of. As we’ll see in Chapter 2, most treatments of participatory visualizations talk about methods and outcomes, but few treat the practice of engaging people in such visualizations as a subject worthy of attention in its own right. This book focuses on ways to look at and understand the practice of making participatory representations – the skills and considerations involved, the choices that practitioners make, and the ways in which participants can engage and make use of them.
We start with parents and kids precisely because of the everyday nature of those interactions. Helping our children understand things is not a matter of stuffing facts and ideas into their heads from outside. We know that for them to learn and grow, it has to mostly come from inside of them. Kids need to engage with situations, try them out, learn them from the inside out. They do this for themselves with their imaginative play, setting things up and manipulating them, creating whole worlds from whatever they have around them.
When our kids were small, their number one toy, in terms of getting actually used and played with over and over again, was just ordinary cardboard and wooden blocks. They could make them be anything they wanted – a house, a fort, a landscape. They were making representations and engaging in them with all of the energy and imagination they could muster.
When we as parents want our kids to learn something, especially something they are struggling with, we meet them where they are. As Jackie did with the compass, we give them tangible ways to understand their world and pick up new abilities. For example, parents best help kids with their writing assignments not by giving them general principles or the same kinds of drills they get in school. Instead they can help them look at specific sentences, words, and paragraphs in the context of what their assignment is about, and try to help them see what works or doesn’t work in that arena. The real help they provide is very close-grained, very close to the materials themselves. The tactile and visual contact with specific words and sentences — moving them around to see what works best, like Tom’s little metal items on the kitchen table — make the difference. It’s not giving kids the answers, instead it’s helping them to see how to come up with the answers themselves.
The parent’s role is to help that process along by the simultaneous close engagement with the materials and medium itself (what they are writing about and how they are doing it), with the interactions with the kids as people, with an attitude of love and hope and respect for their own intelligence — an attitude that tells the kids they will get it, they just haven’t gotten there yet. There is something about that close work with words and sentences and meanings, with styles and effects, with thinking through the consequences of different choices, providing examples but only to help them to see why it does or doesn’t work in the context, that helps kids move toward being able to do it on their own. It’s not much different from other things parents teach their kids, like how to ride bicycles. That skill, too, has to come from inside the child. A parent can’t ride the bikes for them or tell them in the abstract how to do it. Instead, the parent has to help kids to learn the little tricks of balance and navigation, and the confidence that they don’t need a puffing parent running alongside holding the bike up straight. Ultimately what matters is when could get around the block under their own power, wearing that look of pride and freedom on their faces when they had done it.
This fostering of capability in the ability to engage others in making helpful representations is a huge motivation to us. We want to be of help to people trying to achieve and express what they are trying to do with the new media for mapping the connections between ideas. If the considerations in this book can help people see for themselves why something does or doesn’t work, and come up with their own inspirations, it will have been worth it.
That is what the considerations in the remaining chapters speak to.
A second example bridges between the more commonplace examples above and the main content of this book, which mainly explores the kinds of considerations that obtain when engaging people in software-based representational tools. Though as the above discussion tried to state, really the core ideas are common to the most everyday attempts of people to help others by engaging them in making a depiction together.
A committee in a medium-sized public school district (approximately 20,000 students) in the Hudson River Valley region of New York State was tasked with analyzing the alternatives for school building capacity in the district, which has experienced declining enrollment. This highly contentious issue had come up many times before. The district’s superintendent of schools was concerned that the discussion would be unproductive, due to tensions and unsurfaced assumptions between the various interest groups (school administrators, teachers, parents, taxpayers, etc.). Every school building has an active, vocal contingent of parents and teachers who have strong interests in keeping their own local school open. Equally strong and vocal are the many local taxpayers who feel that school taxes are already too high. To address this, the superintendent asked two outside facilitators with expertise in conducting participatory knowledge mapping sessions to help run the meetings.
The two facilitators convened a series of meetings in a library of one of the schools. A committee of twenty parents, teachers, community activists, and administrators met once a week to work through the alternatives. For each meeting, the facilitators prepared an agenda with a hypermedia issue mapping tool, which was projected in front of the group on large screens. The agenda focused on various alternatives, policy matters, process considerations, and other issues.
The facilitators employed a variety of approaches. First, they facilitated a general discussion of the issues involved, using a conventional Issue-Based Information System (IBIS) approach (representing discourse as issues, positions, pros, and cons) to capture and display the discussion as it proceeded. This involved rapid synthesis of what the meeting attendees were saying, thus creating nodes and links in the hypermedia tool that showed the relationship of statements to each other. They also validated the way they captured the statements by frequently asking the participants to look at the maps, asking “Does this capture what you said accurately?” Sometimes participants looked closely and provided detailed feedback (e.g, “Well, not really. What I was really trying to say was this…”). At other times, the heat of the discussion was such that it was difficult for the practitioners to intervene without running the risk of derailing the meeting’s momentum. The practitioners had to make moment-to-moment decisions on how much to intervene, and in what ways.
Between meetings, the practitioners analyzed the maps from the general discussion. They looked for recurring themes and questions and, from these, created a template covering the major considerations that would guide choices between the alternatives (see Figure 2). They then facilitated several sessions using the template to structure conversation about each of the alternatives in turn. By the fourth session, the facilitators were able to induce the participants to conduct an analysis according to the template, while still capturing as much of the side discussion and issues as possible. Also between sessions, the district office distributed via mail all of the map output in text form to all the participants.
At the end of the process, the practitioners held a plenary session for the broader community to understand the final decision. The maps of rationale and templated analysis made the pros and cons for each alternative, as well as many of the comments and points of view, clear and explicit. Even though there was little consensus that the chosen alternative was the best one, the community members completing a post-presentation questionnaire agreed that the process had been conducted in a fair manner, and that the discourse and competing points of view had been made more explicit and comprehensible than in previous years.
In this example, a very different set of considerations and techniques were employed. However, we aim to show that the framework and language we are developing — to describe how one adds value to a conversation with real time representational practice — is as applicable to this as it is to our opening kitchen scene:
As we navigate the second decade of the 21st century, humanity confronts intensely complex challenges, at many scales from personal, to community, to national and global. Finance, health, energy, education, urbanization, terrorism: the dilemmas we face stretch to the very limit our cognitive and interpersonal capacities — to make sense of overwhelming amounts of data; to assess conflicting judgments about its trustworthiness; to resolve polarized interpretations about the implications; and to negotiate effective courses of action to which all parties can commit.
Better data and information/communication technologies (ICT) are not only drivers of these challenges, but have key roles to play in managing them. However:
more data + more processing technology ≠ more insight or wisdom
The nature of the problems we face means we will depend on nuanced human judgment over complex tradeoffs long into the foreseeable future, because despite their scientific and engineering foundations, these are not purely rational problems to be solved from first principles, using tried and tested algorithms or methods. The nature of policy formulation and implementation is fraught with the messiness of human life — social, organizational, political and cultural dimensions cannot be simply wished away. Technically sound engineering blueprints are not sufficient when designing solutions to wicked problems, whose very definition is contested. Somehow, humanity’s collective intelligence must chart courses through these white waters.
None of this is news of course, and many people are working on it. Under the broad heading of collective intelligence (including other strands of our own work) we see community activists, academics, businesses and governments seeking to improve the social fabric of the internet in order to better harness the collective intelligence of human networks, operating largely asynchronously across timezones, augmented by machine intelligence (e.g. www.ci2012.org). Another strand of work uses sensor networks and computational simulation of social systems (e.g. www.futurict.eu) to help us understand, increasingly in real time, the complex adaptive systems permeating society, but whose structure and dynamics we are only beginning to grasp — often with tragic consequences when we consider our lack of control over financial markets or medical epidemics.
Collective Intelligence Cartography (CIC) occupies a different niche. All of the above work will generate symbolic representations of many sorts, around which many meetings will be held to figure out what they mean, and it is here that CIC makes a contribution. Our focus is on a pervasive, critical, and hence surprisingly neglected piece of the collective intelligence jigsaw puzzle, namely, how do we improve our capacity to use such representations when we meet?
This focus on the dynamics of collective intelligence in small scale, synchronous contexts complements the social-semantic web/collective intelligence focus on large scale, asynchronous interaction.
CIC also complements the big data and simulation efforts, shifting attention from machine-mining patterns in relatively low-level data to the highest levels of meaning-making that humans are capable of:
- wrestling with ethical dilemmas
- resolving conflicts between diverse stakeholders
- crafting metaphorical power through stories
- empathizing by placing oneself in another’s shoes
This is the next step from machine-generated information visualizations to their manipulation and annotation as people discuss them, and to the manual crafting of representations (with or without computational aids) that reflect a team’s evolving understanding of a problem or the creative process as a design solution emerges.
This focus takes us back to humanity’s most established mode of information diffusion and collective sensemaking: talking face to face, and inscribing marks in dirt, leaves and clay. In an earlier book, we introduced the concept of maps as follows:
Maps are one of the oldest forms of human communication. Map-making, like painting, pre-dates both number systems and written language. Primitive peoples made maps to orientate themselves in both the living environment and the spiritual worlds. Mapping enabled them to transcend the limitations of private, individual representations of terrain in order to augment group planning, reasoning and memory. Shared, visual representations opened new possibilities for focusing collective attention, re-living the past, envisaging new scenarios, coordinating actions and making decisions.
Maps mediate the inner mental world and outer physical world. They help us make sense of the universe at different scales, from galaxies to DNA, and connect the abstract with the concrete by overlaying meanings onto that world, from astrological deities to signatures for diseases. They help us remember what is important, and explore possible configurations of the unknown. (Preface to Knowledge Cartography, 2008)
Thus, we are interested in how teams build new meaning using ‘representational artifacts’ which take into account all the ‘hard’ technical criteria as well as the ‘softer’ human criteria mentioned above. In millions of meetings today, people will be scribbling and sketching on paper and whiteboards; typing lists of bullet points and outlines on screen; displaying, annotating and sharing photos, diagrams, maps and websites. However, CIC is not about collaboration and visualization software per se (although much of our work has been with hypermedia visualization tools), but about the human skillset needed to wield these and non-digital artifacts effectively, in real time, to build new meaning in a group. We’ve all been in meetings where a shared display added little value — but we’ve all been in meetings when the right representation at the right moment helped to harness the collective energy and wisdom in the room, people connect with each other and the problem at a deeper level, and things move forward tangibly. What does someone do with a representation to catalyse such moments?
We need to attend to the people who know how to do this, and learn from them (and instructively, from those who are less fluent). We argue that what we call Participatory Representational Practice is a poorly understood but vital new civic skill for all types of collaborative knowledge work, where the complexity of the problem requires, or benefits from, the construction of a collectively owned representational artifact — whether scribbled on a napkin or detailed in a multiuser computational modeling tool. Hence, this includes meeting facilitators, but extends to many others, such as students, teachers, designers, academics and policy analysts.
To substantiate this claim, we have been documenting the vacuum in our current understanding, both in academic research and in practitioners’ state-of-the-art: nobody seems to have articulated the experience and skillset of this new breed of ‘cartographers’.
Schön (1983) insisted that there is an artistry to professional practice that, although difficult to describe, nonetheless informs and shapes what practitioners actually do:
Let us search … for an epistemology of practice implicit in the artistic, intuitive processes which some practitioners do bring to situations of uncertainty, instability, uniqueness, and value conflict. (1983: 49)
This book intends to contribute to that search. We describe the process by which we have developed a language to describe the distinctive experience and skillset that marks these people out (Chapter X). To give you a feel for what’s to come, in developing a language capable of expressing what we found when studying such practitioners, we found it necessary to draw from Aesthetics (how practitioners shape and craft the representation), Ethics (how a practitioner’s actions affect other people), Narrative (meaning and causality applied to the flow of events), Sensemaking (the ways in which practitioners deal with situations of doubt or instability) and Improvisation (the spontaneous, creative moves that practitioners can make).
This language has emerged in dialogue with designing ways of seeing this in action by analyzing and visualizing video data at different scales, both quantitatively and qualitatively (Chapter X). This lays the conceptual and empirical foundations for future practitioner training and software development (Chapter X).
To conclude, in an age of unprecedented complexity, we urgently need people who are fluent at “augmenting human intellect” with new kinds of “concept structures”, to borrow Doug Engelbart’s iconic 1963 language. Our schools, universities and workplaces train us to read and write as solo authors and speakers, paying little or no attention to the pervasive role of representations in our thinking with others. Understanding the experience and skillset that people fluent in such practice is a key first step. Perhaps this may come to be seen as a 21st century literacy — reading and writing symbolically in ways designed specifically to build real-time collective intelligence.
We hope this whets your appetite. If so, then let us take you on a journey…
Al Selvin and Simon Buckingham Shum
![Al Selvin, KMi Webinar [click to replay in new tab]](https://simon.buckinghamshum.net/wp-content/uploads/2011/12/Screen-Shot-2020-04-15-at-8.49.09-am.png)
Al Selvin and I are in Florence, participating in the 
I’m delighted to announce that Joanna Kwiat has successfully defended her PhD, entitled “Multi-Perspective Annotation of Digital Stories for Professional Knowledge Sharing”