Collaboration with Ágnes Sándor, Xerox

One of the joys of being a researcher is having the chance to work intensively on a common vision, with someone delightful who brings a sharp mind and complementary approach. In that vein, today sees the end of a hugely productive 6 week visiting OLnet Project fellowship with Ágnes Sándor, from Xerox. We are both concerned with the development of an infrastructure capable of processing the knowledge level claims that scholars make in their writing. Her work brings a natural language processing dimension to the annotation of significant rhetorical moves in texts, complementing our approach of providing socio-semantic tools for human annotation.

By way of context, when we started our work in KMi on this in 1998, which became the EPSRC Scholarly Ontologies Project (2001-04), there was no computational parsing that we knew of capable of detecting scholarly rhetorical moves. A decade on, there is a growing network of researchers focused on this challegne (see blog post on Hyp-ER), the machines have got smarter, and here we are integrating machine and human annotation within our Cohere sensemaking support system.

As we concluded in yesterday’s seminar, there remain many challenges on the technical, user experience, and theoretical fronts, but we’re passing a very satisfying milestone 🙂 My thanks to Ágnes, and of course KMi colleagues Anna and Michelle, and the (human!) OLnet researchers working on the project reports, for getting us this far. Particular thanks to Giota and Elpida for their extra time to be video interviewed as they compared their analyses with XIP’s. They can rest assured that there’s no doubt that we still need people to make sense of the world!… but when a machine takes only milliseconds to highlight interesting passages in a large report, we’re opening up exciting new vistas in sensemaking…

Integrating Human & Machine Document Annotation for Sensemaking

This event took place on 11th November 2010 at 2:30pm
Knowledge Media Institute, The Open University, UK

Simon Buckingham Shum, Ágnes Sándor
Anna De Liddo & Michelle Bachler

We report on progress made during the collaboration between KMi’s Hypermedia Discourse Group and Ágnes Sándor (Xerox Research Centre Europe, Parsing & Semantics Group). This is the outcome of her 6 week OLnet Project Expert Fellowship at the OU, funded by the Hewlett Foundation, to develop Collective Intelligence for the Open Educational Resources (OER) community.

Our research investigates the overlaps and complementarities between the outputs from human analysts making sense of 120 OER project reports, using KMi’s Cohere semantic annotation and knowledge mapping tool, and machine annotation of the corpus by the Xerox Incremental Parser (XIP). XIP’s output is imported into Cohere to explore ways to visualize the combined human+machine output, and we present preliminary results from interviews with some of the analysts to elicit their views on XIP’s annotations.

Opening up Election Debate Mapping

Here’s a follow-up for those of you who’ve been following the real time mapping of the first TV election debate, and are interested to pitch in, or for those of you interested in the technological aspects of these emerging tools for participatory sensemaking.

The original maps were mapped in real time, as the debate was broadcast, using Compendium, a desktop tool optimised for very fast mapping (it is often used to provide visual scaffolding for a team meeting either f2f or online – eg. hostage recovery, science teams, researchers). In the forthcoming release of Compendium v2.0, which I was using, this can be done over the internet by multiple analysts working on the same set of maps, rather like a visual wiki.

However, we can also upload the maps as XML, from Compendium, to our social media platform called Cohere [article]. This brings Web 2.0 capabilities to argument mapping. View the Cohere UK Election debate group which is housing all the election debates we map. Now every contribution to the debate has a unique URL, and can be embedded, tweeted, or linked to in a meaningful way (e.g. is inconsistent with, refutes, is evidence for, was predicted by… etc — whatever move you want to make).

Below, for instance, is one contribution to the election debate by Gordon Brown, embedded (like a YouTube movie) in this blog post — click on it to jump into Cohere, in order to see how it is connected to other contributions from Cameron and Clegg:

We can also embed a specific connection, or whole map, in a website, to enable others to view and contribute to it from many different sites. Below is a static screenshot of the network visualization applet we are using to provide interactive map browsing and editing (I haven’t embedded the live one in this page since you may not be running Java. Here’s the live applet if you want to try it).

Here’s the bird’s eye view of the whole of the first TV debate…

…and once you zoom in on a fragment of interest it becomes legible for navigation and editing:

But if views like this don’t work for you either cognitively or technically (and we’re about to replace this with the Adobe Flash version of flare.prefuse.com) then you can also work with the Connection Neighbourhood view.

Below we walk through the web browser interface via which anyone (who’s signed in) can make a new connection.

Here’s a view focused on the focal contribution to the debate, shown in the centre. Mousing over the menu icon brings up the menu of actions you can perform, which includes “Connect”…

In the pop-up dialogue, I’m asserting the Gordon Brown’s assertion reiterates Nick Clegg’s:

The new connection is confirmed, having been added to the map:

If I Get the Snippet for this specific connection, as shown, I can then embed it in this blog, as an interactive fragment of the map that I want to embed in a new commentary, thus:

Obviously, once there are many people adding new ideas and connections to a web debate of this sort, then there might be three different analysts shown in a given connection, with one analyst connecting contributions from two others (example: climate change analysis).

Another advantage of moving from real time private mapping to participatory, asynchronous mapping, is that each node in the network can now have an arbitrary number of websites added to it as backing evidence for the claim it makes. Obvious examples would be fragments from manifestos, or video clips of previous speeches, etc.

Just edit the node, or get very smart, and load the extension we provide for your Firefox web browser. Now when you’re viewing ANY website, and you want to connect something of importance on it to a node in the debate, just highlight the relevant text, and tie it to the idea in your margin sidebar. In the example below, I’m providing more detail from the NI Campaign on the Conservative party website, to an assertion made by Cameron in the debate… you’ll see my annotation on that page as shown, once you’ve loaded the Firefox extension:

Finally, you may have spotted that while this is a hypertext tool, we’re building on years of hypertext research which ha demonstrated how we can make meaningful connections between nodes in the network, not just the “go to” hyperlinks that dominate the Web at present. By using a connection language to model arguments, we can filter the network to show just specific kinds of links. In contrast to Compendium in which you hand-craft every map (great for precise presentations), in Cohere every view is automatically generated and laid out, since an unknown number of participants may be contributing to a given view. Semantic filters on these potentially very large, complex visualizations now start to become rather useful.

There’s a basic semantic filter into broadly positive, neutral or negative connections:

…which in our experimental Java applet, can be refined by any user-defined mix of connection types, or by using preset searches such (example below) Contrast which searches the database from a focal Idea on a specific subset of connection types of a contrasting nature: challenges, has counterexample, is inconsistent with, refutes:

OK, end of mini-tutorial. Hopefully that excites you about the potential of tools such as this to raise the level of public discourse around complex societal issues, the present election being just one timely example. I have a hunch that we also need to generate a more conventional reports which will be familiar to more people, and while harder to show complex interconnections, they can compress more information in a page/screen. Since it’s just another view onto the database, there would be no need to have to choose between them: just flip as required.

If this gets you going, I should also point you to the great work that my colleagues at Debategraph are doing, as they use their tool to map and open up for debate the political interviews conducted by CNN’s Christiane Amanpour, plus many other pressing political issues.

Cohere at Mozilla Jetpack Design Camp, SXSWi

We just got back from the Mozilla Jetpack for Learning Design Camp at SXSWi (Austin TX), where our Cohere Jetpack was one of the 10 winners of their Design Challenge, co-sponsored by the MacArthur Foundation. Jetpack is Mozilla’s new technology to help web developers add powerful new extensions to the Firefox browser. Read Michelle Bachler’s reflections on Jetpack, and the camp, on the Cohere blog, and give it a try! Slides below…

Contested Collective Intelligence

Cohere Poster, CSCW 2010 CI Wkshp

There’s a workshop coming up at CSCW 2010, focusing on Collective Intelligence In Organizations: Toward a Research Agenda. I look forward to reporting on what looks to be a very stimulating gathering…

Anna and I took the opportunity to develop some thinking around how Hypermedia Discourse and sensemaking (as embodied in our Cohere web app) locate themselves within the broader CI landscape:

De Liddo , A. and Buckingham Shum, S. (2010). Cohere: A Prototype for Contested Collective Intelligence. Workshop on Collective Intelligence in Organizations: Toward a Research Agenda, ACM Computer Supported Cooperative Work (CSCW 2010), February 6-10, 2010, Savannah, Georgia, USA. Available as ePrint: http://oro.open.ac.uk/19554

Abstract: This paper presents the rationale for treating Contested Collective Intelligence (CCI) as a significant and distinctive dimension of the broader Collective Intelligence design space for organizations. CCI is contrasted with other forms of CI, and building on research in sensemaking, and the modeling of dialogue and debate, we motivate a set of requirements for an ideal CCI platform. We then describe a social, semantic annotation tool called Cohere, which serves as our working prototype of the CCI concept, now being deployed in several communities.

Cohere movie for Mozilla Jetpack

As you may have seen, in the Jetpack for Learning Design Challenge, Mozilla Foundation have teamed up with the MacArthur Foundation “to help turn the open Web into a rich learning environment and explore new possibilities for learning online.”

So, we thought we’d have a crack at it with our Cohere Web 3.0 app for making web annotations, meaningful connections between ideas, and mapping arguments. Here’s the movie that tells the story, and you can get the Mozilla extension for Firefox and Flock from the Cohere homepage.


[Hi-res version]

Well done Anna de Liddo and Michelle Bachler for putting this together in record time 🙂

So far, we’re through Round 1…

Future plans include extending to video annotation (hunting for good examples and open source code to annotate YouTube clips etc), transitioning to a more powerful Flash based visualization toolkit (we’re working with Flare), and providing templates for critical thinking, such as the argumentation schemes we already have for our desktop knowledge mapping tool, Compendium (which currently uploads XML maps to Cohere).

Debate Mapping the Climate Sceptics

A lot of our argument visualization research is focused now on Climate Change. Here’s an example, thanks to the work that Anna De Liddo is doing on the our Collective Intelligence project. The BBC journalist Richard Black wrote an introductory piece on climate sceptics:

What do “climate sceptics” believe?

You might think that you know the answer, having heard, seen and read numerous counter-blasts aimed at the Intergovernmental Panel on Climate Change (IPCC) over the course of this year, as the three components of its landmark climate assessment were published.

Despite having reported on climate change for more than a decade, I realised at the beginning of the year that I was not entirely sure.

[…] So I decided I had better try to find out.

The result was a summary of the “Top 10” arguments, and possible responses, which we have re-rendered in the Cohere knowledge mapping tool, to make clearer than prose can show what are the key relationships (supporting or challenging) between the different claims in the debate. (Once you’re logged in, you can then add new connections to support/challenge or link to new web resources backing a given contribution.)

[You’ll need Java to view this one] Here’s an embedded “ConnectionNet” applet providing a porthole view into the debate space — a self-organising graph of the whole debate map.

Click on the lower green View in context button to view either of the following maps properly, within Cohere:

Below is an alternative view we call the “Network Neighborhood”, showing a focal Idea with the incoming and outgoing connections one level deep. This provides a different, more structured way to step through the moves in the debate — clicking in an Idea moves it into the central focal point, updating the incoming/outgoing connections:

See also Mapping the Gaza Crisis for the great work our colleagues are doing with Debategraph…

Cohere semantic annotation Firefox Plugin

The social web is about sharing ideas. Cohere makes Ideas a first class object: something you can point to as a URL, embed in another page, link to semantically (you make up the semantics: is an example of… is analogous to… refutes… predicts…), classify (theory/method/data/politician/supernatural phenomenon…), visualize its connections, share, agree with, build on, challenge…

The Cohere Firefox plugin now makes it possible to annotate, or grab snippets from, a web page and turn them into an Idea in Cohere, or meaningfully connect Ideas:

Cohere: Towards Web 2.0 Argumentation

The first paper on Cohere, our Web 2.0 knowledge and argument mapping application, has been accepted for presentation to the main forum for researchers working at the intersection of computing and argument. See you in Toulouse in May!

Buckingham Shum, S. (2008). Cohere: Towards Web 2.0 Argumentation. Proc. COMMA’08: 2nd International Conference on Computational Models of Argument, 28-30 May 2008, Toulouse. IOS Press: Amsterdam [http://oro.open.ac.uk/10421/]

Abstract:

Students, researchers and professional analysts lack effective tools to make personal and collective sense of problems while working in distributed teams. Central to this work is the process of sharing—and contesting—interpretations via different forms of argument. How does the “Web 2.0” paradigm challenge us to deliver useful, usable tools for online argumentation? This paper reviews the current state of the art in Web Argumentation, describes key features of the Web 2.0 orientation, and identifies some of the tensions that must be negotiated in bringing these worlds together. It then describes how these design principles are interpreted in Cohere, a web tool for social bookmarking, idea-linking, and argument visualization.

(PS: and here are the slides from the day…)