Simon.BuckinghamShum.net

The Argument Interchange Format (AIF) is a community effort to agree a formal lingua franca for computer-supported  argumentation. The first version is presented in Knowledge Engineering Review.

I spent 2 days last week in Glenshee, at an EPSRC-sponsored workshop bringing together a diverse group of researchers working on computing and argumentation (conference/journal). Working from 9am-8pm for 2 days solid (!), our mission was to write a report drafting AIF 2.0. This was all “thanks” to Chris Reed who runs the ARG at Dundee U, specifically his Dialectical Argumentation Machines project, who did a great job as convenor and taskmaster 😉

One of the novel aspects of this gathering is that in contrast to AIF 1.0, which was driven by the needs of computer science and AI perspectives on argumentation, they also invited “less formal” people like Tim van Gelder, David Price and me, whose tools are semiformal at most, but are (correspondingly) achieving more rapid uptake, since we do not have to worry so much as our AI colleagues about the hygiene of our representations for formal processing (the price we pay is that it is much harder to automatically evaluate our dialogue/argument maps — we leave that to human analysts scaffolded by the visual language).

As a contribution to the forthcoming report, I wrote a use case (PDF) which sews together a number of the dialogue and argument mapping tools represented at the workshop (Compendium, Cohere, Debategraph, Rationale, Carneades), which centres on an imaginary team of analysts working on the legitimacy of the 2003 Iraq invasion.

Below I’ve also pasted my position statement, and an introductory piece on approaching the design of something like AIF through a human-centred design lens.

User-centred design perspectives

A user-centred perspective to the design of a formalism such as AIF focuses attention on questions such as:

• What is AIF’s value proposition?
• Who are the stakeholders?
• What are the implications of designing for a heterogenous mix of users and platforms?
• How does AIF fit into the real world activities, tools and intellectual capabilities of the range of envisaged users who are either conducting argument analyses based on material, or engaging in argumentation with each other?

When we consider the socio-technical “ecosystem” of argumentation platforms and users, it is clear that it is heterogeneous: diverse kinds of users and platforms, with different kinds of capability. This has consequences, for instance:

• The passing of data will not be automatic in all cases. Only “one-way” interchange will be possible from more expressive to less expressive platforms. Data from less expressive platforms will require human intervention to provide missing details before more expressive platforms can add value to the analysis.
• There is also the possibility that AIF-interoperability between two systems will fail, not because of limitations on AIF’s part, but because the two systems are in principle incompatible. This might be because the recipient platform requires data that the source cannot provide, or because they have incompatible models of argumentation. The introduction of a mediating representation such as AIF will not help in such situations.
• With “users” ranging from members of the public (e.g. sharing their views in a public debate), to students, to forum moderators, to software developers, to academic argumentation researchers, we are dealing with a vast range of computing expertise, familiarity with argumentation theory, and tasks. To what extent will this shape AIF?

The insights from sociologists of science, and in particular, of how software and data standards are specified (e.g. Bowker and Star, 1999), provide intruiging accounts of the diverse constraints that lead to a given model. These are often compromises: when a diverse community seeks to agree a standard, they are seeking to satisfy many intersecting and sometimes conflicting goals. Ensuring compatibility with one’s own research interests may be in tension with the needs of developers or end-users. The AIF 2.0 standard, like any other, will require not only technical elegance, but a sufficiently aligned mission, and spirit of collaboration that has taken us this far.

Value proposition (VP)

The design of a new “product” must provide a compelling value proposition:

• What problem is solved?
• How much does it cost? — in every sense of the word: e.g. money to employ suitably skilled staff; learning curve to use successfully; compatibility with existing technologies and methods…)
• Who will pay?

What is AIF’s value proposition? To help us learn by analogy, we might challenge ourselves to complete this:

AIF is to < beneficiary community > as

<another model/standard> is to < success story beneficiary community >

We will consider possible answers to this as we consider the possible beneficiaries.

Academic audience

One audience for AIF is the community developing it: researchers in the field that has come to be known as Computational Modelling of Argument (COMMA) are developing it to solve commonly felt problems.

An obvious value proposition question is: Why has AIF 1.0 not taken off and established an active user community already? Consultation with AIF workshop participants identified the following factors:

1. There was no usable reference implementation, only paper specifications, plus a few small prototypes
2. It was not expressive enough for COMMA researchers
3. There was no engagement with communities building argument mapping platforms, who do have real user communities, and data to share. Moreover, it was too complicated for non-COMMA researchers to grasp.
4. A significant proportion of the COMMA community is working with abstract frameworks such as Dung’s, which has a very different model of argumentation from naturalistic argument, and which does not have significant databases of arguments: cross-platform data exchange was not a pressing problem for these researchers whose focus is on the formal properties of argumentation reasoning engines.

Factor 1 (reference implementation) we hope will be addressed as developers work to the draft specification. Factor 2 is addressed through a modular approach to adding in ontologies of importance to more formal COMMA researchers, and Factor 3 through the involvement in AIF 2.0’s design of argument mappers who have significant user communities (see the use case in the appendix). Factor 4 should begin to be addressed by AIF 2.0, which we hope will make it easier for theoretical researchers to obtain substantive datasets from researchers who have material on their platforms.

Continuing the focus on value proposition, it has been argued that apart from being an interchange format, AIF also provides a theoretical sounding board against which researchers can compare and contrast their approaches. Clearly, researchers already have established ways to do this: the argument is that AIF seeks to add a new kind of value by requiring integration at a much finer granularity and precision. For instance, we can now move beyond agreeing that two approaches are “broadly compatible” or “inspired by the same argumentation model”: AIF invites researchers to prove this very concretely, and to learn from the successes and failures of attempting to do so.

Developers

Software developers building systems to model and/or mediate argumentation, will welcome AIF if it saves them time, or leads to more elegant or efficient systems. They will be looking for clean code, modular architecture, solid APIs, etc. But they in turn will be paid (often) by solution providers who (should) bring to bear the end-user requirements.

Solution providers

From a Human-Computer Interaction/Design orientation, solving real world problems is the key driver.  The mission is to add value to people’s lives, including getting an existing job done more efficiently, getting new kinds of jobs done that weren’t possible before digital tools, and enhancing the aesthetic dimension to life.

Several argument mapping groups have spent a long time designing, trialling, evaluating (and pitching to funders) the benefits of informal/semiformal argument structuring tools. We are teaching pupils, students, researchers and other professionals to engage in the normative reconstruction of naturalistic spoken and written discourse, from which they then gain personal and group cognitive benefit. We face new literacy challenges when it comes to reading and writing dialogues and arguments in more structured ways. Who exactly will use the tools?

User-centred strategies/perspectives

Start with success stories: One strategy for deciding what should be in or out of AIF is to examine whatever we might consider to be our biggest success stories in computer-supported argumentation (broadly defined), and ask what should be learnt, if/how they could have been even better with AIF, or how AIF (as an encapsulation of the community’s collective intelligence) would help propagate key design advances/patterns. We have not yet had the chance to do this.

Design challenges: Another strategy (either in the absence of, or in addition to real success stories) is to compete to engineer systems solutions to challenges which the community agrees are meaningful (cf. Robot Olympics). Solution quality drives this, with a given theoretical/academic perspective forced to justify itself in specific contexts. Understanding those patterns is then extremely important to provide the frameworks preventing others from making the same mistakes.

Argument modelling challenges: Apart from engineering challenges, we might also have argument modelling challenges, such as agreed texts that need to be modelled either manually by analysts, using a given tool, or automatically by approaches using computational linguistics (here TREC is the obvious analogy to draw). An initial experiment along these lines was GlobalArgument.net (GA.net).

Dominant user interaction paradigms: Another approach starts by reviewing what are the most compelling modes of usage in argument mapping tools and online deliberation platforms, and deriving requirements from those that AIF must support. Examples would be:

• Document annotation (typically Web) is found in Cohere and OVA, Araucaria supports offline annotation, and other Compendium and bCisive support linking to external files/URLs. AIF must support the notions of:
  • restating/paraphrasing/quoting, grounding nodes in the argument network in external sources, and better, in specific locations in those sources
  • distinguishing the authors of those argument nodes from the authors of original sources.
  • Task-specific argumentation: people always conduct argument analysis or engage in argumentation in a context, and domain-specific tools tend to do better (eg. specialising the notation to the language of the domain, integrating with other technologies in that domain’s ecosystem). What are the implications for AIF?

Use cases: Use cases seek to envisage who would use AIF-interoperable tools, and what benefit they would gain, in order to clarify not only the vision of the overall effort, but also identify any requirements on AIF. Examples include the following:

• Exporting AIF from different platforms, and importing it into an AIF-compatible network visualization package. We assume that these are better than can be provided by passing a generic graph viz package generic graph data. (This might be smoothly delivered via web services etc, should this prove feasible in principle)
• Exporting AIF enables more powerful statistical NLP than is possible on an unstructured corpus.
• AIF aggregators, thus enabling searching across argument databases.
• Use of AIF as structured markup to claim trustworthiness

Categories: Events, Software
Tags: AIF, argument-mapping, argumentation