Argumentation Mining in News Articles Related to the War in Ukraine

Please visualize this guidelines in a desktop or laptop screen, not in a mobile device. If you are not able to visualize the images or the videos shown in this page later, consider opening the page in incognito mode or using another browser.

The purpose of this annotation task is to identify claims and premises in news articles related to the war in Ukraine, as well as the relations between them. We define this concepts below.

Argument Mining Definitions

In this section, we provide definitions of the concepts related to this annotation task along with examples to illustrate them.

Consider Example 1, which presents an argument about the military aid to Ukraine. The writer argues whether the international community should increase military aid to Ukraine, providing reasons in favor and against this claim.

Elementary Discourse Units (EDUs)

To identify the different components in an argument we need to divide the text in segments called EDUs. An EDU is a self-contained unit of text that conveys a single coherent piece of information. EDUs can be a full sentence or a sentence's clause. That is, an EDU must have a subject and a predicate. The predicate must contain a verb. For instance, in the sentence "Providing Ukraine with advanced weaponry, they claim, serves as a deterrent", the subject is "Providing Ukraine with advanced weaponry" and the predicate is "serves as a deterrent". Example 2 delimits the EDUs between square brackets.

Note that not all parts of text have to be part of an EDU. Punctuation marks, conjunctions, and transitional words are often excluded since they don't belong to one segment or another. This is the case for the words "while", "as", "however", "nonetheless", and "because" in Example 2. Also, pieces of text that state indirect speech can be excluded. For instance, the phrases "some argue that", "others caution that", and "proponents believe that" are not part of the EDUs in Example 2. An exception is the phrase "they claim" in "Providing Ukraine with advanced weaponry, they claim, serves as a deterrent". Leaving it out would split the EDU in two parts that do not convey a coherent meaning. Another exception arrises when excluding the indirect speech phrase would result in an meaningless EDU. In Example 3, the phrase "The Kremlin leader justifies" might be considered indirect speech, but removing it would leave the EDU without a clear subject.

Argument Components

There are two types of components in an argument:

• Claim: A claim is a statement that the speaker or writer wants the audience to accept.
• Premise: A premise is a reason or piece of evidence that the arguer puts forward to persuade the audience to accept or refute a statement. Premises form the basis upon which the conclusion (i.e., claim) is built. A premise can correspond to facts, statistics, quotations, reports, examples, findings, physical evidence, or other reasoning methods

In this study EDUs must be annotated as a claim or a premise. Therefore a sentence can contain several argument components. Claims and premises do not overlap, a piece of text can be annotated as a claim or a premise, but not both at the same time. Also, sometimes an EDU does NOT present a stance, assertion, or justification related to any argument, in this cases the EDU should NOT be annotated as a claim or a premise. This is usually the case for the questions posed by the journalist.

Example 4 shows the claims and premises in the argument.

There are indicators that can help to identify claims and premises. For instance, claims can be introduced by phrases such as "I believe", "I think", "I am in favor of", "I do not believe", "for this reason", "however", "nonetheless", and "so". This is the case of the phrase "Nonetheless, proponents believe that a strong response is necessary" in Example 3. Premises can be introduced by phrases such as "because", "for example", "since". This is the case of the phrase "a strong response is necessary because failure to support Ukraine might embolden other authoritarian regimes" in the example. This is not an exhaustive list, other indicators can be used to identify claims and premises. Also, claims and premises can be introduced without any indicator.

An argument can contain multiple premises linked to a claim. A premise does not exist in isolation, it is always supporting or attacking a claim or premise, while a claim can exist without any premises. The claim can appear at the beginning, at the end, or in the middle of the argument. For instance, an argument can follow these structures:

• Premise, premise, premise, therefore Claim
Example 5.

The ongoing conflict has resulted in severe civilian casualties, and has caused international trade disruptions; therefore, a diplomatic resolution is urgently needed.

• Claim because Premise
Example 6.

Ukraine deserves international support because it is defending its sovereignty against aggression.

• In my judgment, Claim
Example 7.

In my judgment, further sanctions on Russia will be ineffective at changing its actions in Ukraine.

Notice that all claims and premises in the past examples have a subject and a predicate because they must be EDUs. The following are examples of text spans that are not claims or premises because they are not EDUs:

Argument Relations

Argument relations describe the connections between different elements within an argument or between multiple arguments. We consider two main types of argument relations:

• Support: A support relation exists between two elements when one element provides evidence, reasons, or justifications for the other element.
• Attack: An attack relation exists between two elements when one element challenges or contradicts the other element.

Both types of relations can exist between: (i) a premise and another premise, (ii) a premise and a claim, or (iii) a claim and another claim. Argument relations have a directionality, where the source component is a reason for or refutation of the target component. In this study, we consider that a text segment can have a maximum of one outgoing relation. That is, a segment can support or attack at most one other segment. However, a segment can be supported or attacked by multiple segments. When in doubt, the annotator should consider the most relevant relation.

A premise does not exist in isolation, it is always supporting or attacking a claim or premise, while a claim can exist without having a relation with other elements.

Relations can be stablished between text segments across different paragraphs. For instance, a common structural pattern in news writing consists of an introduction, body, and conclusion. Typically, argument components presented in the body of the article serve to support or attack the text found in the conclusion. Furthermore, the introduction and conclusion paragraphs often contain similar information, reinforcing key ideas. As a result, argument components introduced at the beginning of the article could be directly linked to those appearing at the end.

In addition, the general argument can be spread across different speakers. It is common for journalists to present their own viewpoints while incorporating quotes from experts, public figures, or references to scientific reports. These elements can be linked regardless of the speaker. Another common scenario is the reporting of interviews, where argument components introduced by the interviewer and the interviewee may support or attack each other.

The next figure shows the relations between claims and premises in the argument. Notice that C1 is supported or attacked by many other segments. However, all segments only support or attack at most one other segment. Also, notice the relations from one premise to another premise. For instance, P3 supports P2 by specifying how providing advanced weaponry serves as a deterrent (it discourages further Russian advances). In general, if premise supports or attacks premise B, then premise A is a more specific premise that provides additional information about premise B.

Text genres and their argumentative structure

News articles can take various forms, each with distinct argumentative patterns. Here we outline some common types of news articles that you may encounter in this annotation task. Understanding these genres can help you better identify and interpret argumentative components within the text. Note that this list is not exhaustive, and genres often overlap, that is, a single article can incorporate elements from multiple genres. At the end of guidelines we provide examples of each genre.

1. Classic News Reporting: This genre focuses on delivering factual information about recent events. As a result, these articles are less likely to contain argumentative components. When annotating, focus on identifying factual claims and premises that support them. Example 11 provides a case of this genre with a strong argumentative structure, while Example 14 is a more straightforward example of classic news reporting.
2. Analysis of Commentary:These articles offer interpretations of events, provide deeper insights, or express opinions. They are rich in argumentative components and often reflect the author's perspective on the issue. There are two main variants:
• Contrastive (Two-sided): It presents arguments from multiple sides of an issue, allowing for support and attack relations between claims. For example, a piece might present Russia's viewpoint on the conflict, followed by an analysis of Ukraine's perspective, with the author drawing connections or highlighting contradictions. Example 4 is an instance of this genre.
• Viewpoint (One-sided):It presents only one perspective, focusing on the support relations. The author will likely provide reasoning and evidence to back up the main claims, but the article will not contain many counterarguments or attack relations. Example 13 illustrates this genre.
3. Interview: The text is typically structured in a question-answer format. The questions rarely contain argument components, as they are not assertions nor do they provide pieces of evidence related to other parts of the text. However, answers provided by the interviewee often contain claims and premises. These answers may also present supporting or opposing arguments, particularly when discussing controversial or complex topics. Example 10 is an instance of this genre.
4. Narrative (Experiences): This type of writing tends to focus on storytelling rather than argumentation. As a result, it rarely contains claims or premises. However, narratives can serve as a source of evidence or examples for a claim. Example 14 is an instance of this genre.
5. Speeches: Speeches, whether political or ceremonial, are designed to persuade or inform an audience. They are often rich in support relations. Sometimes speeches also engage with opposing views, for instance when the speaker quotes an opponent's argument to refute it. In these texts, look for clear claims and the reasoning that supports them, often tied to a particular goal or agenda. Example 12 illustrates this genre.

Annotation Process

For each annotation task, the annotator will be presented with a segment of text from a news article. Follow the steps below in strict order to ensure a consistent annotation process:

• Familiarize with the text: Begin by reading each paragraph in its entirety before identifying claims and premises.
• Annotate Claims: After reading the paragraph, identify and annotate any claims.
• Annotate Premises: Proceed to identify and annotate premises.
• Annotate Premises' Relations: Link premises to claims or other premises within the same paragraph or in other paragraphs, assigning each link as either a support or attack relation.
• Annotate Claim's Relations: Link claims to other claims within the same paragraph or in other paragraphs, assigning each link as either a support or attack relation. Not all claims need to be linked to other elements.

Remember: Each segment annotated as claim or premise should independently convey a meaning. The segment can be a full sentence or a clause. That is, it must have a subject and a predicate. If the segment does not present a stance, assertion, or justification related to any argument, it should not be annotated as a claim or a premise. A paragraph could contain only claims, only premises, or a mix of both. An argument component can support or attack at most one other argument component.

The Annotation Tool

To access the annotation platform click here.

1. When you first access the platform you might get a message saying "the website doesn't support a secure connection with HTTPS" or a similar message. Just click "Visit site" or "Continue to site". If you are unable to access the platform, consider opening the website in incognito mode.
2. You will see the following page. To access, use the user and password that were given to you.

3. You will see a page similar to the following one. Click on the box that corresponds to your username (e.g. if you username is big_blue_bulbasaur11@user.com then click on the box named big_blue_bulbasaur11). Make sure that the numbers on the top right corner are all 0 (as showned in the picture). If they are not, you are in the wrong project.

4. Watch the following video to see how to do the annotations in the platform

Attention Checks

(More) Examples of Annotation

Below are examples of annotated arguments. In all cases [claims] are highlighted in blue and [premises] are highlighted in yellow. In the figures, support relations are shown with green arrows, while attack relations are shown with red arrows.

Notice that the first sentence "The construction of the Ukrainian nation" does not independently convey meaning, therefore it cannot be considered a claim or a premise. Also, the second sentence is a question which conveys meaning by itself, however it does not present a stance, assertion, or justification related to any argument.

This example contains an extract of a speech by Vladimir Putin. Speeches are usually rich in argument components, and they mostly rely on support relations. See the argumentation structure of Example 12 in the next figure.

Note that, overall, Example 13 can be considered an article of the genre of analysis or commentary. However, the second paragraph corresponds to a narrative. This is an example of how different text genres can be present in the same article. Also, it shows how narratives can be used to support claims and premises in an argumentative context. See its argument relations in the next figure.

Examples 14 and 15 do not contain any argumentative structures. Therefore these examples do not contain any annotations. Example 14 simply presents facts which could be used as premises in an argument, but they are not used as such in this case. Premises cannot exist in isolation, that is, they must be used to support or attack other components of the argument. Similarly, Example 15 is a narrative which states a sequence of events without making any assertion.