The application concerns the recognition of bio-molecular events (bio-events) that are mentioned in biomedical literature. A bio-event is a change of the state of a bio-molecule or bio-molecules, e.g. phosphorylation of IkB involves a change on the protein IkB. This EPE downstream application has its roots in the BioNLP Shared Task series on event extraction, specifically the GE tasks in 2009, 2011, 2013, and 2016. Below is an example of a sentence with several events:
An event extraction system is required to (a) identify the biological entities in a sentence; (b) identify the triggers of the events in the text and the event type; and (c) identify the arguments of the events and their relation type.
The majority of the event extraction systems are machine-learned, using rich feature sets extracted from a dependency parse of the underlying text. The TEES system has consistently ranked in the top positions, has been adopted by many in real applications, and will form the basis of the evaluation in the EPE 2017 parsing task. TEES is fully machine-learned, drawing most of its features especially from the shortest paths connecting the entities and event triggers in the dependency graph of the sentence. This makes the system both very sensitive to grammatical structure and independent of individual, specific dependency frameworks, i.e. a suitable down-stream application for extrinsic evaluation of different types of dependency representations and associated parsers.
The MPQA project (Wiebe et al. 2005) defined an annotation scheme and created a corpus of annotated expressions of opinions (or private states). The main building blocks in this scheme are three types of linguistic expressions:
Each instance of these types of expressions is connected to an opinion holder (or source, in the terminology of Wiebe et al., 2005). This is a lingustic expression that refers to the person expressing the opinion or experiencing the emotion. This person may not be explicitly mentioned in the text, for instance if this is the writer of the text. Furthermore, each DSEs and ESEs is associated with a polarity: positive, negative, or neutral.
To exemplify, in the sentence
"The report is full of absurdities," Xirao-Nima said.
the expression full of absurdities is an ESE with a negative polarity, said a DSE, also with a negative polarity, and Xirao-Nima the opinion holder of the DSE as well as of the ESE.
In the EPE shared task, we will evaluate the effect of parsers on the UniTN system (Johansson and Moschitti, 2013), which extracts MPQA-style opinion expressions from text. It carries out this analysis by breaking it into a number of subtasks, viz. (a) marking up the basic types of expressions (DSE, ESE, OSE); (b) polarity classification of DSEs and ESEs; (c) extraction of opinion holders for each expressions; and (d) a reranking step based on rich features modeling the interaction between expressions. Of these steps, (c) and (d) rely on features extracted from syntactic trees and role-semantic dependency graphs, and are likely to be affected by the nature and quality of available parses.
The downstream evaluation of systems in the EPE shared task will be carried out according to the evaluation procedures described by Johansson and Moschitti (2013). Due to the sometimes fuzzy nature of annotation in MPQA (especially of ESEs), they define overlap-based precision and recall measures, in which an expression proposed by the system is counted as partially correct if it has the right label and overlaps with a gold-standard expression.
There are two common data sets that annotate expressions of negation and corresponding scopes, viz. the BioScope (Vincze et al. 2008) and Conan Doyle (Morante & Daelemans, 2012) corpora. In this context, linguistic expressions of negation are commonly called (negation) cues, and for each cue its scope is conceptualized as the part of the enclosing utterance whose interpretation is negated. Additionally, the Conan Doyle annotations introduce the additional notion of semantically negated events (or states) within scopes. Both cues, scopes, and events (if present) are annotated at the surface string level, i.e. as sub-string spans. In the following examples, the cue is colored in red, and the scope in blue:
Mr. Sherlock Holmes, who was usually very late in the mornings, save upon those not infrequent occasions when he was up all night, was seated at the breakfast table.
Mr. Sherlock Holmes, who was usually very late in the mornings, save upon those not infrequent occasions when he was up all night, was seated at the breakfast table.
Accordingly, the analysis of negation is typically interpreted as comprised of at least two inter-related sub-tasks, viz. (a) cue detection and (b) scope resolution. Of these only the latter has been demonstrated to be sensitive to grammatical analysis, as there will often be morpho-syntactic or surface-semantic properties relating a cue with its scope. Thus, the EPE downstream application for negation analysis will limit itself to sub-task (b), scope resolution (including event identification, where applicable), and simulate a perfect solution to sub-task (a) by using gold-standard cue annotations.
The so-called UiO2 system of Lapponi et al. (2012) approaches scope-resolution through token-level sequence labeling, i.e. the scope for a given cue is determined as the union of tokens that are classified as in-scope; events, if present, are further classified as a sub-set of the in-scope tokens. An important source of information to the sequence labeler in this approach are features derived from a syntactic dependency graph, for example distance measures and dependency paths in relation to the negation cue. UiO2 was the best-performing system in the ‘open’ track of the 2012 *SEM Shared Task (Morante & Blanco, 2012) and was later used as one of the downstream applications in the contrastive study of Elming et al. (2013).