CoNLL 2019 Shared Task: Meaning Representation Parsing --- Sample Graphs

Version 1.1; June 29, 2019


Overview
========

This directory contains a collection of 89 sample graphs in the five framworks
represented in the task: AMR, DM, EDS, PSD, and UCCA.  The sentences are drawn
from Section 00 of (the Penn Treebank selection from) the venerable Wall Street
Journal (WSJ) Corpus.  We only include sentences for which all five graph banks
provide annotations.

The purpose of this sample data is twofold: (a) exemplify the uniform graph
representation format (serialized in JSON) adopted for the task and (b) enable
in-depth linguistic comparison across frameworks.

For general information on the frameworks and MRP file format, please see:

  http://mrp.nlpl.eu/index.php?page=2
  http://mrp.nlpl.eu/index.php?page=4#format


Contents
========

The main contents in this release are the JSON files containing sample graphs
in five frameworks:

$ wc -l */*.mrp
   87 amr/wsj.mrp
   89 dm/wsj.mrp
   89 eds/wsj.mrp
   89 psd/wsj.mrp
   87 ucca/wsj.mrp
  441 total

Each file contains the 89 graphs in the intersection of all frameworks (87 in
the case for AMR and UCCA, for the time being).  These graph serializations are
in what is called the JSON Lines format, effectively a stream of JSON objects
with line breaks as the separator character between objects.

To ease human inspection of these graphs, this package also provides graphical
renderings of all graphs, as separate files (one per sentence) in the ‘dot/’
and ‘pdf/’ sub-directories for each framework.  These visualizations have been
created using mtool, the Swiss Army Knife of Meaning Representation:

  https://github.com/cfmrp/mtool

Additionally (as of Version 1.1), the archive also provides an example of the
file formats used during the MRP 2019 evaluation period.  The file ‘input.mrp’
contains one parser input (‘sentence’), represented as an empty MRP graph, for
which participating systems are expected to output five target graphs, i.e. one
in each of the semantic graph frameworks of the task.  The file ‘output.mrp’
demonstrates what would amount to a complete submission for this input: five
distinct semantic graphs, all with id #20001001, each essentially filling in
the missing fields (‘tops’, ‘nodes‘, and ’edges’) from the input.


Known Limitations
=================

None, for the time being.


Release History
===============

[Version 1.1; June 29, 2018]

+ Add sample ‘input.mrp’ and ‘output.mrp’ demonstrating sumbission procedures.

[Version 1.0; June 19, 2018]

+ Re-release in MRP 1.0 serialization: edge ‘attributes’ now (not ‘properties’).

[Version 0.9; April 9, 2018]

+ First release of sample graphs in five frameworks: AMR, DM, EDS, UCCA, and PSD.


Contact
=======

For questions or comments, please do not hesitate to email the task organizers
at: ‘mrp-organizers@nlpl.eu’.

Omri Abend
Jan Hajič
Daniel Hershcovich
Marco Kuhlmann
Stephan Oepen
Tim O'Gorman
Nianwen Xue