2.1.1. Morphosyntactic specifications

The syntax and semantics of the morphosyntactic descriptions (MSDs) are given in the MULTEXT-East morphosyntactic specifications. The specifications have been developed in the formalism and on the basis of specifications for six Western European languages of the EU MULTEXT project from the 1990s' and in cooperation with EAGLES, the Expert Advisory Group on Language Engineering Standards. The first version of these specifications was released as a report of the MULTEXT-East project but have been since extensivelly revised. Nevertheless, the specifications are still structured as a report, and contain introductory chapters, followed by the list of defined categories (parts-of-speech), and then, for each category, a table of attribute-values, and the languages the features are appropriate for. These so called common tables are followed by language particular sections. Each language section is further subdivided, and can contain feature co-occurrence restrictions, examples, notes, and full lists of valid MSDs, as well as localisation information.

The complete specifications are an XML document, encoded according to the TEI Guidelines.

In specifications are maintained on https://github.com/clarinsi/mte-msd, and, for continuity, mirrored here, i.e. on https://nl.ijs.si/ME/V6/msd/

The source TEI encoding is down-converted to several derived formats:

html

Readable format

tables

MSD tabless, including TEI MSDs as feature structures

cooked TEI

Canonical specifications in TEI

source TEI

Editable TEI specifications on GitHub

xslt

Conversion XSLT (and Perl) scripts on GitHub

2.1.2. Morphosyntactic Lexicons

Languages: Bulgarian, Czech, English, Estonian, Hungarian, Macedonian, Persian, Polish, Resian, Romanian, Russian, Serbian, Slovak, Slovene, Ukrainian.

The MULTEXT-East morphosyntactic lexicons have a simple structure, where each lexical entry is composed of the following fields:

• the word-form, which is the inflected form of the word, as it appears in the text, modulo sentence-initial capitalisation;
• the lemma, which is the base-form of the word;
• the MSD, i.e., the morphosyntactic description of the word-form;
• optional frequency of triplet in some reference corpus.

The sizes of the MULTEXT-East lexica vary considerably between the langauges. A few are quite limited, and serve more as a proof of concept and to assign lexical entries to the MSDs, but most contain over 20,000 lemmas and can serve as medium sized morphological lexica for the languages. In addition to explicating the inflectional behaviour of the most common (and, typically, morphologically the most complex) words of the languages, the lexica also served to establish the definitive set of valid MSDs for the languages.

The lexica are available from the CLARIN.SI repository:

• Erjavec, Tomaž; et al., 2010, MULTEXT-East free lexicons 4.0, Slovenian language resource repository CLARIN.SI, http://hdl.handle.net/11356/1041.
• Erjavec, Tomaž; et al., 2010, MULTEXT-East non-commercial lexicons 4.0, Slovenian language resource repository CLARIN.SI, http://hdl.handle.net/11356/1042.

2.1.3. Linguistically annotated ‘1984’

Languages: Bulgarian, Czech, English, Estonian, Hungarian, Macedonian, Persian, Polish, Romanian, Serbian, Slovak, Slovene.

The novel "1984" by George Orwell is the central component of the MULTEXT-East corpus. This parallel corpus contains the novel in the English original (about 100,000 words in length), and its translations into a number of languages.

This corpus, also called the cesAna corpus, contains word level markup, namely context disambiguated lemmas and MSDs, e.g., <w lemma="it" ana="#Pp3ns">it</w>.

Each MSD is linked to a TEI feature-structure library (automatically derived from the specifications, and written in the back matter of the novel), which gives for each MSDs its decomposition into features, e.g.
<fs xml:id="Pp3ns" xml:lang="en" feats="#P0. #P1.p #P2.3 #P3.n #P4.s"/>
and, in the feature library:
<f name="CATEGORY" xml:id="P0." xml:lang="en"><symbol value="Pronoun"/></f>
<f name="Type" xml:id="P1.p" xml:lang="en"><symbol value="personal"/></f>
<f name="Person" xml:id="P2.3" xml:lang="en"><symbol value="third"/></f>
...

The translations of ‘1984’ are sentence aligned with the English original, with hand-validated alignments. From these en-xx alignments the other bi-lingual and one multilingual alignments were produced automatically. The sentence alignments are stand-off, i.e. encoded in separate files with pointers to the aligned sentence identifiers. The alignments are encloding as TEI link groups, containing links of the form:
<link n="1:2" targets="oana-bg.xml#Obg.1.1.6.7 oana-ro.xml#Oro.1.2.7.7 oana-ro.xml#Oro.1.2.7.8"/>

The corpus is further documented in its TEI headers.

The corpus is available from the CLARIN.SI repository:

• Erjavec, Tomaž; et al., 2010, MULTEXT-East "1984" annotated corpus 4.0, Slovenian language resource repository CLARIN.SI, http://hdl.handle.net/11356/1043.

2.2.1. Structural ‘1984’ and alignments

Languages: English, Romanian, Slovene, Czech, Bulgarian, Estonian, Hungarian, Lithuanian, Serbian, Russian.

The multilingual parallel corpus consists of the novel ‘1984’, about 100,000 words in length. The corpus contains extensive headers and markup for document structure, sentences, and various sub-sentence annotations, these similar to the comparable corpus, but better harmonised over languages.

The translations of ‘1984’ are sentence aligned with the English original, with hand-validated alignments. From these en-xx alignments the other bi-lingual and one multilingual alignments were produced automatically. The sentence alignments are stand-off, i.e. encoded in separate files with pointers to the aligned sentence identifiers. The alignments are encloding as TEI link groups, containing links of the form:
<link n="1:2" targets="oana-bg.xml#Obg.1.1.6.7 oana-ro.xml#Oro.1.2.7.7 oana-ro.xml#Oro.1.2.7.8"/>

The corpus is available from the CLARIN.SI repository:

• Erjavec, Tomaž; et al., 2010, MULTEXT-East "1984" annotated corpus 4.0, Slovenian language resource repository CLARIN.SI, http://hdl.handle.net/11356/1043.

2.2.2. Comparable corpus components

Languages: Romanian, Slovene, Czech, Bulgarian, Estonian, Hungarian.

The multilingual comparable corpus contains a fiction part and a news part, where the data is comparable across the languages in terms of the number and size of texts; each of the 12 parts has approx. 100,000 words. The corpus is structurally marked up with over 40 different elements; however. sub-paragraph markup has not been harmonised across the languages.

The comparable corpus and its components are further documented in their TEI headers. The ‘cmp’ corpus is available for download in the directory herecrp/.

2.2.3. Speech corpus components

Languages: Romanian, Slovene, Estonian, Hungarian, (English, Czech, Bulgarian)

MULTEXT-East produced a small corpus of spoken texts taken from the EUROM-1 speech corpus. It comprises the translations (from English) of forty short passages of five thematically connected sentences. This written part of the corpus is included together with the other parts of the cesDoc corpus.

For four languages, the texts have also been read, recorded and included in the distribution. The corpus texts contain links to the spoken passages stored as .wav files. The speech files are, due to their size, stored and distributed in a separate bundle.

The speech corpus and its components are further documented in their TEI headers. The corpus is available in the directory spc/.