ENIAM

The ENIAM categorial syntactic-semantic parser is a tool generating logical forms for Polish sentences. The parser works on raw data and carries out successive stages of text processing: tokenisation, lemmatisation, recognition of syntactic dependencies, annotation with word senses and thematic roles, partial disambiguation and creation of a semantic representation. It integrates information from resources created within CLARIN-PL, among others plWordNet (Słowosieć) and Walenty.

The successive stages of text processing are carried out in a fuzzy processing pipeline. The parser does not disambiguate the ambiguities arising after each processing step on the fly. Instead, it creates a compact representation of the ambiguous result, which is passed to the next stage; disambiguation is performed at the end of the pipeline.

The initial stages of text processing (preceding the determination of the dependency structure) are referred to as preprocessing. They are carried out by the pre program, a standalone network server with which the eniam parser communicates.

Resources are integrated during preprocessing. The processed sentence is annotated, among others, with information from the following resources: SGJP, PoliMorf, plWordNet and Walenty. The subsequent preprocessing stages are tokenisation, lemmatisation, named entity detection, word sense assignment and valence assignment. Abbreviations are also expanded and multi-word expressions recognised at this stage.

The text annotated during preprocessing is then processed by a parser based on categorial grammar (Type Logical Categorial Grammar). This involves generating lexicon entries, determining the dependency structure, assigning semantic valence, disambiguation and creating the semantic representation.

Parsing with categorial grammars amounts to proving theorems in non-commutative intuitionistic linear logic. Grammar rules are universal, language-independent inference rules of the logic. As a result, the grammar is fully lexicalised, i.e. all knowledge about the processed language is contained in the lexicon. LCG connectives express basic linguistic phenomena such as concatenation (building feature vectors), ambiguity, argument requirement, polymorphic arguments, type summation, empty arguments, multiple arguments, parametrised ambiguity and list creation.

The lexicon is generated dynamically for each sentence on the basis of the morphosyntactic information of the word form and the valence schema of the lexeme. It is a direct translation of the information provided by lemmatisation and contained in Walenty into the language of LCG connectives.

The grammar is supplemented with direct speech constructions and a lexicalised description of temporal expressions.

It should be noted that ENIAM does not use a grammar of Polish in the traditional sense of the word. Grammatical constraints between words are imposed by the valence dictionary, supplemented with information on possible modifiers for a given lexeme type. Since categorial grammar is fully lexicalised, there was no need to create general rules stating, e.g., that a noun agrees with an adjective in case, number and gender. This information is contained partly in the valence stating that a noun can be modified by an agreeing adjective, and partly in the procedures translating valence frames into the categorial grammar lexicon.

Usage

To use the parser, first start the pre server (located in the pre subdirectory). The server takes 4 to 10 minutes to start, depending on the hardware, and requires at least 4 GB of RAM. When the server is ready, it prints „Ready!”.

To parse a single sentence „Ala ma kota.”, run the following command in the parser directory:

./eniam 'Ala ma kota.'

This generates the logical form of the sentence in the output directory.

To parse a corpus, provide ENIAM with a file containing sentences separated by newlines. A concurrent version of ENIAM is adapted to parsing corpora; it is run with the command

./eniam.distr file

where file is the file containing the corpus.

The config file contains the ENIAM configuration. It specifies the location of the directories with lexical resources and Walenty, the port and server on which pre is available, and the path to the output directory. It also sets the number of generated solutions, the parser timeout, the memory limit for the parser and the number of parser processes run concurrently by eniam.distr.

The output directory contains logical forms in the form of semantic graphs and formulae of extended first-order logic. Semantic graphs are saved in XML and visualised as PNG images. First-order logic formulae are expressed in MathML.

Parser demo