Architecture of the jsRealB and pyrealb Text Realizers

Guy Lapalme
RALI-DIRO
Université de Montréal
December 2023

This document describes the overall code organization of jsRealB, a system written in JavaScript that can produce English or French sentences from a specification inspired by the constituent syntax formalism. It can be used either within a web page or as a node.js module.

The text realizer is at the very end of a text generation pipeline. Important decisions about What to say have already been made. Sentence structure and most content word choices must also have been decided. But there is still interesting work to do: the realizer must take care of many language peculiarities such as conjugation, word order, elision, number and person agreements. These might seem to be small details, but users appreciate them and find it very annoying or disturbing when they are not properly dealt with. Our realizer allows creating many variations (e.g. negative, passive or interrogative) from a single affirmative sentence structure.

jsRealB has been developed by my students since 2013. The source code is now available on GitHub with a tutorial and demo applications. in 2018, I decided to build on that experience and redesign the system from the ground up. I kept the same external interface using the existing tests and demos to develop the new version. Only the inner workings have been changed and simplified given my experience with the system over the last few years. jsRealB is currently at version 5.0.

I then ported (by hand translation) jsRealB to python to create pyrealb, now at version 3.0. As the JavaScript and python versions share the same code organization (most class and function names are similar) this document applies to both.

Before going into the detailed code organization described in this document, I suggest that you first read the first sections of the jsRealB tutorial and then take a look at these two documents:

This document takes for granted that the user has some knowledge of python and JavaScript.

Classes hierarchy

The basic structure of jsRealB can be understood as a hierarchy of four classes highlighted with a larger border in the following diagram. Terminal, Phrase and Dependent are subclasses of Constituent. This diagram shows for each class, its name, its main local attributes and the most important methods. Arrows indicate subclass relationships.

Parallel class hierarchies have be defined for English and French specific processing: TerminalEn, PhraseEn and DependentEn are subclasses of ConstituentEn; TerminalFr, PhraseFr and DependentFr are subclasses of ConstituentFr. Language specific processing common to PhraseEn and DependentEn is defined in NonTerminalEn and similarly for French.

As can be expected, the same method names appear in the corresponding English and French specific classes. A few do-nothing methods are defined when a processing does not apply to a given language (e.g. some gender related processing in English). This diagram uses the python camel-case convention for class names, JavaScript relies on underscores for separating words in an identifier.

Multiple inheritance is used to determine the appropriate methods according to the language (English or French) in which the instance was originally created. The user does not call the class constructors directly but functions that call an auxiliary function determines the language specific class depending on the current loaded language.

All functions validate their input, in the case of errors, they generate a warning message on the console. Realization is not stopped, the resulting string is the original lemma enclosed in double square brackets.

We now briefly describe the main methods of each class.

Constituent

The constructor initializes the object properties constType with the parameter, prop to an empty object and realization to null. Most Constituent instances also have a link to an object called peng (person, number and gender) and verbs to an object called taux (tense, auxiliary). This object can be shared between constituents; when it is modified by an option, the new value is also available to others. This is used for carrying out the agreements between sentence component (e.g. the number of a subject can affect the verbe of the sentence).

ConstituentEn / ConstituentFr

Most methods are single line functions that return language specific values for default properties, tonic forms of pronouns, etc. But the following functions are more complex:

Terminal

The constructor saves the lemma and call setLemma(),

TerminalEn / TerminalFr

English or French specific processing; these functions are called by Terminal instances:

Phrase

Note that given the applicative order of execution of JavaScript, this constructor is only called once all its parameters have been created. The constructor first calls the Constituent prototype and then initializes its list of elements with the elements parameter: it copies all elements but the last and then calls add(element[last]).

PhraseEn / PhraseFr

Dependent

This is also a subclass of Constituent, its structures closely parallels that of Phrase, but its children are a single Terminal instance and list of dependents (instances of Dependent) instead of elements which are Terminal or Phrase instances.

NonTerminalEn / NonTerminalFr

These classes contains methods shared between Phrase and Dependent instances. In particular, this where is located the Affix Hopping algorithm for English verb conjugation in sentence modifications. There many single line methods for dealing with passive sentences in either French or English. The language specific positioning of adverbs is dealt in a method of this class. In French, the placement of clitic and pronouns is determined here, while there is nothing to do in this respect in English.

Lexicon

These functions can be called by the user to change global information for the realizer. They are described in the documentation.

JSON-tools

jsRealB.js / utils.py

This is not a class, but a file that imports all functions and classes and exports them, so that this is the only file to import to access pybealb. It also adds a utility functions and some useful constants.
All functions that create Terminal, Phrase and Dependent instances are also defined here,