Legacy Character Classes

Character classes as defined in ANSI C and POSIX aim to provide a simple means to determine properties of an arbitrary character. The properties that we can analysis are the character's functional use in text (punctuation, alphabetic, digital, space, etc), relational ( [=c=], case insensitive comparisons ) which the locale mechanism lets us make specific to a language convention, orthographic semantics (uppercase, lowercase) and the character's address range (printable, ASCII).

A software developer working with Ethiopic text will want the same utilities for character analysis as he or she had for single byte text. Unicode aware resources will allow for generic character tests so that script specific functions can be avoided. For example a test for an Ethiopic punctuation might take the form:

and avoids the need for an "isEthiopicPunct" equivalent.

While progress is being made towards Unicode based and multilingual aware pattern matching tools, by and large existing tools remain oriented towards the properties of western, Latin based, scripts. In this section we will look at extension of the character class paradigm to accommodate needs particular to Ethiopic script and languages while trying to maintain a perspective towards syllabic writing systems in general.

Syllabic Classes

The first simple extension to the standard character class lexicon would be the introduction of "[:syllable:]" as the logical analog to "[:alpha:]". "[:syllable:]" would match any character with a syllabic property, it is left to the user to restrict the syllabary of interest.

When we looked at Ethiopic character entity names we had our first introduction to the concept of Ethiopic character classes. In fact the naming convention proposed is built upon recognizing the "rows" and "columns" of the traditional syllabary with discretely named terms. It is a common problem to want to detect either the row or column property of an arbitrary Ethiopic character, as we will examine in more detail shortly. We can employ row and column names to define additional POSIX style character classes as follows:

This approach maintains a Ge'ez script perspective and requires the definition of a considerable number of pattern matching terms meaningful only in the applicable address range. The approach can be generalized for use with other syllabaries as follows:

Note that in the final pattern for detecting a first form syllable some means was required to restrict the context to Ethiopic script so as not to match a first form syllable from any syllabary. The percentage sign, "%", is used here to specify the syllabic context by employing the "modulo" meaning of the symbol.

Locale Based Equivalence Classes

In the strictest sense no two members of the Ethiopic syllabary would have the same same phonemic value. The presence of ኧ (U+12A8) is the telltale indicator that this is not quite the case. As a consequence of the phonemic decay of many Ethiopic syllographs spelling correctness in Ethiopian and Eritrean languages works on the notion of proximity. While each language may recognize a canonical spelling for a given word, a rendering may still be regarded as "correct" or acceptable based on its orthographic distance from the canonical rendering. Further, as with American and British English spellings, the canonical spellings are also allowed to change for the same word across national or linguistic borders.

To handle these conventions, pattern matching software need be made aware of the localized rules. A demonstrative sampling is offered in our next table:

Eritrean conventions are based on the conventions taught in primary education since 1991. Ethiopian Tigrigna stresses the same conventions but is more forgiving in the use of the redundant syllabic series.

The table demonstrates a sample of useful character classes and how they would vary with locale setting. With collectively over 80 languages in Eritrea and Ethiopia the table is not intended to be comprehensive but demonstrative with the most familiar classes. Notably a class to fold all ግዕዝ and ራዕብ forms would be desirable when working with southern language of Ethiopia - where many of the classes shown above would not be applicable. It is also worth noting that Ge'ez and Ari may share character classes while not sharing character phonemes. This helps highlight the separation of spoken language from orthography, in this case without consequence to pattern matching, and is certainly the exception and not the rule, as we will now see.

Demonstrating the classes we can consider the case of the Ethiopian Tafari Mekonnen who worked his way thru a military career up to the commander rank of "Ras". His life took a turn to the orthographically more complex when in 1930 Ras Tafari became Emperor of Ethiopia and assumed the coronation name Haile Selassie I. "Tafari Mekonnen" had only two possible spellings while "Haile Selassie" has numerous (not to imply for a moment though that HIM would have used anything but the canonical form). We'll look at these possibilities along with the female name ዓለምፀሐይ and its many spellings.

To be certain, while the renderings shown are logically possible they are not all necessarily probable, though the character classes used are entirely valid. It should be emphasized also that the renderings shown for languages following Amharic do not indicate the acceptable spellings in those languages but demonstrate how the pattern matching outcome would change with the corresponding locales. Indeed when searching for the same terms in a document known to be in the language indicated a matching pattern appropriate for the language would be applied.

The Syllabic Constraint

Owing to the nature of language morphology where derivational rules are developed that rely on consonants and vowels as disassociated entities we expect to be able to apply these rules to written language. In an open syllabary the two (consonant and vowel) are fused together and so a developer is driven to seek out or create tools to isolate these character properties. Regular expressions resources are now indispensable in this field but are somewhat cumbersome to use when applied to syllabaries. The C-V property of a syllable corresponds directly to the rows and columns of the syllabary itself. The syllabic and form classes allow us to match a single character as a member of a group. This is analogous to folding cases ([:ለ:]) or specifying a specific case ([#4#]). However, a limitation appears when we want to specify intersections of the two. An anticipated pitfall would be to attempt:

which matches two characters in sequence and not a member of the intersection:

A convenient solution is to apply the same logic of the syllabic form matching class in constraint notation.

or in the negative expression:

Applied in a small practical example we can develop an expression for the detection of the basic Amharic plural. We can start by defining a word stem as a sequence of syllables (assumed in the Ethiopic context):

which without the utility of the constraint becomes:

and with the expressive power of a syllabic constraint condenses nicely to:

This is of course a very simple example and it is intended that the power of the operator be evident from visual inspection by those only casually acquainted with regular expressions syntax. Applied to very large and real world text and natural language processing problems we can expect the operator to become an indispensable member of the regular expression toolbox.