Alphabetic Characters not recommended in UTS#39 but part of the DNS Root Zone or Second-Level Reference LGR

This document has been submitted as a UTC document. For convenience in documenting the character list it is presented using an LGR template format. A few minor details of the boilerplate in that template may not be applicable in this context and should be disregarded.

This collection comprises 21 characters from ICANN's Root Zone LGR [RZ-LGR] (plus one character only found in the Second-Level Reference LGR [RefLGR]) that are neither Recommended or for optional Inclusion in UTS#39, as well as their uppercase equivalents, for a total of 38 characters.

Recommendation

These 38 characters should be given Identifier_Type Recommended in light of the extensive research documenting their use by languages with a robust enough infrastructure and usage to qualify for inclusion in the DNS Root Zone LGR as well as in Reference LGRs for the Second Level.

Rationale

The DNS Root Zone is among the most restricted public namespaces, with a mandate to combine security considerations with universal access. It makes little sense for Unicode's default identifiers to differ in the margins, particularly where it comes to letters within a particular script. The number of characters missing from Identifier_Type Recommended is a vanishingly small percentage of the total, and at this writing there is a much larger set of characters that not part of the [RZ-LGR] yet that have Identifier_Type Recommended based on much less thorough analysis.

While any cutoff of the set of characters to be recommended for general identifiers is by necessity somewhat arbitrary in the details, the vetting of characters for the [RZ-LGR] was a particularly thorough process (as described below), and the rationale for each and every included character is documented in the published files. Based on that, it would seem to call for compelling reasons to reject aligning the Identifier_Type of any of the characters presented here.

Background

ICANN has published a series of Second-Level Reference Label Generation Rules [RefLGR] for a variety of scripts and languages. Their combined repertoire is a superset of the combined repertoire for the Root Zone Label Generation Rules [RZ-LGR], mainly adding characters ineligible for the DNS Root Zone, but also HYPHEN-MINUS and various digit sets, but also some characters not or not yet supported in the Root Zone.

The starting point for the Root Zone repertoire is the combined set of IDNA2008 PVALID characters that are letters from any of the Recommended scripts, except Bopomofo, which the IDN community regards as special use (education only). From this set were further subtracted any characters deemed not in general use, such as “historic”, “obsolete”, “phonetic”, “religious use” or otherwise of uncommon use. This analysis relied on available documentation published or archived by the Unicode Consortium but was created independent of the Identifier_Type classification. The resulting Maximal Starting Repertoire [MSR] was publicly reviewed and published.

For each script in the Root Zone, an independent team of local experts reviewed the [MSR] and selected a subset suitable for domain names in the Root Zone for that script; see [RZ-LGR]. This review was based on whether any character could be documented as being used for a language in sufficiently widespread everyday common use. For the Han script, established registry practice was used to make the cutoff.

The [RefLGR] was based on the [RZ-LGR] by relaxing some of the restrictions specific to the Root Zone, besides ASCII and those native digits that are in common use, that included allowing some intra-word punctuation and joiners, as well as one character found in additional character sequences supported for Sinhala. In addition, the [RefLGR] supports the Thaana script, while support for that is in progress for the [RZ-LGR]. The Tibetan script is not yet supported, as work on reviewing the repertoire has not commenced. The [RefLGR] also supports two Limited_Use scripts; these are ignored in this analysis until such time as UTC receives and approves a proposal to change their status.

The subset of characters found in this document includes one character, U+0DA6 ඦ that is found in the [RefLGR] as part of an enumerated character sequence, but not in the [RZ-LGR].

The Root Zone and Reference LGRs, like the MSR, are specified in a format defined in RFC7940, which uses NFC and is limited to lowercase per IDNA2008. Nevertheless, the research on the usage of any cased character carries through to the case pair. Therefore, the uppercase equivalents of all [RefLGR] characters listed here are included in the recommendation above. A few characters are only used as the base character in a combining sequence in the context of a given language.

Note: All characters have tags matching their Identifier_Type values, except Uppercase equivalents are tagged as Uppercase. For all lowercase characters, a reference is given to a source document, from where references can be followed up to a discussion of selection criteria. Some characters may have additional references citing attestation of their use. A comment identifies the language(s) that prompted inclusion into the [RefLGR] as well as their [EGIDS] level, and occasionally, some other information.

Discussion and Review

A comparison of the characters to lists of exemplar characters in CLDR and SIL found a match for all but three of them. The comments and/or reference information have been updated to reflect additional reasons for their inclusion.

• U+068E ڎ ARABIC LETTER DUL - this is found in a Malaysian character standard for Jawi [JW]
• U+0DA6 ඦ SINHALA LETTER SANYAKA JAYANNA - the character is attested in modern use, although in a limited number of sequences.
• U+17CC  ៌  KHMER SIGN ROBAT - this character is used with loan words. These words are commonly used, especially ព តមាន (vartamāna, Information), etc. For additional background and examples, see Section 5.4.7 “Robat Sign” in [Proposal-Khmer].

Contributors

This excerpt was prepared by Asmus Freytag, based on published data found in [RefLGR] and reference information from [MSR]. For details on the process and contributors to those projects, see [RefLGR-Overview], in particular, Section 1, “Overview” and Section 6, “Contributors”. Mark Davis, Michel Suignard and Roozbeh Pournader have contributed feedback.