Technical Reports

Unicode Technical Report #24

Script Names

Summary

This document provides an assignment of script names to all Unicode code points. This information is useful in mechanisms such as regular expressions, where it produces much better results than simple matches on block names.

Status

This document has been reviewed by Unicode members and other interested parties, and has been approved by the Unicode Technical Committee as a Unicode Technical Report. It is a stable document and may be used as reference material or cited as a normative reference from another document.

A Unicode Technical Report (UTR) may contain either informative material or normative specifications, or both. Each UTR may specify a base version of the Unicode Standard. In that case, conformance to the UTR requires conformance to that version or higher.

Contents

• 1 Introduction
• 2 Usage Model
• 3 Values
• 4 Data File
• References
• Modifications

1 Introduction

Scripts.txt provides data for a mapping from Unicode characters to script names. This information is useful for mechanisms such as regular expressions, where it produces much better results than simple matches on block names. There are quite a number of problems with using block names to distinguish characters:

• Blocks are simply ranges, and contain code points that are unassigned
• Characters from the same script may be in several different blocks
• Characters from different scripts may be in the same block

For more information, see Character Blocks in UTR #18: Unicode Regular Expression Guidelines [UTR18].

2 Usage Model

Although script names are generally much more useful than simple block names, they cannot be applied blindly. The script assignment is particularly oriented towards mechanisms such as regular expressions, and is not intended to be used for other purposes such as graphology, history, font choice, or other unrelated purposes. The definition of script names in the data file do not preclude the assignment of scripts in appropriate ways for these other purposes.

The script name data provides a mapping from each Unicode code point to either a specific script such as Cyrillic, or to one of two special values:

• Common - For characters that may be used within multiple scripts, or any unassigned code points.
• Inherited - For characters that inherit their script from the preceding characters.

The script names form a full partition of the code space: every codepoint is assigned a single script name. As new scripts are added to the standard, additional script names will be added.

In many cases, programs will override the script name based upon the context of the surrounding characters, especially for the case of Common. A simple heuristic is to use the script of the preceding character, which works well in many cases. However, this may not always produce optimal results: for example, in the text "... gamma (γ) is ..." this heuristic would cause matching parentheses to be in different scripts. Thus more sophisticated programs may use more complex heuristics.

In general, programs should only use specific script values in conjunction with both Common and Inherited. That is, to distinguish characters appropriate for Greek, one would use:

( ( Greek | Common ) Inherited ? ) *

That is, characters that are either in Greek or in Common, optionally followed by those in Inherited. Specific languages may commonly use multiple scripts, so for Japanese one would use:

( ( Hiragana | Katakana | Han | Latin | Common ) Inherited ? ) *

Given this usage model, the current data is weighted on inclusiveness: a character is in a specific script (rather than Common or Inherited) only when it is clearly not used within other scripts. As more data on individual characters is collected, characters may move from the Common group to a more specific script (including Inherited).

3 Values

The following table lists the Script Name values used in the data file. The names are not case-sensitive. Where possible, it also includes corresponding codes from the forthcoming ISO 15924: Code for the Representation of Names of Scripts [ISO15924]. The order in which the scripts are listed here or in the data file is not significant. 

Note: The forthcoming ISO 15924 provides an enumeration of four-letter script codes. In some cases the match between these script names and the ISO 15924 codes is not precise, since the goals are somewhat different. ISO 15924 is aimed primarily at the bibliographic identification of scripts; because of that it occasionally identifies varieties of scripts that may be useful for book cataloging, but which are not considered distinct as scripts in the Unicode Standard. For example, ISO 15924 has separate script codes for the Fraktur and Gaelic varieties of the Latin script. Where there are no corresponding ISO 15924 codes, the "private use" ones starting with Q are used. These are in italics in the table above.

4 Data File

The Scripts.txt data file is available at [Data]. The format of the file is similar to that of Blocks.txt [Blocks]. The fields are separated by semicolons. The first field contains either a single code point, or the first and last code points in a range separated by "..". The second field provides the script name for that range. The comment (after a #) indicates the general category, and the character name. For a range, it adds the count in square brackets and uses the names for the first and last characters in the range. For example:

0B01;       ORIYA # Mn ORIYA SIGN CANDRABINDU
0B02..0B03; ORIYA # Mc [2] ORIYA SIGN ANUSVARA..ORIYA SIGN VISARGA

The value COMMON is the default value, given to all code points that are not explicitly mentioned in the data file.

There is an additional set of Script Charts [Charts] that can be used to see the assignment of scripts. These charts show the entire range of Unicode characters broken down by script name (and general category where the script is Common or Inherited). If your browser is not set up for Unicode, see Display Problems.

References

Modifications

The following summarizes modifications from the previous version of this document.

Copyright © 1999-2001 Unicode, Inc. All Rights Reserved. The Unicode Consortium makes no expressed or implied warranty of any kind, and assumes no liability for errors or omissions. No liability is assumed for incidental and consequential damages in connection with or arising out of the use of the information or programs contained or accompanying this technical report.

Unicode and the Unicode logo are trademarks of Unicode, Inc., and are registered in some jurisdictions.