[Unicode]  Unicode Collation Algorithm
 

UCA Conformance Tests

Version 6.2.0
2012-09-12

The following files provide conformance tests for the Unicode Collation Algorithm (UTS #10: Unicode Collation Algorithm).

These files are large, and thus packaged in zip format to save download time.

Format

There are four different files:

The format is illustrated by the following example:

0385 0021;  # (΅) GREEK DIALYTIKA TONOS  [0316 015D | 0020 0032 0020 | 0002 0002 0002 |]

The part before the semicolon is the hex representation of a sequence of Unicode code points. After the hash mark is a comment. This comment is purely informational, and may change in the future. Currently it consists of the characters of the sequence in parentheses, the name of the first code point, and a representation of the  sort key for the sequence.

The sort key representation is in square brackets. It uses a vertical bar for the ZERO separator. Between the bars are the primary, secondary, tertiary, and quaternary weights (if any), in hex.

Note: The sort key is purely informational. UCA does not require the production of any particular sort key, as long as the results of comparisons match.

Testing

The files are designed so each line in the file will order as being greater than or equal to the previous one, when using the UCA and the Default Unicode Collation Element Table. A test program can read in each line, compare it to the last line, and signal an error if order is not correct. The exact comparison that should be used is as follows:

  1. Read the next line.
  2. Parse each sequence up to the semicolon, and convert it into a Unicode string.
  3. Compare that string with the string on the previous line, according to the UCA implementation, with strength = identical level (using S3.10).
  4. If the last string is greater than the current string, then stop with an error.
  5. Continue to the next line (step 1).

If there are any errors, then the UCA implementation is not compliant.

These files contain test cases that include ill-formed strings, with surrogate code points. Implementations that do not weight surrogate code points the same way as reserved code points may filter out such lines lines in the test cases, before testing for conformance.

Notes:
  • This test is only valid for an untailored DUCET table.
  • Beginning with UCA 6.2, the test data strings are compared with strength = identical, using UCA S3.10 as a tie-breaker, which compares the NFD forms of the strings in code point order. Before UCA 6.2, the test files did not use strength = identical, and instead used as a tie-breaker the comparison of the unnormalized strings.
    Therefore, implementations which use the UCA test files to test multiple versions of UCA need to use different tie-breaker comparisons depending on the UCA version.