L2/12-041
Title: Overriding Default Properties for PUA -- Core Specification Text Issues
Source: Ken Whistler and Editorial Committee
Date: January 27, 2012
Action: For consideration by the UTC
During the process of text editing and cleanup of the Core Specification for
Unicode 6.1,
the editorial committee ran across a potential inconsistency regarding the
details
of what the standard claims about overriding default properties for PUA
characters.
I am bringing the details of the text issues to the attention of the UTC, for
discussion
and decision.
Current Text for Unicode 6.1
In the latest current draft of the Core Specification for Unicode 6.1, the
relevant
text in Chapter 3, Section 3.5 states (on p. 72):
Default property values are also provided for private-use characters.
Because the
interpretation of private-use characters is subject to private agreement
between the
parties which exchange them, the default property values for those
characters
are overridable by higher-level protocols, to match the agreed-upon
semantics
for the characters. See Section 16.5, Private-Use Characters.
[Note that this is new text added to the draft of Unicode 6.1, in an attempt to
clarify the issue of default property values and PUA. So this isn't already
published text from Unicode 6.0.]
The relevant text in the latest current draft of the Core Specification for
Unicode 6.1,
Section 16.5 states (on p. 555):
The General_Category value of private-use characters in the Unicode Standard
is Private_Use (gc=Co). This value is normatively defined and cannot be
changed
by private agreement. This means that no private agreement can change which
character codes are reserved for private use. However, many Unicode
algorithms
use character properties which are derived by reference to the
General_Category
property. Private agreements may override such derivations for private-use
characters, except where overriding is expressly disallowed in the
conformance
statement for a specific algorithm. In other words, private agreements may
define
which private-use characters should be treated like spaces, digits, letters,
punctuation, and so on, by all parties to those private agreements.
For all properties other than General_Category and the normalization-related
properties, the Unicode Character Database provides default values
for private-use characters. These default property values should be
considered
informative...
[That text is also proposed text new for Unicode 6.1. It is not already
published text
from Unicode 6.0, which only has a very short statement about default property
values for PUA characters.]
Text Issues
The basic text issue raised in the editorial committee is that the draft text in
Section 3.5 is not entirely consistent with the more extensive statement in
Section 16.5. The issue is for General_Category, in particular.
A secondary issue was also raised about the text in Section 16.5: The claim
is that the sentence, "This value is normatively defined and cannot be changed
by private agreement." confuses normativity with overridability. Personally, I
disagree with that assessment, but see how the sentence might be read that
way, so concur that it could use an editorial improvement.
Because the basic text issue concerns Chapter 3, Conformance text, because
in principle, the UTC has already reviewed and agreed upon the text of Chapter 3
for Unicode 6.1, and because the issue is inherently a little tricky, the
editorial
committee deemed it advisable to bring the issue to the attention of the UTC
for discussion and resolution.
Suggested Text Changes
To make the discussion a little easier, I'll provide textual emendation
suggestions
here, which I think may address the problems noted.
First, for Section 3.5, the main issue is that General_Category default values
are not all overridable by private agreement, and there are important
caveats spelled out in more detail in Section 16.5. My suggested emendation
of the text, then, would be:
Default property values are also provided for private-use characters.
Because the
interpretation of private-use characters is subject to private agreement
between the
parties which exchange them, the most default
property values for those characters
are overridable by higher-level protocols, to match the agreed-upon
semantics
for the characters. There are important exceptions for the
General_Category
and a few other properties. See Section 16.5, Private-Use Characters.
Then a textual correction for the objectionable sentence in Section 16.5 could
be:
This value is normatively defined and ; it also
cannot be changed
by private agreement.
Questions
Do those minimal text changes correctly reflect the intention of the UTC
regarding this issue
of overriding default property values for PUA characters?
If so, should the editorial committee proceed with those text changes for the
Unicode 6.1
Core Specification text?
Are there other textual suggestions for how to solve these issues in a different
way?