[RFCs/IDs] [Plain Text] [WG] [Email] [Diff1] [Diff2] [Nits]
Versions: (draft-alvestrand-idna-bidi) 00 01
02 03
L2/09-046
Network Working Group H. Alvestrand, Ed.
Internet-Draft Google
Intended status: Standards Track C. Karp
Expires: June 3, 2009 Swedish Museum of Natural History
November 30, 2008
An updated IDNA criterion for right-to-left scripts
draft-ietf-idnabis-bidi-03
Status of this Memo
By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on June 3, 2009.
Abstract
The use of right-to-left scripts in internationalized domain names
has presented several challenges. This memo discusses some problems
with these scripts, and some shortcomings in the 2003 IDNA BIDI
criterion. Based on this discussion, it proposes a new BIDI rule for
IDNA labels.
Alvestrand & Karp Expires June 3, 2009 [Page 1]
Internet-Draft IDNA RTL fix November 2008
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Purpose and applicability . . . . . . . . . . . . . . . . 3
1.2. Background and history . . . . . . . . . . . . . . . . . . 3
1.3. Layout of the rest of this document . . . . . . . . . . . 4
1.4. Terminology . . . . . . . . . . . . . . . . . . . . . . . 4
2. A replacement for the RFC 3454 BIDI rule . . . . . . . . . . . 6
3. A requirement set for the BIDI rule . . . . . . . . . . . . . 7
4. Examples of issues found . . . . . . . . . . . . . . . . . . . 10
4.1. Dhivehi . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.2. Yiddish . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.3. Strings with numbers . . . . . . . . . . . . . . . . . . . 12
5. Troublesome situations and guidelines . . . . . . . . . . . . 12
6. Other issues in need of resolution . . . . . . . . . . . . . . 13
7. Compatibility considerations . . . . . . . . . . . . . . . . . 14
7.1. Backwards compatibility considerations . . . . . . . . . . 14
7.2. Forward compatibility considerations . . . . . . . . . . . 15
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15
9. Security Considerations . . . . . . . . . . . . . . . . . . . 15
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 16
11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 16
11.1. Normative references . . . . . . . . . . . . . . . . . . . 16
11.2. Informative references . . . . . . . . . . . . . . . . . . 16
Appendix A. Change log . . . . . . . . . . . . . . . . . . . . . 17
A.1. Changes from draft-alvestrand-00 to -01 . . . . . . . . . 17
A.2. Changes from alvestrand-01 to -02 . . . . . . . . . . . . 17
A.3. Changes from alvestrand-02 to -03 . . . . . . . . . . . . 17
A.4. Changes from alvestrand-03 to -04 . . . . . . . . . . . . 17
A.5. Changes from draft-alvestrand-04 to draft-ietf -00 . . . . 18
A.6. Changes from idnabis -00 to -01 . . . . . . . . . . . . . 18
A.7. Changes from idnabis -01 to -02 . . . . . . . . . . . . . 18
A.8. Changes from idnabis -02 to -03 . . . . . . . . . . . . . 19
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 19
Intellectual Property and Copyright Statements . . . . . . . . . . 20
Alvestrand & Karp Expires June 3, 2009 [Page 2]
Internet-Draft IDNA RTL fix November 2008
1. Introduction
1.1. Purpose and applicability
This document's purpose is to establish a rule that can be applied to
Internationalized Domain Name (IDN) labels in Unicode form (U-labels)
containing right-to-left characters.
When labels satisfy the rule, and when certain other conditions are
satisfied, they can be used with a minimal chance of these labels
being displayed in a confusing way by a bidirectional display
algorithm.
This specification is not intended to place any requirements on
domain names that do not contain right-to-left characters.
1.2. Background and history
The IDNA specification "Stringprep" [RFC3454] makes the following
statement in its section 6 on the BIDI algorithm:
3) If a string contains any RandALCat character, a RandALCat
character MUST be the first character of the string, and a
RandALCat character MUST be the last character of the string.
(A RandALCat character is a character with unambiguously right-to-
left directionality.)
The reasoning behind this prohibition was to ensure that every
component of a displayed domain name has an unambiguously preferred
direction. However, this makes certain words in languages written
with right-to-left scripts invalid as IDN labels, and in at least one
case means that all the words of an entire language are forbidden as
IDN labels.
This is illustrated below with examples taken from the Dhivehi and
Yiddish languages, as written with the Thaana and Hebrew scripts,
respectively.
In investigating this problem, it was realized that the RFC 3454
specification did not explicitly state the requirement to be
fulfilled, and therefore, it was impossible to tell whether a simple
relaxation of the rule would continue to fulfil the requirement. A
further investigation led to the conclusion that for one reasonable
set of requirements, IDNA2003's BIDI restriction did not fulfil the
requirements. This document therefore proposes replacing the RFC
3454 BIDI requirement in its entirety.
Alvestrand & Karp Expires June 3, 2009 [Page 3]
Internet-Draft IDNA RTL fix November 2008
While the document proposes completely new text, most reasonable
labels that were allowed under the old criterion will also be allowed
under the new criterion, so the operational impact of the rule change
is limited.
1.3. Layout of the rest of this document
Section 2 defines a test, the "BIDI test", that can be used on a
domain name label to check how safe it is to use in a domain name of
possibly mixed directionality.- one user of that test is the IDNA2008
protocol[I-D.ietf-idnabis-protocol].
Section 3 sets out the requirements for defining a BIDI rule.
Section 4 gives detailed examples that serve as justification for the
change proposed here.
Section 5 to Section 9 describe various situations that can occur
when dealing with domain names with characters of different
directionality.
Only Section 1.4 and Section 2 are normative.
1.4. Terminology
In this memo, we use "network order" to describe the sequence of
characters as transmitted on the wire or stored in a file; the terms
"first", "next", "previous", "before" and "after" are used to refer
to the relationship of characters and labels in network order.
We use "display order" to talk about the sequence of characters as
imaged on a display medium; the terms "left" and "right" are used to
refer to the relationship of characters and labels in display order.
Most of the time, the examples use the abbreviations for the Unicode
BIDI classes to denote the directionality of the characters; in some
examples, the convention that uppercase characters are of class R or
AL, and lowercase characters are of class L is used - thus, the
example string ABC.abc would consist of 3 right-to-left characters
and 3 left-to-right characters.
The directionality of such examples is determined by context - for
instance, in the sentence "ABC.abc is displayed as CBA.abc", the
first example string is in network order, the second example string
is in display order.
The term "paragraph" is used in the sense of the Unicode BIDI
specification [UAX9] - it means "a block of text that has an overall
Alvestrand & Karp Expires June 3, 2009 [Page 4]
Internet-Draft IDNA RTL fix November 2008
direction, either left-to-right or right-to-left", approximately; see
UAX 9 for the details.
"LTR" and "RTL" are abbreviations for "right to left" and "left to
right", respectively.
The terminology used for the BIDI properties of Unicode characters is
taken from the Unicode Standard[Unicode]
For reference, here are the values that the Unicode BIDI property can
have:
o L - Left-to-right - most letters in LTR scripts
o R - Right-to-left - most letters in non-Arabic RTL scripts
o AL - Arabic letters - most letters in the Arabic script
o EN - European Number (0-9, and Extended Arabic-Indic numbers)
o ES - European Number Separator (+ and -)
o ET - European Number Terminator (currency symbols, the hash sign,
the percent sign and so on)
o AN - Arabic Number
o CS - Common Number Separator (. , / : et al)
o NSM - Non spacing Mark - most combining accents
o BN - Boundary Neutral - control characters
o B - Paragraph Separator
o S - Segment Separator
o WS - Whitespace, including the SPACE character
o ON - Other Neutrals, including @, &, parentheses, MIDDLE DOT
o LRE, LRO, RLE, RLO, PDF - these are "directional control
characters", and are not used in IDNA labels.
The other terminology used to describe IDNA concepts is defined in
[I-D.ietf-idnabis-defs]
Alvestrand & Karp Expires June 3, 2009 [Page 5]
Internet-Draft IDNA RTL fix November 2008
2. A replacement for the RFC 3454 BIDI rule
The following test has been developed for BIDI domain name labels.
The requirements that this test satisifes are described in Section 3.
A label containing a character of type R, AL or AN MUST satisfy all 7
of the rules below.
The main bullets give the rule, subordinate bullets (if any) give
justifications or examples of things that break if this rule is not
present.
1. Only characters with the BIDI properties L, R, AL, AN, EN, ES,
BN, ON and NSM are allowed.
* B, S and WS are excluded because they are separators or
spaces.
* LRE, LRO, RLE, RLO, PDF are excluded because they are BIDI
controls.
* ET is excluded because the string L ET does not satisfy the
Character Grouping requirement.
* CS is excluded because the string L CS does not satisfy the
Character Grouping requirement.
2. ES and ON are not allowed in the first position
* ES R and ON R do not satisfy the Character Grouping
requirement.
3. ES and ON, followed by zero or more NSM, is not allowed in the
last position
* L ON and L ES both fail the Character Grouping requirement.
4. If an R, AL or AN is present, no L may be present.
5. If an EN is present, no AN may be present, and vice versa.
6. The first character may not be an NSM
7. The first character may not be an EN (European Number) or an AN
(Arabic Number).
* If the character on both sides of a CS is an EN or an AN, the
labels fail the Character Grouping requirement.
Alvestrand & Karp Expires June 3, 2009 [Page 6]
Internet-Draft IDNA RTL fix November 2008
* Some domain names where some of the labels use leading EN and
AN may be problem-free, but there's no way of verifying this
while looking at a single label in isolation..
* We could achieve stability by barring numbers at the end of
labels, but this may be more disruptive in practice.
The following guarantees can be made based on the above:
o In a domain name consisting of only labels that pass the test, the
requirements of Section 3 are satisfied.
o In a domain name consisting of only LDH-labels and labels that
pass the test, the requirements of Section 3 are satisfied as long
as a label that starts with an ASCII digit does not come after a
right-to-left label that ends in a digit.
No guarantee is given for other combinations.
3. A requirement set for the BIDI rule
One issue with RFC 3454 was that it did not give an explicit
justification for the BIDI rule, thus it was hard to tell if a
modified rule would continue to fulfil the purpose for which the RFC
3454 rule was written.
This document proposes an explicit justification, by stating a set of
requirements for which it is possible to test whether or not the
modified rule fulfils the requirement.
All the text in this document assumes that text containing the labels
under consideration will be displayed using the Unicode bidirectional
algorithm [UAX9].
The requirements proposed are these:
o Label Uniqueness: No two labels, when presented in display order
in the same paragraph, should have the same sequence of characters
without also having the same sequence of characters in network
order, both when the paragraph has LTR direction and when the
paragraph has RTL direction. (This is the criterion that is
explicit in RFC 3454). (Note that a label displayed in an RTL
paragraph may display the same as a different label displayed in a
LTR paragraph, and still satisfy this criterion.)
o Character Grouping: When displaying a string of labels, using the
Unicode BIDI algorithm to reorder the characters for display, the
Alvestrand & Karp Expires June 3, 2009 [Page 7]
Internet-Draft IDNA RTL fix November 2008
characters of each label should remain grouped between the
characters delimiting the labels, both when the string is embedded
in a paragraph with LTR direction and when it's embedded in a
paragraph with RTL direction.
Several stronger statements were considered and rejected, because
they seem to be impossible to fulfil within the constraints of the
Unicode bidirectional algorithm. These include:
o The appearance of a label should be unaffected by its embedding
context. This proved impossible even for ASCII labels; the label
"123-456" will have a different display order in an RTL context
than in a LTR context.
o The sequence of labels should be consistent with network order.
This proved impossible - a domain name consisting of the labels
(in network order) L1.R1.R2.L2 will be displayed as L1.R2.R1.L2 in
an LTR context. (In a RTL context, it will be displayed as
L2.R2.R1.L1).
o The Label Uniqueness property should hold true between LTR
paragraphs and RTL paragraphs. This was shown to be unsound.
o No two domain names should be displayed the same, even under
differing directionality. This was shown to be unsound, since the
domain name (network) ABC.abc will have display order CBA.abc in
an LTR context and abc.CBA in an RTL context, while the domain
name (network) abc.ABC will have display order abc.CBA in an LTR
context and CBA.abc in an RTL context.
One specific requirement was thought to be problematic, but turned
out to be satisfied by a string that obeys the proposed rules:
o The Character Grouping requirement should be satisfied when
directional controls (LRE, RLE, RLO, LRO, PDF) are used in the
same paragraph (outside of the labels). Because these controls
affect presentation order in non-obvious ways, by affecting the
"sor" and "eor" properties of the Unicode BIDI algorithm, the
conditions above require extra testing in order to figure out
whether or not they influence the display of the domain name.
Testing found that for the strings allowed under the rule
presented in this document, directional controls do not influence
the display of the domain name.
In the following descriptions, first-level bullets are used to
indicate rules or normative statements; second-level bullets are
commentary.
Alvestrand & Karp Expires June 3, 2009 [Page 8]
Internet-Draft IDNA RTL fix November 2008
The Character Grouping requirement can be more formally stated as:
o Let "Delimiterchars" be a set of characters with the Unicode BIDI
properties CS, WS, ON. (These are commonly used to delimit labels
- both the FULL STOP and the space are included.)
* ET, though it commonly occurs next to domain names in practice,
is problematic: the context R CS L EN ET (for instance A.a1%)
makes the label L EN not satisfy the character grouping
requirement.
* ES commonly occurs in labels as HYPHEN-MINUS, but could also be
used as a delimiter (for instance, the plus sign). It is left
out here.
o Let "unproblematic label" be a label that either satisfies the
requirements, or does not contain any character with the BIDI
properties R, AL or AN, and does not begin with a character with
the BIDI property EN. (Informally, "it does not start with a
number".)
A label L satisfies the Character Grouping requirement when, for any
Delimiter Character D1 and D2, and for any label S1 and S2 that is
either a label satisfying the requirements or an unproblematic label,
the following holds true:
If the string formed by concatenating S1, D1, L, D2 and S2 is
reordered according to the BIDI algorithm, then all the characters of
L in the reordered string are between D1 and D2, and no other
characters are between D1 and D2, both if the overall paragraph
direction is LTR and if the overall paragraph direction is RTL.
Note that the definition is self-referential, since S1 and S2 are
constrained to be "legal" by this definition; this makes testing
changes to proposed rules a little complex, but does not create
problems for testing whether or not a given proposed rule satisfies
the criterion.
(The "zero-length" case represents the case where a domain name is
next to something that isn't a domain name, separated by a delimiter
character).
The Label Uniqueness requirement can be formally stated as:
If two labels L and L', embedded as for the test above, displayed in
paragraphs with the same directionality, are reordered by the BIDI
algorithm into the same sequence of codepoints, at most one of the
labels L and L' is a legal label.
Alvestrand & Karp Expires June 3, 2009 [Page 9]
Internet-Draft IDNA RTL fix November 2008
4. Examples of issues found
4.1. Dhivehi
Dhivehi, the official language of the Maldives, is written with the
Thaana script. This displays some of the characteristics of Arabic
script, including its directional properties, and the indication of
vowels by the diacritical marking of consonantal base characters.
This marking is obligatory, and both double vowels and syllable-final
consonants are indicated by the marking of special unvoiced
characters. Every Dhivehi word therefore ends with a combining mark.
The word for "computer", which is romanized as "konpeetaru", is
written with the following sequence of Unicode code points:
U+0786 THAANA LETTER KAAFU (AL)
U+07AE THAANA OBOFILI (NSM)
U+0782 THAANA LETTER NOONU (AL)
U+07B0 THAANA SUKUN (NSM)
U+0795 THAANA LETTER PAVIYANI (AL)
U+07A9 THAANA LETTER EEBEEFILI (AL)
U+0793 THAANA LETTER TAVIYANI (AL)
U+07A6 THAANA ABAFILI (NSM)
U+0783 THAANA LETTER RAA (AL)
U+07AA THAANA UBIUFILI (NSM)
The directionality class of U+07AA in the Unicode database [Unicode]
is NSM (non-spacing mark), which is not R or AL; a conformant
implementation of the IDNA2003 algorithm will say that "this is not
in RandALCat", and refuse to encode the string.
4.2. Yiddish
Yiddish is one of several languages written with the Hebrew script
(others include Hebrew and Ladino). This is basically a consonantal
alphabet (also termed an "abjad") but Yiddish is written using an
extended form that is fully vocalic. The vowels are indicated in
several ways, of which one is by repurposing letters that are
consonants in Hebrew. Other letters are used both as vowels and
Alvestrand & Karp Expires June 3, 2009 [Page 10]
Internet-Draft IDNA RTL fix November 2008
consonants, with combining marks, called "points", used to
differentiate between them. Finally, some base characters can
indicate several different vowels, which are also disambiguated by
combining marks. Pointed characters can appear in word-final
position and may therefore also be needed at the end of labels. This
is not an invariable attribute of a Yiddish string and there is thus
greater latitude here than there is with Dhivehi.
The organization now known as the "YIVO Institute for Jewish
Research" developed orthographic rules for modern Standard Yiddish
during the 1930s on the basis of work conducted in several venues
since earlier in that century. These are given in, "The Standardized
Yiddish Orthography: Rules of Yiddish Spelling" [SYO], and are taken
as normatively descriptive of modern Standard Yiddish in any context
where that notion is deemed relevant. They have been applied
exclusively in all Yiddish dictionaries published since their
establishment, and are similarly dominant in academic and
bibliographic regards.
It therefore appears appropriate for this repertoire also to be
supported fully by IDNA. This presents no difficulty with characters
in initial and medial positions, but pointed characters are regularly
used in final position as well. All of the characters in the SYO
repertoire appear in both marked and unmarked form with one
exception: the HEBREW LETTER PE (U+05E4). The SYO only permits this
with a HEBREW POINT DAGESH (U+05BC), providing the Yiddish equivalent
to the Latin letter "p", or a HEBREW POINT RAFE (U+05BF), equivalent
to the Latin letter "f". There is, however, a separate unpointed
allograph, the HEBREW LETTER FINAL PE (U+05E3), for the latter
character when it appears in final position. The constraint on the
use of the SYO repertoire resulting from the proscription of
combining marks at the end of RTL strings thus reduces to nothing
more, or less, than the equivalent of saying that a string of Latin
characters cannot end with the letter "p". It must also be noted
that the HEBREW LETTER PE with HEBREW POINT DAGESH is characteristic
of almost all traditional Yiddish orthographies that predate (or
remain in use in parallel to) the SYO, being the first pointed
character to appear in any of them.
A more general instantiation of the basic problem can be seen in the
representation of the YIVO acronym. This is written with the Hebrew
letters YOD YOD HIRIQ VAV VAV ALEF QAMATS, where HIRIQ and QAMATS are
combining points:
U+05D9 HEBREW LETTER YOD (R)
U+05B4 HEBREW POINT HIRIQ (NSM)
Alvestrand & Karp Expires June 3, 2009 [Page 11]
Internet-Draft IDNA RTL fix November 2008
U+05D5 HEBREW LETTER VAV (R)
U+05D0 HEBREW LETTER ALEF (R)
U+05B8 HEBREW POINT QAMATS (NSM)
The directionality class of U+05B8 HEBREW POINT QAMATS in the Unicode
database is NSM, which again causes the IDNA2003 algorithm to reject
the string.
It may also be noted that all of the combined characters mentioned
above exist in precomposed form at separate positions in the Unicode
chart. However, by invoking Stringprep, the IDNA2003 algorithm also
rejects those codepoints, for reasons not discussed here.
4.3. Strings with numbers
By requiring that the first or last character of a string be category
R or AL, RFC 3454 prohibited a string containing right-to-left
characters from ending with a number.
Consider the strings ALEF 5 (HEBREW LETTER ALEF + DIGIT FIVE) and 5
ALEF. Displayed in a LTR context, the first one will be displayed
from left to right as 5 ALEF (with the 5 being considered right-to-
left because of the leading ALEF), while 5 ALEF will be displayed in
exactly the same order (5 taking the direction from context).
Clearly, only one of those should be permitted as a registered label,
but barring them both seems to require justification.
5. Troublesome situations and guidelines
There are situations in which labels that satisfy the rule above will
be displayed in a surprising fashion; the most important of these is
the case where a label ending in a character with BIDI property AL,
AN or R occurs before a label beginning with a character of BIDI
property EN. In that case, the number will appear to move into the
label containing the right-to-left character, violating the Character
Grouping requirement.
If the label that occurs after the right-to-left label itself
satisfies the BIDI criterion, the requirements will be satisfied in
all cases (this is the reason why the criterion talks about strings
containing L in some cases). However, the WG concluded that this
could not be required for several reasons:
Alvestrand & Karp Expires June 3, 2009 [Page 12]
Internet-Draft IDNA RTL fix November 2008
o There is a large current deployment of ASCII domain names starting
with digits. These cannot possibly be invalidated.
o Domain names are often constructed piecemeal, for instance by
combining a string with the content of a search list. This may
occur after IDNA processing, and thus in part of the code that is
not IDNA-aware, making detection of the undesirable combination
impossible.
o Even if a label is registered under a "safe" label, there may be a
DNAME [RFC2672]with an "unsafe" label that points to the "safe"
label, thus creating seemingly-valid names that would not satisfy
the criterion.
o Wildcards create the odd situation where a label is "valid" (can
be looked up successfully) without the zone owner's knowing that
this label exists. So an owner of a zone whose name starts with a
digit and contains a wildcard has no way of controlling whether or
not names with RTL labels in them are looked up in his zone.
So rather than trying to suggest rules that disallow all such
undesirable situations, this document merely warns about the
possibility.
6. Other issues in need of resolution
This document concerns itself only with the rules that are needed
when dealing with domain names with characters that have differing
BIDI properties, and considers characters only in terms of their BIDI
properties. All other issues with these scripts have to be
considered in other contexts.
One such issue is the need to keep numbers separate; several scripts,
such as Arabic, have multiple sets of numbers. The algorithm in this
document disallows occurrences of AN-class characters ("Arabic-Indic
digits", U+0660 to U+0669) together with EN-class characters (which
includes "European" digits, U+0030 to U+0039 and "extended Arabic-
Indic digits", U+06F0 to U+06F9), but does not help in preventing the
mixing of, for instance, Bengali digits (U+09E6 to U+09EF) and
Gujarati digits (U+0AE6 to U+0AEF), both of which have BIDI class L.
A registry or script community that wishes to create rules for the
mixing of digits in a label will be able to specify these
restrictions at the registry level; rules can also be specified at
the protocol level, but while the example above involves right-to-
left characters, this is not inherently a bidi problem.
Another set of issues concerns the proper display of IDNs with a
Alvestrand & Karp Expires June 3, 2009 [Page 13]
Internet-Draft IDNA RTL fix November 2008
mixture of LTR and RTL labels, or only RTL labels.
It is unrealistic to expect that applications will display domain
names using embedded formatting codes between their labels (for one
thing, no reliable algorithms for identifying domain names in running
text exist); thus, the display order will be determined by the
bidirectional algorithm. Thus, a sequence (in network order) of
R1.R2.ltr will be displayed in the order 2R.1R.ltr in an LTR context,
which might surprise someone expecting to see labels displayed in
hierarchical order. Again, this memo does not attempt to suggest a
solution to this problem.
7. Compatibility considerations
7.1. Backwards compatibility considerations
As with any change to an existing standard, it is important to
consider what happens with existing implementations when the change
is introduced. The following troublesome cases have been noted:
o Old program used to input the newly-allowed string. If the old
program checks the input against RFC 3454, the string will not be
allowed, and that domain name will remain inaccessible.
o Old program is asked to display the newly-allowed string, and
checks it against RFC 3454 before displaying. The program will
perform some kind of fallback, most likely displaying the string
in A-label form.
o Old program tries to display the newly-allowed string. If the old
program has code for displaying the last character of a string
that is different from the code used to display the characters in
the middle of the string, display may be inconsistent and cause
confusion.
One particular example of the last case is if a program chooses to
examine the last character (in network order) of a string in order to
determine its directionality, rather than its first; if it finds an
NSM character and tries to display the string as if it was a left-to-
right string, the resulting display may be interesting, but not
useful.
The editors believe that these cases will have less harmful impact in
practice than continuing to deny the use of words from the languages
for which these strings are necessary as IDN labels.
This specification does not forbid using leading European numbers in
Alvestrand & Karp Expires June 3, 2009 [Page 14]
Internet-Draft IDNA RTL fix November 2008
ASCII-only labels, since this would in conflict with a large
installed base of such labels, and would increase the scope of the
specification from RTL labels to all labels. The harm resulting from
this limitation of scope is described in Section 5. Zone managers,
both registries and private zone managers, can check for this
particular condition before they allow registration of any string
with right-to-left characters in it; generally it is best to not
allow registration of any right-to-left strings in a zone where the
label at the level above begins with a digit.
7.2. Forward compatibility considerations
This text is, intentionally, specified strictly in terms of the
Unicode BIDI properties. The determination that the condition is
sufficient to fulfil the criteria depends on the Unicode BIDI
algorithm; it is unlikely that drastic changes will be made to this
algorithm.
However, the determination of validity for any string depends on the
Unicode BIDI property values, which are not declared immutable by the
Unicode Consortium. Furthermore, the behaviour of the algorithm for
any given character is likely to be linguistically and culturally
sensitive, so that it's not unlikely that later versions of the
Unicode standard may change the BIDI properties assigned to certain
Unicode characters.
This memo does not propose a solution for this problem.
8. IANA Considerations
This document makes no request of IANA.
Note to RFC Editor: this section may be removed on publication as an
RFC.
9. Security Considerations
This modification will allow some strings to be used in Stringprep
contexts that are not allowed today. It is possible that differences
in the interpretation of the specification between old and new
implementations could pose a security risk, but it is difficult to
envision any specific instantiation of this.
Any rational attempt to compute, for instance, a hash over an
identifier processed by Stringprep would use network order for its
computation, and thus be unaffected by the changes proposed here.
Alvestrand & Karp Expires June 3, 2009 [Page 15]
Internet-Draft IDNA RTL fix November 2008
While it is not believed to pose a problem, if display routines had
been written with specific knowledge of the RFC 3454 Stringprep
prohibitions, it is possible that the potential problems noted under
"backwards compatibility" could cause new kinds of confusion.
10. Acknowledgements
While the listed editors held the pen, this document represents the
joint work and conclusions of an ad hoc design team. In addition to
the editors this consisted of, in alphabetic order, Tina Dam, Patrik
Faltstrom, and John Klensin. Many further specific contributions and
helpful comments were received from the people listed below, and
others who have contributed to the development and use of the IDNA
protocols.
The team wishes in particular to thank Roozbeh Pournader for calling
its attention to the issue with the Thaana script, Paul Hoffmann for
pointing out the need to be explicit about backwards compatibility
considerations, Ken Whistler for suggesting the basis of the
formalized "character grouping" requirement, Mark Davis for
commentary, Erik van der Poel for careful review, comments and
verification of the rulesets, and Marcos Sanz, Andrew Sullivan and
Pete Resnick for reviews.
11. References
11.1. Normative references
[I-D.ietf-idnabis-defs]
Klensin, J., "Internationalized Domain Names for
Applications (IDNA): Definitions and Document Framework",
draft-ietf-idnabis-defs-01 (work in progress),
November 2008.
[UAX9] Davis, M., "Unicode Standard Annex #9: The Bidirectional
Algorithm, revision 15", 03 2005.
[Unicode] Unicode, "The Unicode Standard - version 5.1", 2008.
11.2. Informative references
[I-D.ietf-idnabis-protocol]
Klensin, J., "Internationalized Domain Names in
Applications (IDNA): Protocol",
draft-ietf-idnabis-protocol-06 (work in progress),
November 2008.
Alvestrand & Karp Expires June 3, 2009 [Page 16]
Internet-Draft IDNA RTL fix November 2008
[RFC2672] Crawford, M., "Non-Terminal DNS Name Redirection",
RFC 2672, August 1999.
[RFC3454] Hoffman, P. and M. Blanchet, "Preparation of
Internationalized Strings ("stringprep")", RFC 3454,
December 2002.
[SYO] "The Standardized Yiddish Orthography: Rules of Yiddish
Spelling, 6th ed., , New York, ISBN 0-914512-25-0",",
1999.
Appendix A. Change log
This appendix is intended to be removed by the RFC Editor when this
document is published as an RFC.
A.1. Changes from draft-alvestrand-00 to -01
Suggested a possible new algorithm.
Multiple smaller changes.
A.2. Changes from alvestrand-01 to -02
Date of publication updated.
Change log added.
A.3. Changes from alvestrand-02 to -03
Intro changed to reflect addressing the deeper issues with the BIDI
algorithm.
Gave formalized criteria for "valid strings", and documented the new
set of requirements for strings that satisfy the criteria.
Removed most of section 5, "Other problems", and noted that this memo
focuses ONLY on issues that can be evaluated by looking at the BIDI
properties of characters.
A.4. Changes from alvestrand-03 to -04
Added back AN to the list of allowed characters; it had been left out
by accident in -03.
Removed some rules that were redundant.
Alvestrand & Karp Expires June 3, 2009 [Page 17]
Internet-Draft IDNA RTL fix November 2008
Added some considerations for backwards compatibility and interaction
with ASCII labels that start with a number.
Mentioned the issue with DNAME pointing to a zone containing RTL
labels in the security considerations section.
Wording updates in multiple places, including some spelling errors.
Rewrote the introduction section.
Split references into "normative" and "informative".
A.5. Changes from draft-alvestrand-04 to draft-ietf -00
Changed name of draft.
Added a couple of "note in draft" statements to remind the WG of open
issues.
Noted that BIDI controls in the paragraph are unproblematic with the
given ruleset.
A.6. Changes from idnabis -00 to -01
Added text to section 5 describing issues with mixture of numbers in
labels
Addressed some of the issues raised by Mark Davis in March 2008 in
regard to document clarity.
Changed the formulation of the label uniqueness requirement to be
consistent with the text under "Labels with numbers".
Spell-checked document.
A.7. Changes from idnabis -01 to -02
Changed the domain of applicability to be only labels containing RTL
characters, described the conditions under which harm may result from
putting RTL labels next to other labels, and how to detect them.
A number of clarification and formatting changes in response to
reviews.
Alvestrand & Karp Expires June 3, 2009 [Page 18]
Internet-Draft IDNA RTL fix November 2008
A.8. Changes from idnabis -02 to -03
Rearranged section list so that the normative material is collected
at the front.
Moved list of BIDI properties into "terminology"
Clarified that only terminology and the BIDI test is normative
Changed reference to point to -defs for definitions instead of
-rationale
Minor fixes in response to comments, wording cleanups, removed all
tentative language.
Authors' Addresses
Harald Tveit Alvestrand (editor)
Google
Beddingen 10
Trondheim, 7014
Norway
Email: harald@alvestrand.no
Cary Karp
Swedish Museum of Natural History
Frescativ. 40
Stockholm, 10405
Sweden
Phone: +46 8 5195 4055
Fax:
Email: ck@nrm.museum
URI:
Alvestrand & Karp Expires June 3, 2009 [Page 19]
Internet-Draft IDNA RTL fix November 2008
Full Copyright Statement
Copyright (C) The IETF Trust (2008).
This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Intellectual Property
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
Alvestrand & Karp Expires June 3, 2009 [Page 20]
Html markup produced by rfcmarkup 1.73, available from
http://tools.ietf.org/tools/rfcmarkup/