L2/01-191R
Dotting the i�s
Kent Karlsson and Vladas Tumasonis
2001-05-05
This is a proposal to update the SpecialCasing.txt data file in the Unicode Character Database. The current handling of dots above for lowercase i�s and j�s in SpecialCasing.txt for case mapping is not sufficient, in particular for Lithuanian where an explicit dot above sometimes needs to be introduced. This proposal also attempts a somewhat more systematic treatment of dots above lowercase i�s and j�s for other languages too.
The dot above lowercase i and lowercase j are 'soft' in the sense that they usually disappear upon uppercasing as well as upon given accents above the i or j. There are, however exceptions to this.� For these exceptions, where the dot is not 'soft', a 'hard dot above' (U+0307) is the best way to deal with this matter.� For Turkish, the soft dot must be �hardened� for uppercasing (when there are no accents above, otherwise the soft dot is already gone), but for Lithuanian it must be �hardened� before accenting above, but not for uppercasing.
The tables in the exposition are not complete.� The formal table in the update to SpecialCasing.txt are, however, intended to be complete.
to upper and to title
Normal
����������� Any lowercase variant of i or j with an unblocked extra dot above, if there are no more accents above on that base letter: remove the extra dot, then uppercase. This removes any spurious dot above, a dot that is not recommended to be there in the first place.
|
|
i+dot (no more accents above) |
I |
|
|
i-ogonek+dot (no more accents above) |
I-ogonek [etc.] |
|
|
j+dot (no more accents above) |
J |
Lithuanian
����������� Any lowercase variant of i or j with an unblocked extra dot above, even if there are more accents above on that base letter: remove the extra dot, then uppercase.
|
|
i+dot |
I |
|
|
j+dot |
J |
Turkish
����������� An i with an unblocked extra dot above, if there are no more accents above on that base letter: keep the extra dot, but don�t add another one (for the cases below), then uppercase. This, again, takes care of the spurious case where
|
|
i (no more accents above) |
I-dot |
|
|
i+dot (no more accents above) |
I-dot |
to lower
Normal
����������� Any lowercase or uppercase variant of i or j with an unblocked extra dot above, if there are no more accents above on that base letter: remove the extra dot.
|
|
i+dot (no more accents above) |
i |
|
|
|
i-ogonek+dot (no more accents above) |
i-ogonek |
|
|
|
... |
... |
|
|
|
j+dot (no more accents above) |
j |
|
|
|
I-dot (if more accents above) |
i -dot |
|
|
|
I-dot (if no more accents above) |
i (already in UniData.txt) |
|
|
|
I -dot (if more accents above) |
i -dot (for NFD�NFC consistency; already in UniData) |
|
|
|
I -dot (if no more accents above) |
i (for NFD�NFC consistency) |
|
|
|
J -dot (if no more accents above) |
j (some degree of systematic...) |
|
Lithuanian
����������� Any lowercase variant of i or j with an unblocked extra dot above, if there are no more accents above on that base letter: remove the extra dot. Uppercase I�s and J�s that have extra accents above must get an extra dot above inserted.
|
|
I (if more accents above) |
i -dot |
|
|
J (if more accents above) |
j -dot |
|
|
I-ogonek (if more accents above) |
i-ogonek -dot |
|
|
I-grave |
i -dot -grave |
|
|
I-acute |
i -dot -acute |
|
|
I-tilde |
i -dot -tilde |
For NFD�NFC consistency a number of �I-letters� that are not used in Lithuanian must be handled too.
Turkish
����������� Any lowercase variant of i or j with an unblocked extra dot above, if there are no more accents above on that base letter: remove the extra dot.� Turkish and Azeri (at least) use a dotless i as the lowercase of I. It should not be used if there are more accents above (then use an ordinary i which then looses the dot...).
|
|
I (no more accents above) |
i-dotless |
Suggested changes to SpecialCasing.txt regarding dotting i�s and j�s
The exposition tables above were not intended to be complete.� The formal tables below are intended to be complete enough to cover the orthographic requirements and also be such that NFD and NFC are handled consistently. Cases like barred i or j-crosstail are not covered. Review and comments are welcome.� The intent is for these modifications to be included in Unicode 3.2, or if possible, in an update to Unicode 3.1.
Old lines (to remove)
1st-------------------
# characters
where they are 1-1, and does not have locale-specific mappings.)
2nd-------------------
# The
<condition_list> is optional. Where present, it consists of one or more
locales or contexts,
# separated by spaces.
3rd-------------------
# A locale is
defined as:
# <locale> := <ISO_639_code> ( "_" <ISO_3166_code>
( "_" <variant> )? )?
# <ISO_3166_code> := 2-letter ISO country code,
# <ISO_639_code> := 2-letter ISO language code
4th-------------------
# A context is
one of the following choices:
5th-------------------
# AFTER_i: The
last base character was "i" 0069
6th-------------------
7th-------------------
#
================================================================================
# Locale-sensitive mappings
#
================================================================================
# Lithuanian
0307; 0307; ; ; lt AFTER_i; # Remove DOT ABOVE after "i" with upper
or titlecase
# Turkish, Azeri
0049; 0131; 0049; 0049; tr; # LATIN CAPITAL LETTER I
0069; 0069; 0130; 0130; tr; # LATIN SMALL LETTER I
0049; 0131; 0049; 0049; az; # LATIN CAPITAL LETTER I
0069; 0069; 0130; 0130; az; # LATIN SMALL LETTER I
# Note: the following cases are already in the UnicodeData file.
# 0131; 0131; 0049; 0049; tr; # LATIN SMALL LETTER DOTLESS I
# 0130; 0069; 0130; 0130; tr; # LATIN CAPITAL LETTER I WITH DOT ABOVE
end-------------------
New lines (to insert, replacing the old ones listed above)
1st-------------------
# characters
where they are 1-1, and does not have language-specific mappings.)
#
# Note that when case mapping a string in a normal form,
# the result need not be in any normal form.
#
2nd-------------------
# The
<condition_list> is optional. Where present, it consists of one or more
# contexts, one of which may be a language code, separated by spaces.
3rd-------------------
# A _subset_
of RFC 3066 conforming language codes, _sufficient for this file_,
# can be described as:
# <langcode> := two-letter ISO 639-1 language code
4th-------------------
# A context is
a <langcode> or one of the following choices (test on original string):
5th-------------------
# AFTER_i: The
last preceding base character was "i" (0069), "j" (006A),
# or has a canonical decomposition that begins with an "i" or
"j" but has no
# combining characters above (i.e., i-ogonek (012F), i-tilde-below (1E2D),
# or i-dot-below (1ECB)); AND no combining character class 230 (above) has
# intervened. (Neither i-stroke (0268) or j-crosstailed (029D) need be
# specially handled below, while they also have a soft dot above that
# is lost on normal uppercase or accenting above.)
#
# AFTER_CAP_I: The last preceding base character was "I" (0049),
"J" (004A),
# or has a canonical decomposition that begins with an "I" or
"J" but has no
# combining characters above (i.e., I-ogonek (012E), I-tilde-below (1E2C),
# or I-dot-below (1ECA)); AND no combining character class 230 (above) has
# intervened. (I-stroke (0197) need not be specially handled below, while
# it also has a soft dot above in lowercase form.)
#
# MORE_ACCENTS_ABOVE: The current combining sequence has at least one class 230
# (above) combining character after the currently considered character.
6th-------------------[no old text]
#-----
# Normal dotting/undotting of i's and j's (capital and small):
#-----
# Remove spurious explicit dot above small i or j when case mapping,
# if no more accents above:
0307; ; ; ; AFTER_i NON_MORE_ACCENTS_ABOVE # COMBINING DOT ABOVE
# Remove explicit dot above capital i or j when lowercasing,
# if no more accents above (mainly for NFC-NFD consistency for i--I-dot):
0307; ; 0307; 0307; AFTER_CAP_I NON_MORE_ACCENTS_ABOVE # COMBINING DOT ABOVE
# For NFC-NFD consistency for I-dot--i:
0130; 0069 0307; 0130; 0130; MORE_ACCENTS_ABOVE # LATIN CAPITAL LETTER I WITH
DOT
# Note: the following cases are already in the UnicodeData file.
# 0131; 0131; 0049; 0049; # LATIN SMALL LETTER DOTLESS I
# 0130; 0069; 0130; 0130; [NON_MORE_ACCENTS_ABOVE] # LATIN CAPITAL LETTER I
WITH DOT ABOVE
7th-------------------
#
================================================================================
# Language-sensitive mappings
# ================================================================================
#
# Lithuanian:
#
# Remove dot above small i's or j's when uppercasing,
# even if there are more accents above:
0307; 0307; ; ; lt AFTER_i # COMBINING DOT ABOVE
# Introduce an explicit dot above when lowercasing capital I's and J's
# if there are more accents above (grave, acute, tilde above, and ogonek
# occur in Lithuanian; the rest are just for consistency between NFC and NFD):
0049; 0069 0307; 0049; 0049; lt MORE_ACCENTS_ABOVE # LATIN CAPITAL LETTER I
004A; 006A 0307; 004A; 004A; lt MORE_ACCENTS_ABOVE # LATIN CAPITAL LETTER J
012E; 012F 0307; 012E; 012E; lt MORE_ACCENTS_ABOVE # LATIN CAPITAL LETTER I
WITH OGONEK
00CC; 0069 0307 0300; 00CC; 00CC; lt # LATIN CAPITAL LETTER I WITH GRAVE
00CD; 0069 0307 0301; 00CD; 00CD; lt # LATIN CAPITAL LETTER I WITH ACUTE
0128; 0069 0307 0303; 0128; 0128; lt # LATIN CAPITAL LETTER I WITH TILDE
1E2C; 1E2D
0307; 1E2C; 1E2C; lt MORE_ACCENTS_ABOVE # LATIN CAPITAL LETTER I WITH TILDE
BELOW
1ECA; 1ECB 0307; 1ECA; 1ECA; lt MORE_ACCENTS_ABOVE # LATIN CAPITAL LETTER I
WITH DOT BELOW
00CE; 0049 0307 0302; 00CE; 00CE; lt # LATIN CAPITAL LETTER I WITH CIRCUMFLEX
0134; 004A 0307 0302; 0134; 0134; lt # LATIN CAPITAL LETTER J WITH CIRCUMFLEX
0128; 0049 0307 0303; 0128; 0128; lt # LATIN CAPITAL LETTER I WITH TILDE
012A; 0049 0307 0304; 012A; 012A; lt # LATIN CAPITAL LETTER I WITH MACRON
012C; 0049 0307 0306; 012C; 012C; lt # LATIN CAPITAL LETTER I WITH BREVE
01CF; 0049 0307 030C; 01CF; 01CF; lt # LATIN CAPITAL LETTER I WITH CARON
0208; 0049 0307 030F; 0208; 0208; lt # LATIN CAPITAL LETTER I WITH DOUBLE GRAVE
020A; 0049 0307 0311; 020A; 020A; lt # LATIN CAPITAL LETTER I WITH INVERTED
BREVE
1E2E; 0049 0307 0308 0301; 1E2E; 1E2E; lt # LATIN CAPITAL LETTER I WITH
DIAERESIS AND ACUTE
1EC8; 0049 0307 0309; 1EC8; 1EC8; lt # LATIN CAPITAL LETTER I WITH HOOK ABOVE
#
# Turkish, Azeri:
#
# Remove spurious dot above small i's when lowercasing, if no more accents
above:
0307; ; 0307; 0307; tr AFTER_i NON_MORE_ACCENTS_ABOVE # COMBINING DOT ABOVE
0307; ; 0307; 0307; az AFTER_i NON_MORE_ACCENTS_ABOVE # COMBINING DOT ABOVE
# I�i-dotless and I-dot--i-with-soft-dot are case pairs in Turkish and Azeri,
# when there are no more accents above (otherwise use the ordinary casing
rules):
0069; 0069; 0130; 0130; tr NON_MORE_ACCENTS_ABOVE # LATIN SMALL LETTER I
0069; 0069; 0130; 0130; az NON_MORE_ACCENTS_ABOVE # LATIN SMALL LETTER I
0049; 0131; 0049; 0049; tr NON_MORE_ACCENTS_ABOVE # LATIN CAPITAL LETTER I
0049; 0131; 0049; 0049; az NON_MORE_ACCENTS_ABOVE # LATIN CAPITAL LETTER I
end-------------