RE: Normalization in panlingual application

From: Philippe Verdy (verdy_p@wanadoo.fr)
Date: Sun Sep 23 2007 - 01:39:46 CDT

  • Next message: Philippe Verdy: "RE: New Public Review Issue: Proposed Update UTS #18"

    Kenneth Whistler wrote:
    > À : verdy_p@wanadoo.fr
    > Philippe Verdy said:
    > > The fact that IDN makes some use of it (now in a non-conforming way
    > because
    > > it uses its own rules to define its own sets of mappings, and to
    > preserve
    > > compatibility with future evolutions, it does not automatically
    > integrate
    > > all Unicode additions) is another problem,
    >
    > And this claim about IDN is also false.
    >
    (...)
    >
    > The fact that the output of NamePrep is not the same as simply
    > normalizing a string with NFKC is beside the point. Of course
    > it is different, because NamePrep specifies the use of various
    > mappings and character prohibitions in addition to NFKC
    > normalization.

    These prohibitions, and the fact that IDN is also late in accepting newer
    Unicode versions simply means that it is different. What it does is to
    effectively reject what NFKC allows (and even mandates if we use the same
    version of Unicode). That's why I say it is not NFKC. Its implementation for
    IDN, MUST remain distinct from the implementation for Unicode.

    I have found and verified this when I attempted to use the same code. I had
    a make an separate implementation for IDN (actually I could reuse most of
    the code, but I needed a separate data table, that was not just the past 3.2
    data, but included also the IDN prohibitions that even affect its NFKC
    result)...



    This archive was generated by hypermail 2.1.5 : Sun Sep 23 2007 - 01:42:16 CDT