RE: MS/Unix BOM FAQ again (small fix)

From: Rick Cameron (Rick.Cameron@crystaldecisions.com)
Date: Wed Apr 10 2002 - 12:45:22 EDT


So the original statement was correct. If the file starts with FF FE, it
must be a little-endian encoding; but you can't tell whether it's UTF-16 or
UTF-32.

- rick cameron

-----Original Message-----
From: Mark Davis [mailto:mark@macchiato.com]
Sent: Tuesday, 9 April 2002 20:36
To: Kenneth Whistler
Cc: unicode@unicode.org; kenw@sybase.com
Subject: Re: MS/Unix BOM FAQ again (small fix)

Sorry, I meant to write "since UTF-32LE, for example, could start with bytes
FF FE."

It would be the start of a character like U+1FEFF, which would be

FF FE 01 00

Mark

—————

Γνῶθι σαυτόν — Θαλῆς
[For transliteration, see http://oss.software.ibm.com/cgi-bin/icu/tr]

http://www.macchiato.com

----- Original Message -----
From: "Kenneth Whistler" <kenw@sybase.com>
To: <mark@macchiato.com>
Cc: <unicode@unicode.org>; <kenw@sybase.com>
Sent: Tuesday, April 09, 2002 19:23
Subject: Re: MS/Unix BOM FAQ again (small fix)

> > I agree, there are different ways to look at it. But the statement
> >
> > > > > A Unicode text file beginning with FEFF is big-endian, and a
> > > > > file beginning with FFFE (not a legal
Unicode
> > > > > character for any other purpose) is little-endian
> >
> > is just plain wrong, since UTF-32, for example, could start with
bytes
> > FE FF.
>
> Um, not legally in open interchange.
>
> Either you have big-endian UTF-32 <FE FF nn mm ..> which would
correspond
> to U-FEFFnnmm ... -- and that is out-of-range for both Unicode and
10646.
>
> Or you have little-endian UTF-32 <FE FF nn 00 ..> which would
correspond
> to U-00nnFFFE ..., where nn could be 00..10, but all such values are
> noncharacters, and cannot be used in open interchange.
>
> So if serialized "Unicode text" starts off <FE FF ...> and purports
to be legal,
> it cannot be UTF-32, it cannot be UTF-8, and it cannot be
little-endian.
>
> --Ken
>



This archive was generated by hypermail 2.1.2 : Wed Apr 10 2002 - 12:01:56 EDT