Re: pre-HTML5 and the BOM

David Starner wrote:

> In the environment that UTF-8 was developed for, a BOM is a nuisance;
> a BOM will stop the shell from properly interpreting a hashbang, and
> other existing programs will lose the BOM, duplicate the BOM, and
> scatter BOMs throughout files. Given the number of text-like file
> formats (like old-school PNM) and number of scripts depending on
> existing behavior, these aren't going to be changed.

We've been hearing the story about hashbang for many, many years now,
and I still don't understand why the following logic hasn't been made
part of the low-level I/O process in such environments:

"When reading a text file that could be UTF-8 or some other ACE, if the
first three bytes of the file are EF BB BF, discard them and assume the
file is UTF-8."

> As I said before, Unicode simplified but did not solve the fact that
> text from other operating systems requires some modification before
> working just right. But I don't think that Unicode should recommend
> unconditionally the UTF-8 BOM, because it is problematic in the field
> of use UTF-8 was created for and is still used for.

I think there is a middle ground of tolerance between unconditionally
recommending it and unconditionally recommending against it.

--
Doug Ewell | Thornton, Colorado, USA
http://www.ewellic.org | @DougEwell ­