Feedback on the proposal to change U+FFFD generation when decoding ill-formed UTF-8
Asmus Freytag via Unicode
unicode at unicode.org
Thu Jun 1 09:21:28 CDT 2017
On 6/1/2017 2:32 AM, Henri Sivonen via Unicode wrote:
> On Wed, May 31, 2017 at 10:38 PM, Doug Ewell via Unicode
> <unicode at unicode.org> wrote:
>> Henri Sivonen wrote:
>>> If anything, I hope this thread results in the establishment of a
>>> requirement for proposals to come with proper research about what
>>> multiple prominent implementations to about the subject matter of a
>>> proposal concerning changes to text about implementation behavior.
>> Considering that several folks have objected that the U+FFFD
>> recommendation is perceived as having the weight of a requirement, I
>> think adding Henri's good advice above as a "requirement" seems
>> heavy-handed. Who will judge how much research qualifies as "proper"?
I agree with Henri on these general points:
1) Requiring extensive research on implementation practice is crucial in
dealing with any changes to long standing definitions, algorithms,
properties and recommendations.
2) Not having a perfect definition of what "extensive" means is not an
excuse to do nothing.
3) Evaluating only the proposer's implementation (or only ICU) is not
4) Changing a recommendation that many implementers (or worse, an
implementers' collective) have chosen to adopt is a breaking change.
5) Breaking changes to fundamental algorithms require extraordinarily
strong justification including, but not limited to "proof" that the
existing definition/recommendation is not workable or presents grave
security risks that cannot be mitigated any other way.
I continue to see a disturbing lack of appreciation of these issues in
some of the replies to this discussion (and some past decisions by the UTC).
More information about the Unicode