Feedback on the proposal to change U+FFFD generation when decoding ill-formed UTF-8
Shawn Steele via Unicode
unicode at unicode.org
Tue May 30 12:11:40 CDT 2017
> Which is to completely reverse the current recommendation in Unicode 9.0. While I agree that this might help you fending off a bug report, it would create chances for bug reports for Ruby, Python3, many if not all Web browsers,...
& Windows & .Net
Changing the behavior of the Windows / .Net SDK is a non-starter.
> Essentially, "overlong" is a word like "dragon" or "ghost": Everybody knows what it means, but everybody knows they don't exist.
Yes, this is trying to improve the language for a scenario that CANNOT HAPPEN. We're trying to optimize a case for data that implementations should never encounter. It is sort of exactly like optimizing for the case where your data input is actually a dragon and not UTF-8 text.
Since it is illegal, then the "at least 1 FFFD but as many as you want to emit (or just fail)" is fine.
More information about the Unicode