2017-08-24 19:17 GMT+02:00 Andre Schappo via Unicode <unicode_at_unicode.org>:
>
> Because there are many systems that can now handle BMP characters but not
> cannot handle SMP characters.
>
> One example being systems that use mysql utf8 (3 byte encoding) and have
> not yet updated to utf8mb4 (4 byte encoding)
>
Mysql's utf8 is known to cause severe problems, notably on wikis installed
by default with it: the presence of any non-BMP character (SMP or emojis
are now very frequent and available on almost all modern smartphones) in
the edited text will cause its **silent** truncation when uploading it to
the server (when it will save the text to the database) even if any unsaved
preview was correct. You will see the truncation when the page is loaded
again.
Mysql's "utf8" should have been dropped since long and replaced by utf8mb4
or setup so that data send to an "utf8"-encoded database would cause a SQL
error that cannot be silently ignored with truncation (or it least it
should only cause the non-BMP characters to be filtered out, without
silently deleting everything that follows).
This is an old severe bug of Mysql (on the server itself) or in the
connection protocol, or internal filters used by Mysql client library, that
has caused many severe security issues (such as discarding logs or todo
lists, or loss of pending commercial transactions such as lists of payments
to process to a bank or truncated billings sent to customers, or loss of
contact address or name, or broken complete addresses for product delivery
to a customer, or missing items in a delivered box and lost products in the
middle of their routing).
This is a demosntration that not signaling encoding errors to an
application, or not clearly specifiying that an API may cause encoding
exceptions that must be caught and must not ignored in applications, can
hurt. Even if you use "utf8mb4" encoding errors are still possible and must
not be ignored as the final result will be unpredictable.
Received on Thu Aug 24 2017 - 13:25:20 CDT
This archive was generated by hypermail 2.2.0 : Thu Aug 24 2017 - 13:25:20 CDT