Re: Why doesn't Ideographic (ID) in UAX#14 have half-width katakana?

Thank you, Ken, for your dedicated archeological efforts.

I would like to emphasize that, at the time, UAX#14 reflected observed
behavior, in particular (but not exclusively) for MS products some of
which (at the time) used an LB algorithm that effectively matched an
untailored UAX#14.

However, recently, the W3C has spent considerable effort to look into
different layout-related algorithms and specification. If, in that
context, a consensus approach is developed that would point to a better
"default" behavior for untailored UAX#14-style line breaking, I would
regard that as a critical mass of support to allow UTC to consider
tinkering with such a long-standing set of property assignments.

This would be true, especially, if it can be demonstrated that (other
than matching legacy behavior) there's no context that would benefit
from the existing classification. I note that this was something several
posters implied.

So, if implementers of the legacy behavior are amenable to achieve this
by tailoring, and if the change augments the number of situations where
untailored UAX#14-style line breaking can be used, that would be a win
that might offset the cost of a disruptive change.

We've heard arguments why the proposed change is technically superior
for Japanese. We now need to find out whether there are contexts where a
change would adversely affect users/implementers. Following that, we
would look for endorsements of the proposal from implementers or other
standards organizations such as W3C (and, if at all possible, agreement
from those implementers who use the untailored algorithm now). With
these three preconditions in place, I would support an effort of the UTC
to revisit this question.

A./

On 5/1/2015 9:48 AM, Ken Whistler wrote:
> Suzuki-san,
>
> On 5/1/2015 8:25 AM, suzuki toshiya wrote:
>>
>> Excuse me, there is any discussion record how UAX#14 class for
>> halfwidth-katakana in 15 years ago? If there is such, I want to
>> see a sample text (of halfwidth-katakana) and expected layout
>> result for it.
>
> The *founding* document for the UTC discussion of the initial
> Line_Break property values 15 years ago was:
>
> http://www.unicode.org/L2/L1999/99179.pdf
>
> and the corresponding table draft (before approval and conversion
> into the final format that was published with UTR #14 -- later
> /UAX/ #14) was:
>
> http://www.unicode.org/L2/L1999/99180.pdf
>
> There is nothing different or surprising in terms of values there. The
> halfwidth
> katakana were lb=AL and the fullwidth katakana were lb=ID in
> that earliest draft, as of 1999.
>
> What is new information, perhaps, is the explicit correlation that can
> be found
> in those documents with the East_Asian_Width properties, and the
> explanation
> in L2/99-179 that the EAW property values were explicitly used to
> make distinctions for the initial LB values.
>
> There is no sample text or expected layout results from that time period,
> because that was not the basis for the original UTC decisions on any
> of this.
> Initial LB values were generated based on existing General_Category
> and EAW values, using general principles. They were not generated by
> examining and specifying in detail the line breaking behavior for
> every single script in the standard, and then working back from those
> detailed specifications to attempt to create a universal specification
> that would replicate all of that detailed behavior. Such an approach
> would have been nearly impossible, given the state of all the data,
> and might have taken a decade to complete.
>
> That said, Japanese line breaking was no doubt considered as part of
> the overall background, because the initial design for UTR #14 was
> informed
> by experience in implementation of line breaking algorithms at Microsoft
> in the 90's.
>
>>
>> You commented that the UAX#14 class should not be changed but
>> the tailoring of the line breaking behaviour would solve
>> the problem (as Firefox and IE11 did). However, some developers
>> may wonder "there might be a reason why UTC put halfwidth-katakana
>> to AL - without understanding it, we could not determine whether
>> the proposed tailoring should be enabled always, or enabled
>> only for a specific environment (e.g. locale, surrounding text)".
>
> See above, in L2/99-179. *That* was the justification. It had nothing
> to do with specific environment, locale, or surrounding text.
>
>>
>> If UTC can supply the "expected layout result for halfwidth-
>> katakana (used to define the class in current UAX#14)", it
>> would be helpful for the developers to evaluate the proposed
>> tailoring algorithm.
>
> UAX #14 was never intended to be a detailed, script-by-script
> specification of line layout results. It is a default, generic, universal
> algorithm for line breaking that does a decent, generic job of
> line breaking in generic contexts without tailoring or specific
> knowledge of language, locale, or typographical conventions in use.
>
> UAX #14 is not a replacement for full specification of kinsoku
> rules for Japanese, in particular. Nor is it intended as any kind
> of replacement for JIS X 4051.
>
> Please understand this: UAX #14 does *NOT* tell anyone how
> Japanese text *should* line break. Instead, it is Japanese typographers,
> users and standardizers who tell implementers of line break
> algorithms for Japanese what the expectations for Japanese text should
> be, in what contexts. It is then the job of the UTC and of the
> platform and application vendors to negotiate the details of
> which part of that expected behavior makes sense to try to
> cover by tweaking the default line-breaking algorithm and the
> Line_Break property values for Unicode characters, and which
> part of that expected behavior makes sense to try to cover
> by adjusting commonly accessible and agreed upon tailoring
> behavior (or public standards like CSS), and finally which part of that
> expected behavior should instead be addressed by value-added, proprietary
> implementations of high end publishing software.
>
> Regards,
>
> --Ken
>>
>>
>
Received on Fri May 01 2015 - 14:13:55 CDT