[Unicode]   Common Locale Data Repository : Bug Tracking Home | Site Map | Search
 
Modify

CLDR Ticket #10193(closed tools: fixed)

Opened 8 months ago

Last modified 3 months ago

sample attribute in parseLenient needs to be a distinguishing attribute

Reported by: emmons Owned by: mark
Component: util Data Locale:
Phase: dvet Review: emmons
Weeks: Data Xpath:
Xref:

Description

In the supplemental data we have:

             <distinguishingItems attributes="key request id _q registry alt iso4217 iso3166 mzone from to type numberSystem count"/>

The new "parseLenient" data adds paths that have for example,

[@sample='$']

"sample" needs to be distintuishing attribute for resolution of xpaths. I'm fairly convinced that this is what is causing the data loading issues in smoketest ST.

Attachments

Change History

comment:1 follow-up: ↓ 2 Changed 8 months ago by emmons

Survey Tool XPathTable.java uses

   Collection<String> distinguishing = sm.getSupplementalDataInfo().getDistinguishingAttributes();

to determine which attributes are distinguishing and which are not. This API uses the distinguishingItems data in supplementalMetadata.xml rather than the DTD to determine whether an attribute is distinguishing or not.

Not sure if the ST needs to change to use DtdData to get this information, but imho as long as these APIs exist, the supplementalMetadata needs to be in sync with the DTD.

comment:2 in reply to: ↑ 1 Changed 8 months ago by srl

Replying to emmons:

Survey Tool XPathTable.java uses

   Collection<String> distinguishing = sm.getSupplementalDataInfo().getDistinguishingAttributes();

to determine which attributes are distinguishing and which are not. This API uses the distinguishingItems data in supplementalMetadata.xml rather than the DTD to determine whether an attribute is distinguishing or not.

Not sure if the ST needs to change to use DtdData to get this information, but imho as long as these APIs exist, the supplementalMetadata needs to be in sync with the DTD.

They are supposed to be in sync, so it should be an error (supposed to be a test failure even?) if they aren't.

comment:3 Changed 8 months ago by mark

Ok, I figured out what it is.

We have not used the distinguishingItems for some time; those should have been deprecated long, long ago. The problem is that whether an attribute is distinguishing depends on more context, like the element it is in. None of the backend have used that for some releases, for example, in CLDRFile:

    public static boolean isDistinguishing(DtdType type, String elementName, String attribute) {
        return DtdData.getInstance(type).isDistinguishing(elementName, attribute);
    }

I poked around some more, and the following should be removed, along with the data that they use.

SupplementalDataInfo.java

    public Collection<String> getDistinguishingAttributes() {
    public List<String> getSerialElements() {

supplementalDataInfo.xml

  <serialElements>
  <distinguishing>
  <blocking> // luckily no API to remove

It appears the only callers are in the front-end (getSerialElements is only called by a test)

XPathTable calls getDistinguishingAttributes in

    public synchronized Set<String> getUndistinguishingElements() {

That method is semantically "getUndistinguishingAttributesPlus"; gets all the attributes from DtdData, then removes those that are in getDistinguishingAttributes(); also removes alt and draft.

It is only called by Map<String, String> getUndistinguishingElementsFor(String path, XPathParts xpp). That method gets all the attributes on any element that are not distinguishing. It returns a map from those attributes to the last value they have. ("last" meaning as you go though the path from right to left, so ldml/foo[@abc="def"]/bar[@abc="ghi"] will return {abc=ghi}).

That is only called by DataSection.DataRow.getNonDistinguishingAttributes, which just gets the values for a path. That is only called by toJSONString, in

JSONObject jo = new JSONObject();...jo.put("extraAttributes", getNonDistinguishingAttributes());

So, what I suggest is having a method

XPathParts.getSpecialNondistinguishingAttributes()

And use that in the innards of getUndistinguishingElementsFor, which becomes:

    public Map<String, String> getUndistinguishingElementsFor(String path, XPathParts xpp) {
        return XPathParts.getFrozenInstance(path).getSpecialNondistinguishingAttributes();
    }

NOTE: if this method is called all the time, it would make sense to cache the map in the frozen XPathParts (the map is normally null).

comment:4 Changed 8 months ago by mark

Note: there is one behavioral difference in the just-checked-in code. Since it uses DtdData, if the path passed in is illegal according to the dtd, an exception is thrown. So people should be on the lookout for that.

comment:5 Changed 8 months ago by emmons

  • Status changed from new to accepted
  • Component changed from unknown to util
  • Priority changed from assess to blocker
  • Milestone changed from UNSCH to 32
  • Owner changed from anybody to mark
  • Type changed from unknown to tools

comment:6 Changed 7 months ago by mark

  • Phase changed from dsub to dvet

comment:7 Changed 4 months ago by mark

  • Priority changed from blocker to major

comment:8 Changed 3 months ago by mark

  • Status changed from accepted to reviewing
  • Review set to emmons

comment:9 Changed 3 months ago by emmons

  • Status changed from reviewing to closed
  • Resolution set to fixed
View

Add a comment

Modify Ticket

Action
as closed
Next status will be 'new'
Next status will be 'closed'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.