[Unicode]   Common Locale Data Repository : Bug Tracking Home | Site Map | Search
 
Modify

CLDR Ticket #11525(new)

Opened 2 weeks ago

Last modified 13 days ago

Shorten some tests in cldr-unittest TestAll

Reported by: tbishop Owned by: emmons
Component: other-unittest Data Locale:
Phase: dsub Review:
Weeks: Data Xpath:
Xref:

Description

TestUnicodeMapTime includes very time-consuming tests that are run unless shortTest is true. On an ordinary run of TestAll we want the short test, but currently we get the long test, depending on factors to be clarified in studying this ticket. This ticket is to fix that for TestUnicodeMapTime, and maybe also other tests that have similar problems and solutions.

Attachments

Change History

comment:1 Changed 2 weeks ago by tbishop

It turns out that the time-consuming part is NOT related to 1000 iterations in TestUnicodeMapTime:

    public void TestUnicodeMapTime() {
        boolean shortTest = getInclusion() < 10;
        double hashTime, umTime, icuTime, treeTime;
        int warmup = shortTest ? 1 : 20;
        umTime = checkUnicodeMapSetTime(warmup, 0);
        hashTime = checkUnicodeMapSetTime(warmup, 1);
        logln("Percentage: " + pf.format(hashTime / umTime));
        treeTime = checkUnicodeMapSetTime(warmup, 3);
        logln("Percentage: " + pf.format(treeTime / umTime));
        //logln(map1.toString());

        if (shortTest) {
            return;
        }

        umTime = checkUnicodeMapGetTime(1000, 0);
        hashTime = checkUnicodeMapGetTime(1000, 1);
        logln("Percentage: " + pf.format(hashTime / umTime));
        icuTime = checkUnicodeMapGetTime(1000, 2);
        logln("Percentage: " + pf.format(icuTime / umTime));
        treeTime = checkUnicodeMapGetTime(1000, 3);
        logln("Percentage: " + pf.format(treeTime / umTime));
    }

Even with shortTest == true (getInclusion() == 0), and even with iterations == warmup == 1, checkUnicodeMapSetTime(1, 0) takes 11.16 seconds, and checkUnicodeMapSetTime(1, 3) takes 16.43 seconds.

checkUnicodeMapSetTime(1, 1) is fast, only taking .0015 seconds.

    private double checkUnicodeMapSetTime(int iterations, int type) {
        _checkUnicodeMapSetTime(1, type);
        double result = _checkUnicodeMapSetTime(iterations, type);
        logln((type == 0 ? "UnicodeMap" : type == 1 ? "HashMap" : type == 2 ? "ICU" : "TreeMap") + "\t" + nf.format(result));
        return result;
    }

    private double _checkUnicodeMapSetTime(int iterations, int type) {
        UnicodeMap<String> map1 = SCRIPTS;
        Map<Integer, String> map2 = map1.putAllCodepointsInto(new HashMap<Integer, String>());
        Map<Integer, String> map3 = new TreeMap<Integer, String>(map2);
        System.gc();
        double start = System.currentTimeMillis();
        for (int j = 0; j < iterations; ++j)
            for (int cp = 0; cp <= SET_LIMIT; ++cp) {
                int enumValue = UCharacter.getIntPropertyValue(cp, propEnum);
                if (enumValue <= 0) continue; // for smaller set
                String value = UCharacter.getPropertyValueName(propEnum, enumValue, UProperty.NameChoice.LONG);
                switch (type) {
                case 0:
                    map1.put(cp, value);
                    break;
                case 1:
                    map2.put(cp, value);
                    break;
                case 3:
                    map3.put(cp, value);
                    break;
                }
            }
        double end = System.currentTimeMillis();
        return (end - start) / 1000 / iterations;
    }
Last edited 2 weeks ago by tbishop (previous) (diff)

comment:2 Changed 13 days ago by mark

Actually you could just skip that test unless we are in exhaustive mode.

View

Add a comment

Modify Ticket

Action
as new
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.