[Unicode]   Common Locale Data Repository : Bug Tracking Home | Site Map | Search

CLDR Ticket #6726(closed enhancement: fixed)

Opened 4 years ago

Last modified 3 years ago

java/tools/readme.txt misleading at best

Reported by: srl Owned by: srl
Component: xxx-tools Data Locale:
Phase: dvet Review: emmons
Weeks: Data Xpath:


cleanup tools on how to build JSON


Change History

comment:1 Changed 4 years ago by srl

see icu-users thread from sept 26th 2013 about JSON Hello World.

comment:2 Changed 4 years ago by emmons

  • Status changed from new to assigned
  • Component changed from unknown to tools
  • Priority changed from assess to medium
  • Milestone changed from UNSCH to 25rc
  • Owner changed from anybody to srl
  • Type changed from unknown to enhancement

comment:3 Changed 4 years ago by rxaviers@…

Some info to consider fixing README:


The tool is "org.unicode.cldr.json.Ldml2JsonConverter" , not com.ibm.icu.dev.tool.cldr.LDML2ICUConverter.

There is a shell script in tools/scripts/CLDRWrapper, but it's missing in the current JSON.zip file. Should be included.


1). Download the tools.zip.
2). go to tools/java and do "ant clean jar"

( Yes, the "all" target should create the jar for you and doesn't - this is a known issue ).

3). There is a shell script in tools/scripts/CLDRWrapper, that is a nice shell wrapper for all of the CLDR tooling. I usually just do "cp CLDRWrapper /usr/local/bin/Ldml2JsonConverter". You may need to modify the script so that it will know that Ldml2JsonConverter is in org.unicode.cldr.json and not somewhere else., also make sure that the CLDRCLASSPATH is right in the script. Mine is (assuming that CLDR_DIR is "/build/cldr" ).


4). Run /usr/local/bin/Ldml2JsonConverter -h to get the usage statement. If you want "everything", I would suggest options "-t main -r true"

comment:4 Changed 4 years ago by illimar.tambek@…

I tried following the instructions in the readme and here, but I am stuck after doing "ant clean jar".

First, I don't want to copy the CLDRWrapper to my /usr/local/bin. I'd like to keep it where it is, so I just do "cp scripts/CLDRWrapper Ldml2JsonConverter and then ./Ldml2JsonConverter -h", but I get the error:

Exception in thread "main" java.lang.NoClassDefFoundError: org/unicode/cldr/tool/Ldml2JsonConverter

And that happens even if my CLASSPATH seems correct: CLDRCLASSPATH=${HOME}/icu4j/icu4j.jar:${HOME}/icu4j/classes/:${HOME}/Dow‌​nloads/cldr24tools/tools/java/cldr.jar.

I think I got a bit closer with this:
"java -classpath cldr.jar org.unicode.cldr.tool.Ldml2JsonConverter" - but now I am getting the following error:
Exception in thread "main" java.lang.NoClassDefFoundError: com/ibm/icu/text/DateFormat at org.unicode.cldr.json.Ldml2JsonConverter.<clinit>(Ldml2JsonConverter.java:42) Caused by: java.lang.ClassNotFoundException: com.ibm.icu.text.DateFormat

It looks like there is no DateFormat class in classes/com/ibm/icu/text/. I assume it's supposed to be there, in the tools.zip, no? Or do I have to download something else as well? ICU?

PS. Running OSX 10.8 here.

I think the instructions need to be MUCH better than they are now. Even a non-tech person should be able to figure out how to convert LDML to JSON (remmeber - not all localizers are programmers, let alone java programmers!).

comment:5 Changed 4 years ago by rxaviers@…

Yes, the "all" target should create the jar for you and doesn't - this is a known issue

What's the bug number opened for that?

comment:6 Changed 4 years ago by rxaviers@…

tools/scripts/CLDRWrapper is available at svn trunk. But, it's not available at http://www.unicode.org/Public/cldr/latest/tools.zip.

Should we include this script in the tools.zip?

comment:7 Changed 4 years ago by Rafael Xavier <rxaviers@…>

illimar, I got the it running. That's what I've done:

$ svn co http://unicode.org/repos/cldr/trunk /tmp/cldr
$ cd /tmp/cldr
$ ln -s tools/scripts/CLDRWrapper 

Then, change tools/scripts/CLDRWrapper according to the below:

Index: tools/scripts/CLDRWrapper
--- tools/scripts/CLDRWrapper	(revision 9527)
+++ tools/scripts/CLDRWrapper	(working copy)
@@ -15,7 +15,7 @@
 ## Change the following if needed
 # where icu4j lives (or other classes)
 # the command to use with java (or empty for none)
 # how to run java
@@ -50,6 +50,7 @@
     "ShowLanguages") PKG=org.unicode.cldr.tool;;
     "LDML2ICUConverter") PKG=org.unicode.cldr.icu;;
     "ConsoleCheckCLDR") PKG=org.unicode.cldr.test;;
+    "Ldml2JsonConverter") PKG=org.unicode.cldr.json;;
     *)  PKG=org.unicode.cldr.tool;;
$ cd tools/java/
$ ant clean jar
$ cd ../../
$ export CLDR_DIR=/tmp/cldr/
$ ./Ldml2JsonConverter -h

comment:8 Changed 4 years ago by Rafael Xavier <rxaviers@…>

Although, I could not run the converter anyway...

$ ./Ldml2JsonConverter -r true -t main -m pt_BR
commondir	=	/tmp/cldr/common/
destdir	=	/tmp/Generated/cldr/
match	=	pt_BR
type	=	main
resolved	=	true
draftstatus	=	unconfirmed
coverage	=	optional
fullnumbers	=	false
other	=	false
identity	=	true
Processing file main/pt_BR
  languages.json = 587 values
  scripts.json = 217 values
  territories.json = 305 values
  variants.json = 67 values
  transformNames.json = 14 values
  measurementSystemNames.json = 7 values
Exception in thread "main" java.lang.IllegalArgumentException: Comparing two different DTDs: cldr, cldr
	at org.unicode.cldr.util.DtdData$DtdComparator.compare(DtdData.java:451)
	at org.unicode.cldr.util.DtdData$DtdComparator.compare(DtdData.java:436)
	at org.unicode.cldr.json.CldrItem.compareTo(CldrItem.java:264)
	at org.unicode.cldr.json.CldrItem.compareTo(CldrItem.java:14)
	at java.util.ComparableTimSort.countRunAndMakeAscending(ComparableTimSort.java:290)
	at java.util.ComparableTimSort.sort(ComparableTimSort.java:171)
	at java.util.ComparableTimSort.sort(ComparableTimSort.java:146)
	at java.util.Arrays.sort(Arrays.java:472)
	at java.util.Collections.sort(Collections.java:155)
	at org.unicode.cldr.json.Ldml2JsonConverter.resolveSortingItems(Ldml2JsonConverter.java:477)
	at org.unicode.cldr.json.Ldml2JsonConverter.convertCldrItems(Ldml2JsonConverter.java:397)
	at org.unicode.cldr.json.Ldml2JsonConverter.processDirectory(Ldml2JsonConverter.java:875)
	at org.unicode.cldr.json.Ldml2JsonConverter.main(Ldml2JsonConverter.java:84)

comment:9 Changed 4 years ago by srl

  • Milestone changed from 25rc to 26dsub


comment:10 Changed 4 years ago by me@…

I tried the instructions by Rafael and it finally worked. You need to checkout the svn, do not use the tools.zip.

One missing piece:

ln -s tools/scripts/CLDRWrapper 

should be

ln -s tools/scripts/CLDRWrapper Ldml2JsonConverter

Because the symlink's file name matters, so the symlink cannot be named CLDRWrapper. It has to be named Ldml2JsonConverter. And the provided CLDRWrapper does not handle Ldml2JsonConverter yet, so that's why you also need to follow Rafael's diff patch.

comment:11 Changed 3 years ago by srl

  • Milestone changed from 26dsub to 26dvet

comment:12 Changed 3 years ago by srl

Hi folks.
I'd rather not update CLDRWrapper - I should remove it from the docs.

Instead, you can do this:

java -DCLDR_DIR=/wherever -jar cldr.jar ldml2json -t main ...

sound good? then java -jar figures out the classpath stuff.

comment:13 Changed 3 years ago by srl

We don't even have a page that explains how to run the tools. Hmm. Lots of breakage.

comment:14 Changed 3 years ago by srl

OK, I've started a new tools page at http://cldr.unicode.org/tools and updated the readme to link there. More work to do.

comment:15 Changed 3 years ago by srl

  • Status changed from assigned to reviewing
  • Review set to emmons

comment:16 Changed 3 years ago by srl

more to do on the online pages, but code changes done for now

comment:17 Changed 3 years ago by emmons

  • Status changed from reviewing to closed
  • Resolution set to fixed

comment:18 Changed 3 years ago by markus

  • Phase set to dvet
  • Milestone changed from 26dvet to 26

Add a comment

Modify Ticket

as closed
Next status will be 'new'
Next status will be 'closed'

E-mail address and user name can be saved in the Preferences.

Note: See TracTickets for help on using tickets.