International Components for Unicode for Java (ICU4J)

Read Me for ICU4J 4.2.1.1

Date
April 8, 2010

Note: This is an update release of ICU4J 4.2. This release contains several bug fixes and updated data, but does not introduce any new APIs or functionalities.

For the most recent release, see the ICU4J download site.

Contents

• Introduction to ICU4J
• What Is New In This Release?
• License Information
• Platform Dependencies
• How to Download ICU4J
• The Structure and Contents of ICU4J
• Where to Get Documentation
• How to Install and Build
• How to modularize ICU4J
• Trying Out ICU4J
• ICU4J Resource Information
• About ICU4J Time Zone
• Where to Find More Information
• Submitting Comments, Requesting Features and Reporting Bugs

Introduction to ICU4J

The International Components for Unicode (ICU) library provides robust and full-featured Unicode services on a wide variety of platforms. ICU supports the most current version of the Unicode standard, including support for supplementary characters (needed for GB 18030 repertoire support).

Java provides a strong foundation for global programs, and IBM and the ICU team played a key role in providing globalization technology to Java. But because of its long release schedule, Java cannot always keep up with evolving standards. The ICU team continues to extend Java's Unicode and internationalization support, focusing on improving performance, keeping current with the Unicode standard, and providing richer APIs, while remaining as compatible as possible with the original Java text and internationalization API design.

ICU4J is an add-on to the regular JRE that provides:

• Collation – rule-based, up-to-date Unicode Collation Algorithm (UCA) sorting order
      For fast multilingual string comparison; faster and more complete than the J2SE implementation
• Supplementary Characters – String manipulation and character properties
      Required for proper GB 18030 and JIS X 0213 repertoire support
• Charset Detection – Recognition of various single and multibyte charsets
      Useful for recognizing untagged text data
• UnicodeSet – standard set operations optimized for sets of Unicode characters
      UnicodeSets can be built from string patterns using any Unicode properties.
• Transforms – a flexible mechanism for Unicode text conversions
      Including Full/Halfwidth conversions, Normalization, Case conversions, Hex conversions, and transliterations between scripts (50+ pairs)
• Unicode Normalization – NFC, NFD, NFKD, NFKC
      For canonical text representations, needed for XML and the net
• International Calendars – Arabic, Buddhist, Chinese, Hebrew, Japanese, Ethiopic, Islamic, Coptic and other calendars
      Required for correct presentation of dates in certain countries
• Number Format Enhancements – Scientific Notation, Spelled-out, etc.
      Enhancements to the normal Java number formatting. The spell-out format is used for checks and similar documents
• Enhanced Word-Break Detection – Rule-based, supports Thai
      Required for correct support of Thai
• Unicode Text Compression – Standard compression of Unicode text
      Suitable for large numbers of small fields, where LZW and similar schemes do not apply
• Charset Conversion – Conversion to and from different charsets.
      Plugs into Java CharsetProvider Service Provider Interface (SPI)

Note: We continue to provide assistance to Sun, and in some cases, ICU4J support has been rolled into a later release of Java. For example, the Thai word-break is now in Java 1.4. However, the most current and complete version is always found in ICU4J.

What Is New In This Release?

See the ICU 4.2 download page about new features in this release. The list of API changes since the previous ICU4J release is available here.

Summary of changes in 4.2.1.1 (April 8, 2010)

Time zone data update

ICU4J 4.2.1.1 updates the time zone data to version 2010h from the Olson tz database.

Bug fixes

This release contains following bug fixes ported from ICU4J 4.4 development code stream

• #7107 Synchronization issue in TimeZone.getDisplayName
• #7137 ICU4J SimpleDateFormat takes too much time for initializing DateFormatSymbols
• #7140 SimpleDateFormat constructor takes too much time on initializing NumberingSystem
• #7169 DatTimePatternGenerator threading problem

Summary of changes in 4.2.1 (July 1, 2009)

Locale data update

ICU4J 4.2.1 updates the CLDR version to 1.7.1.

Time zone data update

ICU4J 4.2.1 updates the time zone data to version 2009j from the Olson tz database.

Bug fixes

This release contains following bug fixes (not a complete list)

• #6918 Date formatting should honor @numbers
• #7014 TimeZone.getTimeZone("Asia/Amman") returns GMT with tzdata 2009i
• #7017 ICU4J time zone test failure with tzdata2009i

License Information

The ICU projects (ICU4C and ICU4J) use the X license. The X license is suitable for commercial use and is a recommended free software license that is compatible with the GNU GPL license. This became effective with release 1.8.1 of ICU4C and release 1.3.1 of ICU4J in mid-2001. All new ICU releases will adopt the X license; previous ICU releases continue to utilize the IPL (IBM Public License). Users of previous releases of ICU who want to adopt new ICU releases will need to accept the terms and conditions of the X license.

The main effect of the change is to provide GPL compatibility. The X license is listed as GPL compatible, see the GNU page at http://www.gnu.org/philosophy/license-list.html#GPLCompatibleLicenses. This means that GPL projects can now use ICU code, it does not mean that projects using ICU become subject to GPL.

The IBM version contains the essential text of the license, omitting the X-specific trademarks and copyright notices. The full copy of ICU's license is included in the download package.

Platform Dependencies

By default ICU4J depends on functionality available in J2SE 5 or later releases. The binary distribution of ICU4J jar file may have a problem with older JRE versions. We provide the ability to build a variant of ICU4J for JRE 1.3 or 1.4. If you want to use ICU4J running on JRE 1.4, you should build ICU4J libraries from the source package with JDK 1.4. With older JDK releases, some build targets may not work.

The table below shows operating systems and JRE/JDK versions currently used by the ICU development team.

How to Download ICU4J

There are two ways to download the ICU4J releases.

• Official Release:
  If you want to use ICU4J (as opposed to developing it), your best bet is to download an official, packaged version of the ICU4J library files. These versions are tested more thoroughly than day-to-day development builds, and they are packaged in jar files for convenient download. These packaged files can be found at the ICU Downloads page.

• Subversion Source Repository:
  If you are interested in developing features, patches, or bug fixes for ICU4J, you should probably be working with the latest version of the ICU4J source code. You will need to check the code out of our Subversion repository to ensure that you have the most recent version of all of the files. There are several ways to do this. Please follow the directions that are contained on the Source Repository page for details.

For more details on how to download ICU4J directly from the web site, please see the ICU downloads page at http://www.icu-project.org/download/

The Structure and Contents of ICU4J

Below, $icu4j_root is the placement of the icu directory in your file system, like "drive:\...\icu4j" in your environment. "drive:\..." stands for any drive and any directory on that drive that you chose to install icu4j into.

Information and build files:

The source directories mirror the package structure of the code.
Core packages become part of the ICU4J jar file.
Charset packages become part of the ICU4J charset jar file.
LocaleSPI packages become part of the ICU4J Locale service provider jar file.
RichText classes are Core and API, but can be removed from icu4j.jar, and can be built into their own jar.
API packages contain classes with supported API.

Building ICU4J creates and populates the following directories:

ICU4J data is stored in the following locations:

Where to get Documentation

The ICU user's guide contains lots of general information about ICU, in its C, C++, and Java incarnations.

The complete API documentation for ICU4J (javadoc) is available on the ICU4J web site, and can be built from the sources:

• Index to all ICU4J API
• Charset Detector – Detection of charset from a byte stream
• International Calendars – Buddhist, Chinese, Coptic, Ethiopic, Gregorian, Hebrew, Indian, Islamic, Japanese.
• Unicode Normalization – Canonical text representation for W3C.
• Number Format Enhancements – Scientific Notation, Spelled out.
• Enhanced word-break detection – Rule-based, supports Thai
• Transliteration – A general framework for converting text from one format to another, e.g. Cyrillic to Latin, or Hex to Unicode.
• Unicode Text Compression & Decompression – 2:1 compression on English Unicode text.
• Collation - Rule-based sorting, Efficient multi-lingual searching

How to Install and Build

To install ICU4J, simply place the prebuilt jar file icu4j.jar on your Java CLASSPATH. If you need Charset API support please place icu4j-charsets.jar on your class path. No other files are needed.

To build ICU4J, you will need a J2SE SDK and the Ant build system. We strongly recommend using the Ant build system to build ICU4J. It's recommended to install both the J2SE SDK and Ant somewhere outside the ICU4J directory. For example, on Linux you might install these in /usr/local.

• Install J2SE SDK 6.0. (You can use any version of J2SE SDK 1.3 or newer, but but J2SE SDK 5.0 is the minimum version required to enable the all available ICU4J features.)
• Install the Ant build system. Ant is a portable, Java-based build system similar to make. ICU4J uses Ant because it introduces no other dependencies, it's portable, and it's easier to manage than a collection of makefiles. We currently build ICU4J using a single makefile on all platforms Ant. The build system requires Ant 1.6 or later.

  Installing Ant is straightforward. Download it (see http://ant.apache.org/bindownload.cgi), extract it onto your system, set some environment variables, and add its bin directory to your path. For example:

          set JAVA_HOME=C:\jdk1.6.0
          set ANT_HOME=C:\ant
          set PATH=%PATH%;%ANT_HOME%\bin

  See the current Ant documentation for details.

Once the J2SE SDK and Ant are installed, building is just a matter of typing ant in the ICU4J root directory. This causes the Ant build system to perform a build as specified by the file build.xml, located in the ICU4J root directory. You can give Ant options like -verbose, and you can specify targets. Ant will only build what's been changed and will resolve dependencies properly. For example:

C:\icu4j>ant
Buildfile: build.xml

checkAntVersion:

warnAntVersion:

initBase:
    [mkdir] Created dir: C:\icu4j\classes
     [echo] java home: C:\jdk1.6.0
     [echo] java version: 1.6.0
     [echo] ant java version: 1.6
     [echo] Apache Ant version 1.7.0 compiled on December 13 2006
     [echo] ICU4JDEV with Windows XP 5.1 build 2600 Service Pack 3 on x86
     [echo] clover initstring = '${clover.initstring}'
     [echo] target runtime environment: JAVASE6
     [echo] Initialized at 2009-04-25 at 05:25:03 EDT

buildMangle:
    [javac] Compiling 1 source file to C:\icu4j\classes

initSrc:

displayBuildEnvWarning:

doMangle:
     [echo] Running source code preprocessor [java com.ibm.icu.dev.tool.docs.Cod
eMangler -dJAVASE6 -n @preprocessor.txt]

init:

coreData:
     [copy] Copying 1 file to C:\icu4j\classes\com\ibm\icu

icudata:
    [unjar] Expanding: C:\icu4j\src\com\ibm\icu\impl\data\icudata.jar into c:\ic
u4j\classes
     [copy] Copying 1 file to C:\icu4j\classes\META-INF

durationdata:
     [copy] Copying 16 files to C:\icu4j\classes\com\ibm\icu\impl\duration\i
mpl\data

core:
    [javac] Compiling 351 source files to C:\icu4j\classes
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.

BUILD SUCCESSFUL
Total time: 44 seconds

The following are some targets that you can provide to ant. For more targets run ant -projecthelp or see the build.xml file.

For more information, read the Ant documentation and the build.xml file.

After doing a build it is a good idea to run all the icu4j tests by typing
"ant check" or "java -classpath classes com.ibm.icu.dev.test.TestAll -nothrow".

Eclipse users: See the ICU4J site for information on how to configure Eclipse to build and develop ICU4J on Eclipse IDE.

Note: To install/build ICU4J Locale Service Provider, please refer Read Me for ICU4J Locale Service Provider.

How to modularize ICU4J

Some clients may not wish to ship all of ICU4J with their application, since the application might only use a small part of ICU4J. ICU4J release 2.6 and later provide build options to build individual ICU4J 'modules' for a more compact distribution. For more details, please refer to the section Modularization of ICU4J in the ICU user's guide article Packaging ICU.

Trying Out ICU4J

Note: the demos provided with ICU4J are for the most part undocumented. This list can show you where to look, but you'll have to experiment a bit. The demos (with the exception of richedit) are unsupported and may change or disappear without notice.

The icu4j.jar file contains only the core ICU4J classes, not the demo classes, so unless you build ICU4J there is little to try out.

Charset

java -cp $icu4j_root/icu4j.jar:$icu4j_root/icu4j-charsets.jar <your program>

Rich Edit

java -jar $icu4j_root/richedit/richedit.jar

With a fuller command line you can try out other options, for example:

java -classpath $icu4j_root/richedit/richedit.jar com.ibm.richtext.demo.EditDemo [-swing][file]

This will use an awt GUI, or a swing GUI if -swing is passed on the command line. It will open a text file if one is provided, otherwise it will open a blank page. Click to type.

You can add tabs to the tab ruler by clicking in the ruler while holding down the control key. Clicking on an existing tab changes between left, right, center, and decimal tabs. Dragging a tab moves it, dragging it off the ruler removes it.

You can experiment with complex text by using the keymap functions. Please note that these are mainly for demo purposes, for real work with Arabic or Hebrew you will want to use an input method. You will need to use a font that supports Arabic or Hebrew, 'Lucida Sans' (provided with Java) supports these languages.

Other demos

The other demo programs are not supported and exist only to let you experiment with the ICU4J classes. First, build ICU4J using ant all. Then try one of the following:

• java -classpath classes com.ibm.icu.dev.demo.calendar.CalendarApp
• java -classpath classes com.ibm.icu.dev.demo.holiday.HolidayCalendarDemo
• java -classpath classes com.ibm.icu.dev.demo.rbnf.RbnfDemo
• java -classpath classes com.ibm.icu.dev.demo.translit.Demo

ICU4J Resource Information

By default the ICU4J distribution includes all of the standard resource information. It is located under the directory com/ibm/icu/impl/data. Depending on the service, the data is in different locations and in different formats. Note: This will continue to change from release to release, so clients should not depend on the exact organization of the data in ICU4J.

• The primary locale data is under the directory icudt42b, as a set of ".res" files whose names are the locale identifiers. Locale naming is documented the com.ibm.icu.util.ULocale class, and the use of these names in searching for resources is documented in com.ibm.icu.util.UResourceBundle.
• The collation data is under the directory icudt42b/coll, as a set of ".res" files.
• The rule-based transliterator data is under the directory icudt42b/translit as a set of ".res" files. (Note: the Han transliterator test data is no longer included in the core icu4j.jar file by default.)
• The rule-based number format data is under the directory icudt42b/rbnf as a set of ".res" files.
• The break iterator data is directly under the data directory, as a set of ".brk" files, named according to the type of break and the locale where there are locale-specific versions.
• The holiday data is under the data directory, as a set of ".class" files, named "HolidayBundle_" followed by the locale ID.
• The character property data as well as assorted normalization data and default unicode collation algorithm (UCA) data is found under the data directory as a set of ".icu" files.
• The character set converter data is under the directory icudt42b, as a set of ".cnv" files. These files are currently included only in icu-charset.jar.
• The time zone data is named zoneinfo.res under the directory icudt42b.

Some of the data files alias or otherwise reference data from other data files. One reason for this is because some locale names have changed. For example, he_IL used to be iw_IL. In order to support both names but not duplicate the data, one of the resource files refers to the other file's data. In other cases, a file may alias a portion of another file's data in order to save space. Currently ICU4J provides no tool for revealing these dependencies.

Note: Java's Locale class silently converts the language code "he" to "iw" when you construct the Locale (for versions of Java through Java 5). Thus Java cannot be used to locate resources that use the "he" language code. ICU, on the other hand, does not perform this conversion in ULocale, and instead uses aliasing in the locale data to represent the same set of data under different locale ids.

Resource files that use locale ids form a hierarchy, with up to four levels: a root, language, region (country), and variant. Searches for locale data attempt to match as far down the hierarchy as possible, for example, "he_IL" will match he_IL, but "he_US" will match he (since there is no US variant for he, and "xx_YY will match root (the default fallback locale) since there is no xx language code in the locale hierarchy. Again, see java.util.ResourceBundle for more information.

Currently ICU4J provides no tool for revealing these dependencies between data files, so trimming the data directly in the ICU4J project is a hit-or-miss affair. The key point when you remove data is to make sure to remove all dependencies on that data as well. For example, if you remove he.res, you need to remove he_IL.res, since it is lower in the hierarchy, and you must remove iw.res, since it references he.res, and iw_IL.res, since it depends on it (and also references he_IL.res).

Unfortunately, the jar tool in the JDK provides no way to remove items from a jar file. Thus you have to extract the resources, remove the ones you don't want, and then create a new jar file with the remining resources. See the jar tool information for how to do this. Before 'rejaring' the files, be sure to thoroughly test your application with the remaining resources, making sure each required resource is present.

Using additional resource files with ICU4J

Warning: Resource file formats can change across releases of ICU4J!

The format of ICU4J resources is not part of the API. Clients who develop their own resources for use with ICU4J should be prepared to regenerate them when they move to new releases of ICU4J.

We are still developing ICU4J's resource mechanism. Currently it is not possible to mix icu's new binary .res resources with traditional java-style .class or .txt resources. We might allow for this in a future release, but since the resource data and format is not formally supported, you run the risk of incompatibilities with future releases of ICU4J.

Resource data in ICU4J is checked in to the repository as a jar file containing the resource binaries, icudata.jar. This means that inspecting the contents of these resources is difficult. They currently are compiled from ICU4C .txt file data. You can view the contents of the ICU4C text resource files to understand the contents of the ICU4J resources.

The files in icudata.jar get extracted to com/ibm/icu/impl/data in the build directory when the 'core' target is built. Building the 'resources' target will force the resources to once again be extracted. Extraction will overwrite any corresponding resource files already in that directory.

Building ICU4J Resources from ICU4C

Generating Data from CLDR

1. Checkout CLDR. $cldr_root in the following steps is the root directory where the CLDR source files checked out.
2. Update $cldr_root/common to 'release-1-7-0' tag
3. Update $cldr_root/tools to 'release-1-7-0' tag
4. Checkout ICU4C with tag 'release-4-2'
5. Checkout ICU4J with tag 'release-4-2'
6. Build ICU4J
7. Build ICU4C
8. Change to $cldr_root/tools/java directory
9. Build CLDR using ant after pointing ICU4J_CLASSES env var to the newly build ICU4J
10. cd to $icu4c_root/source/data directory
11. Follow the instructions in the cldr-icu-readme.txt
12. Build ICU4C data from CLDR
13. Build ICU4J data from ICU4C data by following the procedures in $icu4c_root/source/data/icu4j-readme.txt
14. cd to $icu4j_root dir
15. Build and test icu4j

About ICU4J Time Zone

ICU4J 4.2 includes time zone data version 2009g, which is the latest one as of the release date. However, time zone data is frequently updated in response to changes made by local governments around the world. If you need to update the time zone data, please refer the ICU user guide topic Updating the Time Zone Data.

Starting with ICU4J 4.0, you can optionally configure ICU4J date and time service classes to use underlying JDK TimeZone implementation (see the ICU4J API reference TimeZone for the details). When this configuration is enabled, ICU's own time zone data won't be used and you have to get time zone data patches from the JRE vendor.

Where to Find More Information

http://www.icu-project.org/ is the home page of International Components for Unicode development project

http://www.ibm.com/software/globalization/icu/ is a pointer to general information about the International Components for Unicode hosted by IBM

http://www.ibm.com/software/globalization/ is a pointer to information on how to make applications global.

Submitting Comments, Requesting Features and Reporting Bugs

Your comments are important to making ICU4J successful. We are committed to investigate any bug reports or suggestions, and will use your feedback to help plan future releases.

To submit comments, request features and report bugs, please see ICU bug database information or contact us through the ICU Support mailing list. While we are not able to respond individually to each comment, we do review all comments.

Thank you for your interest in ICU4J!

Copyright © 2002-2010 International Business Machines Corporation and others. All Rights Reserved.
4400 North First Street, San José, CA 95193, USA