Roman,
> # This little script also demonstrates that Arabic rendering is not
> # that complicated after all (it makes you wonder why some
> software
> # companies are still asking hundreds of dollars from poor students
> # who just want to print their Arabic texts) and that even Perl 4 can
> # handle Unicode text in UTF-8 without any nifty new add-ons.
>
Adapting a program or OS to work with Arabic is not a simple matter of drawing
Arabic presentation forms and applying the Unicode BIDI algorithm. It
generally means both editing and line-wrapping Arabic text, and doing
everywhere with the same quality that English text is supported. Doing forms
alone can be done in a few pages of code and data, as you illustrate. For
examples of what needs to be done for editing, see:
http://www.ibm.com/java/education/international-text/index.html
> # Until the Unicode Consortium publishes its Unicode Technical
> # Report #9 (Bidirectional Algorithm Reference Implementation)
> # at http://www.unicode.org/unicode/reports/techreports.html
> # let us oversimplify things a bit and reverse everything:
>
The Bidirectional Algorithm is fully specified in the Unicode Standard,
Version 2.0 (plus online errata). You can order it online if you want, through
amazon.com or unicode.org.
If you are doing a simple reversal simply for illustration, that's not a
problem, but I want to make sure that your readers understand that that is not
a compliant use of Unicode rendering for Arabic and Hebrew.
> # Bitstream's Cyberbit font offers 57 of the other 466 optional
> # ligatures in the U+FB50 Arabic Presentation Forms-A block:
>
...
> # The following table lists the presentation variants of each
> # character. Each value from the U+0600 block means that the
> # necessary glyph variant has not been assigned a code in Unicode's
> # U+FA00 compatibility zone. You may want to insert your private
> # glyphs or approximation glyphs for them:
>
This is one of the items that makes full Arabic support more interesting. Each
font generally will have a different set of ligatures, and the font is also
not limited to the presentation forms that are encoded in Unicode. (The
particular set of ligatures that happens to be in Unicode is there because it
was in an earlier version of ISO 10646.) For examples of font technology that
supports arbitrary mappings from characters to glyphs, look at:
http://www.adobe.com/supportservice/devrelations/opentype/main.htm
http://fonts.apple.com/WhitePapers/GXvsOTLayout.html
http://www.truetype.demon.co.uk/ttgx.htm
Fonts fall into roughly two categories:
- simple fonts: those restricted to the set of ligatures encoded as
presentation forms in Unicode.
- complex fonts: those which have ligatures that are not associated with a
Unicode character, and for each ligature, contain a mapping between a sequence
of Unicode characters (possibly within a context) and the resultant glyph code
for the ligature in question.
This means that in order to do a comprehensive job, one would need to handle
both simple and complex fonts. For simple fonts, each font needs to be
queried to see exactly which ligatures it does have. For complex fonts, the
mappings have to be loaded and used in order to determine the correct
ligatures.
Roman Czyborra wrote:
> #!/usr/local/bin/perl
>
> # arabjoin - a simple filter to render Arabic text
> # © 1998-06-18 roman@czyborra.com
> # Freeware license at http://czyborra.com/
> # Latest version at http://czyborra.com/unicode/
> # PostScript printout at http://czyborra.com/unicode/arabjoin.ps.gz
>
> # This filter takes Arabic text (encoded in UTF-8 using the Unicode
> # characters from the U+0600 Arabic block in logical order) as input
> # and performs Arabic glyph joining on it and outputs a UTF-8 octet
> # stream that is no longer logically arranged but in a visual order
> # which gives readable results when formatted with a simple Unicode
> # renderer like Yudit that does not handle Arabic differently yet
> # but simply outputs all glyphs in left-to-right order.
>
> # This little script also demonstrates that Arabic rendering is not
> # that complicated after all (it makes you wonder why some software
> # companies are still asking hundreds of dollars from poor students
> # who just want to print their Arabic texts) and that even Perl 4 can
> # handle Unicode text in UTF-8 without any nifty new add-ons.
>
> # Usage examples:
>
> # echo "Ø£Ù?ÙÑاÙ? باÙÑØ1اÙÑÙÖ!" | arabjoin
> # prints !ﻢï»üïº*ï»åï»üïº*ïºë Ù?ï»*ﻫïºÉ
> # which is the Arabic version of "Hello world!"
>
> # | recode ISO-8859-6..UTF-8 | arabjoin | uniprint -f cyberbit.ttf
> # prints an Arabic mail of charset=iso-8859-6-i on your printer
>
> # | arabjoin | xviewer yudit
> # delegates an Arabic UTF-8 message to a better viewer
>
> # ftp://sunsite.unc.edu/pub/Linux/apps/editors/X/ has uniprint in yudit-1.0
> # ftp://ftp.iro.umontreal.ca/pub/contrib/pinard/pretest/ has recode-3.4g
> # http://czyborra.com/unicode/ has arabjoin
> # http://czyborra.com/unix/ has xviewer
> # http://www.bitstream.com/cyberbit.htm or
> # ftp://ccic.ifcss.org/pub/software/fonts/unicode/ms-win/ or
> # ftp://ftp.irdu.nus.sg/pub/language/bitstream/ has cyberbit.ttf
>
> # This is how we do it: First we learn the presentation forms of each
> # Arabic letter from the end of this script:
>
> while(<DATA>)
> {
> ($char, $_) = /^(\S+)\s+(\S+)/;
> ($isolated{$char},$final{$char},$medial{$char},$initial{$char}) =
> /([\xC0-\xFF][\x80-\xBF]+)/g;
> }
>
> # Then learn the (incomplete set of) transparent characters:
>
> foreach $char (split (" ", "
> Ù? Ùå Ù* Ù* Ù* Ù* Ù°
> Ûó Ûò Ûô Ûs Û? Ûú Ûü ÛÝ Û¡ Û¢ Û£ Û¤ Û§ Û¨ Ûª Û« Û¬ Û?"))
> {
> $transparent{$char}=1;
> }
>
> # Finally we can process our text:
>
> while (<>)
> {
> s/\n$//; # chop off the end of the line so it won't jump upfront
>
> @uchar = # UTF-8 character chunks
> /([\x00-\x7F]|[\xC0-\xFF][\x80-\xBF]+)/g;
>
> # We walk through the line of text and do contextual analysis:
>
> for ($i = $[; $i <= $#uchar; $i = $j)
> {
> for ($b=$uchar[$j=$i]; $transparent{$c=$uchar[++$j]};){};
>
> # The following assignment is the heart of the algorithm.
> # It reduces the Arabic joining algorithm described on
> # pages 6-24 to 6-26 of the Arabic character block description
> # in the Unicode 2.0 Standard to four lines of Perl:
>
> $uchar[$i] = $a && $final{$c} && $medial{$b}
> || $final{$c} && $initial{$b}
> || $a && $final{$b}
> || $isolated{$b}
> || $b;
>
> $a = $initial{$b} && $final{$c};
> }
>
> # Until the Unicode Consortium publishes its Unicode Technical
> # Report #9 (Bidirectional Algorithm Reference Implementation)
> # at http://www.unicode.org/unicode/reports/techreports.html
> # let us oversimplify things a bit and reverse everything:
>
> $_= join ('', reverse @uchar);
>
> # The following 8 obligatory LAM+ALEF ligatures are encoded in the
> # U+FE70 Arabic Presentation Forms-B block in Unicode's
> # compatibility zone:
>
> s/ïºÇï»ü/ﻵ/g;
> s/ïºÇï»Ý/ﻶ/g;
> s/ïºÑï»ü/ï»·/g;
> s/ïºÑï»Ý/ﻸ/g;
> s/ïºàï»ü/ï»1/g;
> s/ïºàï»Ý/ﻺ/g;
> s/ïº*ï»ü/ï»»/g;
> s/ïº*ï»Ý/ï»*/g;
>
> # Bitstream's Cyberbit font offers 57 of the other 466 optional
> # ligatures in the U+FB50 Arabic Presentation Forms-A block:
>
> s/ﻢïºó/ï°*/g;
> s/ï»2ï»ì/ï°2/g;
> s/ïº*ï»ü/ï°¿/g;
> s/ﺢï»ü/ï±*/g;
> s/ïº|ï»ü/ï±*/g;
> s/ﻢï»ü/ï±Ç/g;
> s/ï»°ï»ü/ï±É/g;
> s/ï»2ï»ü/ï±Ñ/g;
> s/ﻢﻧ/ï±*/g;
> s/ÙåÙë/ï±*/g;
> s/Ù*Ùë/ï±ü/g;
> s/Ù*Ùë/ï±Ý/g;
> s/Ù*Ùë/ﱡ/g;
> s/Ù*Ùë/ï±¢/g;
> s/ﺮïºí/ﱪ/g;
> s/ï»|ïºí/ï±?/g;
> s/ï»2ïºí/ﱯ/g;
> s/ﺮïºò/ï±°/g;
> s/ï»|ïºò/ï±3/g;
> s/ï»2ïºò/ï±µ/g;
> s/ï»2ﻨ/ï2*/g;
> s/ﺮﻴ/ï2ë/g;
> s/ï»|ï»´/ï2î/g;
> s/ïºÝïºë/ï2ú/g;
> s/ﺤïºë/ï2*/g;
> s/ﺨïºë/ï2*/g;
> s/ﻤïºë/ï2ü/g;
> s/ïºÝïºó/ï2¡/g;
> s/ﺤïºó/ï2¢/g;
> s/ﺨïºó/ï2£/g;
> s/ﻤïºó/ï2¤/g;
> s/ﻤïº?/ï2|/g;
> s/ﻤïºü/ï2¨/g;
> s/ﻤﺣ/ï2ª/g;
> s/ﻤﺧ/ï2¬/g;
> s/ﻤïº3/ï2°/g;
> s/ïºÝï»ü/ï3â/g;
> s/ﺤï»ü/ï3S/g;
> s/ﺨï»ü/ï3?/g;
> s/ﻤï»ü/ï3å/g;
> s/ﻬï»ü/ï3*/g;
> s/ïºÝﻣ/ï3*/g;
> s/ﺤﻣ/ï3*/g;
> s/ﺨﻣ/ï3*/g;
> s/ﻤﻣ/ï3ë/g;
> s/ïºÝﻧ/ï3í/g;
> s/ﺤﻧ/ï3ì/g;
> s/ﺨﻧ/ï3î/g;
> s/ﻤﻧ/ï3ï/g;
> s/ïºÝï»3/ï3s/g;
> s/ﺤï»3/ï3?/g;
> s/ﺨï»3/ï3ú/g;
> s/ﻤï»3/ï3*/g;
> s/ﺤﻤï»ü/ï¶à/g;
> s/ﻪï»Ýï»üïº*/ï·2/g;
> s/ﻢï»Ýïº3ï»?/ﻪﻴï»Ýï»?/g;
> s/ﻪï»üïº*ï»Ýïºü/ï»*ïºü/g;
>
> print "$_\n";
> }
>
> # The following table lists the presentation variants of each
> # character. Each value from the U+0600 block means that the
> # necessary glyph variant has not been assigned a code in Unicode's
> # U+FA00 compatibility zone. You may want to insert your private
> # glyphs or approximation glyphs for them:
>
> __END__
> Ø¡ ïº*
> Ø¢ ïº*ïºÇ
> Ø£ ïºÉïºÑ
> ؤ ïºÖïº?
> Ø¥ ïº?ïºà
> Ø| ïºâïºSïºåïº?
> ا ïº*ïº*
> ب ïº*ïº*ïºíïºë
> Ø© ïºìïºî
> ت ïºïïº?ïºòïºó
> Ø« ïºôïºsïºúïº?
> ج ïº*ïº*ïºÝïºü
> Ø? ﺡﺢﺤﺣ
> Ø® ﺥïº|ﺨﺧ
> د ﺩﺪ
> ذ ﺫﺬ
> ر �ﺮ
> Ø2 ﺯﺰ
> Ø3 ﺱïº2ﺴïº3
> ش ﺵﺶﺸﺷ
> ص ïº1ﺺïº*ﺻ
> ض ïº*ïº*ï»*ﺿ
> Ø· ï»*ï»Çï»Ñï»É
> ظ ï»Öï»?ï»àï»?
> Ø1 ï»âï»Sï»åï»?
> غ ï»*ï»*ï»*ï»*
> Ù* Ù*Ù*Ù*Ù*
> Ù* ï»ëï»íï»îï»ì
> ÙÇ ï»ïï»?ï»òï»ó
> ÙÉ ï»ôï»sï»úï»?
> ÙÑ ï»*ï»*ï»Ýï»ü
> ÙÖ ï»¡ï»¢ï»¤ï»£
> Ù? ﻥï»|ﻨﻧ
> Ù? ﻩﻪﻬﻫ
> Ùà ï»?ï»®
> Ùâ ﻯﻰ // ﯩﯨ
> ÙS ï»±ï»2ï»´ï»3
> Ù± ï?* // ï?ë
> Ù2 Ù2Ù2
> Ù3 Ù3Ù3
> Ù´ Ù´
> ٵ ٵٵ
> ٶ ٶٶ
> Ù· ï¯*Ù·
> ٸ ٸٸٸٸ
> Ù1 ï?|ï?§ï?©ï?¨
> Ùº ï?*ï?üï?¡ï?Ý
> Ù» ï?íï?ìï?ïï?î
> Ù* Ù*Ù*Ù*Ù*
> Ù* Ù*Ù*Ù*Ù*
> Ù* ï??ï?óï?ôï?ò
> ٿ ����
> Ú* ï?sï??ï?*ï?ú
> Ú* Ú*Ú*Ú*Ú*
> ÚÇ ÚÇÚÇÚÇÚÇ
> ÚÉ ï?¶ï?·ï?1ï?¸
> ÚÑ ï?2ï?3ï?µï?´
> ÚÖ ÚÖÚÖÚÖÚÖ
> Ú? ï?ºï?»ï?*ï?*
> Ú? ï?*ï?¿ï®*ï®*
> Úà ï®àï®â
> Úâ ÚâÚâ
> ÚS ÚSÚS
> Ú? Ú?Ú?
> Úå ï®Ñï®Ö
> Ú* ï®Çï®É
> Ú* ï®?ï®?
> Ú* Ú*Ú*
> Ú* Ú*Ú*
> Úë ï®åï®*
> Úí ÚíÚí
> Úì ÚìÚì
> Úî ÚîÚî
> Úï ÚïÚï
> Ú? ÚïÚ?
> Úó ÚóÚó
> Úò ï®Sï®?
> Úô ÚôÚô
> Ús ÚsÚsÚsÚs
> Ú? Ú?Ú?Ú?Ú?
> Úú ÚúÚúÚúÚú
> Ú* Ú*Ú*Ú*Ú*
> Ú* Ú*Ú*Ú*Ú*
> Úü ÚüÚüÚüÚü
> ÚÝ ÚÝÚÝÚÝÚÝ
> Ú¡ Ú¡Ú¡Ú¡Ú¡
> Ú¢ Ú¢Ú¢Ú¢Ú¢
> Ú£ Ú£Ú£Ú£Ú£
> Ú¤ ï?ªï?«ï??ï?¬
> Ú¥ Ú¥Ú¥Ú¥Ú¥
> Ú| ï?®ï?¯ï?±ï?°
> Ú§ Ú§Ú§Ú§Ú§
> Ú¨ Ú¨Ú¨Ú¨Ú¨
> Ú© ï®*ï®*ï®ëï®*
> Úª ÚªÚªÚªÚª
> Ú« Ú«Ú«Ú«Ú«
> Ú¬ Ú¬Ú¬Ú¬Ú¬
> Ú? ï¯ìï¯îï¯?ï¯ï
> Ú® Ú®Ú®Ú®Ú®
> Ú¯ ï®íï®ìï®ïï®î
> Ú° Ú°Ú°Ú°Ú°
> Ú± ï®sï®?ï®*ï®ú
> Ú2 Ú2Ú2Ú2Ú2
> Ú3 ï®?ï®óï®ôï®ò
> Ú´ Ú´Ú´Ú´Ú´
> Úµ ÚµÚµÚµÚµ
> Ú¶ Ú¶Ú¶Ú¶Ú¶
> Ú· Ú·Ú·Ú·Ú·
> Úº ï®*ï®üÚºÚº
> Ú» ï®Ýﮡﮣﮢ
> Ú* Ú*Ú*Ú*Ú*
> Ú* Ú*Ú*Ú*Ú*
> Ú* ﮪﮫï®?ﮬ
> Û* ﮤﮥ
> Û* ï®|ﮧﮩﮨ
> ÛÇ ÛÇÛÇ
> ÛÉ ÛÉÛÉ
> ÛÑ ÛÑÛÑ
> ÛÖ ï¯Ýﯡ
> Û? ï¯ôï¯s
> Û? ï¯óï¯ò
> Ûà ï¯?ï¯ú
> Ûâ ﯢﯣ
> ÛS ÛSÛS
> Û? ï¯*ï¯ü
> Ûå ï¯*ï¯*ﯿï¯*
> Û* Û*Û*
> Û* Û*Û*Û*Û*
> Û* ﯤﯥﯧï¯|
> Û* Û*Û*Û*Û*
> ÛÇ ÛÇÛÇ
> ÛÉ ÛÉÛÉ
> ÛÑ ÛÑÛÑ
> ÛÖ ÛÖÛÖ
> Û? Û?Û?
> Û? Û?Û?
> Ûà ÛàÛà
> Ûâ ÛâÛâ
> ÛS ÛSÛS
> Û? Û?Û?
> Ûå ÛåÛåÛåÛå
> Û* Û*Û*
> Û* Û*Û*Û*Û*
> Û* Û*Û*Û*Û*
> Ûë ÛëÛëÛëÛë
> Ûí ﮮﮯ
> Ûì ï®°ï®±
> Ûï Ûï
> â** â**â**â**â**
This archive was generated by hypermail 2.1.2 : Tue Jul 10 2001 - 17:20:40 EDT