ISO/IEC 8859

From Wikipedia, the free encyclopedia

ISO 8859, more formally ISO/IEC 8859, is a joint ISO and IEC standard for 8-bit character encodings for use by computers. The standard is divided into numbered, separately published parts, such as ISO/IEC 8859-1, ISO/IEC 8859-2, etc., each of which may be informally referred to as a standard in itself. There are currently 15 parts as of 2006 excluding the abandoned ISO/IEC 8859-12 standard.

ISO/IEC 8859 parts 1, 2, 3 and 4 are also Ecma International ECMA-94.

Contents

[edit] Introduction

While the bit patterns of the 95 printable ASCII characters are sufficient to exchange information in modern English, most other languages that use the Latin alphabet need additional symbols not covered by ASCII, such as ß (German), ñ (Spanish), å (Swedish and other Nordic languages) and ő (Hungarian). ISO 8859 sought to remedy this problem by utilizing the eighth bit in an 8-bit byte in order to allow positions for another 128 characters. (This bit was previously used for data transmission protocol information, or was left unused.) However, more characters were needed than could fit in a single 8-bit character encoding, so several mappings were developed, including at least 10 just to cover the Latin script.

The ISO 8859-n encodings only contain printable characters, and were designed to be used in conjunction with control characters mapped to the unassigned bytes. To this end a series of encodings registered with the IANA add the C0 control set (control characters mapped to bytes 0 to 31) from ISO 646 and the C1 control set (control characters mapped to bytes 127 to 159) from ISO 6429, resulting in full 8-bit character maps with most, if not all, bytes assigned. These sets have ISO-8859-n as their preferred MIME name or, in cases where a preferred MIME name isn't specified, their canonical name. Many people use the terms ISO 8859-n and ISO-8859-n interchangeably. ISO 8859-11 did not get such a charset assigned, presumably because it was almost identical to TIS 620.

[edit] Characters

The ISO 8859 standard is designed for reliable information exchange, not typography; the standard omits symbols needed for high-quality typography, such as optional ligatures, curly quotation marks, dashes, etc. As a result, high-quality typesetting systems often use proprietary or idiosyncratic extensions on top of the ASCII and ISO 8859 standards, or use Unicode instead.

As a rule of thumb, if a character or symbol was not already part of a widely used data-processing character set and was also not usually provided on typewriter keyboards for a national language, it didn't get in. Hence the directional double quotation marks « and » used for some European languages were included, but not the directional double quotation marks “ and ” used for English and some other languages. French didn't get its œ and Œ ligatures because they could be typed as 'oe'. Ÿ, needed for all-caps text, was left out as well. These characters were, however, included later with ISO 8859-15, which also introduced the new euro sign character €. Likewise Dutch did not get the 'ij' and 'IJ' letters, because Dutch speakers had gotten used to typing these as two letters instead. Romanian did not initially get its 'Ș/ș' and 'Ț/ț' (with comma) letters, because these letters were initially unified with 'Ş/ş' and 'Ţ/ţ' (with cedilla) by the Unicode Consortium, considering the shapes with comma beneath to be glyph variants of the shapes with cedilla. However, the letters with explicit comma below were later added to the Unicode standard and are also in ISO 8859-16.

Most of the ISO 8859 encodings provide diacritic marks required for various European languages using the Latin script. Others provide non-Latin alphabets: Greek, Cyrillic, Hebrew, Arabic and Thai. Most of the encodings contain only spacing characters although the Thai, Hebrew, and Arabic ones do also contain combining characters. However, the standard makes no provision for the scripts of East Asian languages (CJK), as their ideographic writing systems require many thousands of code points. Although it uses Latin based characters, Vietnamese does not fit into 96 positions (without using combining diacritics) either. Each Japanese syllabic alphabet (hiragana or katakana, see Kana) would fit, but like several other alphabets of the world they aren't encoded in the ISO 8859 system.

[edit] The Parts of ISO 8859

ISO 8859 is divided into the following parts:

Part 1 Latin-1
Western European		 Perhaps the most widely used part of ISO 8859, covering most Western European languages: Danish, Dutch (partial[1]), English, Faeroese, Finnish (partial[2]), French (partial[2]), German, Icelandic, Irish, Italian, Norwegian, Portuguese, Rhaeto-Romanic, Scottish Gaelic, Spanish, and Swedish. Languages from other parts of the world are also covered, including: Eastern European Albanian, Southeast Asian Indonesian, as well as the African languages Afrikaans and Swahili. The missing euro sign and capital Ÿ are in the revised version ISO 8859-15. The corresponding IANA-approved character set ISO-8859-1 is the default encoding for legacy HTML documents and for documents transmitted via MIME messages, such as HTTP responses when the document's media type is "text" (as in "text/html").
Part 2 Latin-2
Central European		 Supports those Central and Eastern European languages that use the Latin alphabet, including Bosnian, Polish, Croatian, Czech, Slovak, Slovenian, Serbian, and Hungarian. The missing euro sign can be found in version ISO 8859-16.
Part 3 Latin-3
South European		 Turkish, Maltese, and Esperanto. Largely superseded by ISO 8859-9 for Turkish and Unicode for Esperanto.
Part 4 Latin-4
North European		 Estonian, Latvian, Lithuanian, Greenlandic, and Sami.
Part 5 Latin/Cyrillic Covers mostly Slavic languages that use a Cyrillic alphabet, including Belarusian, Bulgarian, Macedonian, Russian, Serbian, and Ukrainian (partial[3]).
Part 6 Latin/Arabic Covers the most common Arabic language characters. Doesn't support other languages using the Arabic script. Needs to be BiDi and cursive joining processed for display.
Part 7 Latin/Greek Covers the modern Greek language (monotonic orthography). Can also be used for Ancient Greek written without accents or in monotonic orthography, but lacks the diacritics for polytonic orthography. These were introduced with Unicode.
Part 8 Latin/Hebrew Covers the modern Hebrew alphabet as used in Israel. In practice two different encodings exist, logical order (needs to be BiDi processed for display) and visual (left-to-right) order (in effect, after bidi processing and line breaking).
Part 9 Latin-5
Turkish		 Largely the same as ISO 8859-1, replacing the rarely used Icelandic letters with Turkish ones. It is also used for Kurdish.
Part 10 Latin-6
Nordic		 a rearrangement of Latin-4. Considered more useful for Nordic languages. Baltic languages use Latin-4 more.
Part 11 Latin/Thai Contains most characters needed for the Thai language. Same as TIS 620.
non-existent
Part 12		 Latin/Devanagari The work in making a part of 8859 for Devanagari was officially abandoned in 1997. ISCII and Unicode/ISO/IEC 10646 cover Devanagari.
Part 13 Latin-7
Baltic Rim		 Added some characters for Baltic languages which were missing from Latin-4 and Latin-6.
Part 14 Latin-8
Celtic		 Covers Celtic languages such as Gaelic and the Breton language.
Part 15 Latin-9 A revision of 8859-1 that removes some little-used symbols, replacing them with the euro sign € and the letters Š, š, Ž, ž, Œ, œ, and Ÿ, which completes the coverage of French, Finnish and Estonian.
Part 16 Latin-10
South-Eastern European		 Intended for Albanian, Croatian, Hungarian, Italian, Polish, Romanian and Slovenian, but also Finnish, French, German and Irish Gaelic (new orthography). The focus lies more on letters than symbols. The currency sign is replaced with the euro sign.
  1. ^ only the IJ/ij (letter IJ) is missing, which is usually represented as IJ.
  2. ^ a b missing characters are in ISO 8859-15.
  3. ^ missing Ґ/ґ characters were reintroduced into Ukrainian in 1991.

Each part of ISO 8859 is designed to support languages that often borrow from each other, so the characters needed by each language are usually accommodated by a single part. However, there are some characters and language combinations that are not accommodated without transcriptions. Efforts were made to make conversions as smooth as possible. For example, German has all of its seven special characters at the same positions in all Latin variants (1-4, 9-10, 13-16), and in many positions the characters only differ in the diacritics between the sets. In particular, variants 1-4 were designed jointly, and have the property that every encoded character appears either at a given position or not at all.

[edit] Table

Comparison of the various parts of ISO 8859
Binary Oct Dec Hex 1 2 3 4 5 6 7 8 9 10 11 13 14 15 16
1010 0000 240 160 A0 Non-breaking space (NBSP)
1010 0001 241 161 A1 ¡ Ą Ħ Ą Ё   ‘   ¡ Ą ก ” Ḃ ¡ Ą
1010 0010 242 162 A2 ¢ ˘ ĸ Ђ   ’ ¢ ¢ Ē ข ¢ ḃ ¢ ą
1010 0011 243 163 A3 £ Ł £ Ŗ Ѓ   £ Ģ ฃ £ Ł
1010 0100 244 164 A4 ¤ Є ¤ € ¤ Ī ค ¤ Ċ €
1010 0101 245 165 A5 ¥ Ľ   Ĩ Ѕ   ₯ ¥ Ĩ ฅ „ ċ ¥ „
1010 0110 246 166 A6 ¦ Ś Ĥ Ļ І   ¦ Ķ ฆ ¦ Ḋ Š
1010 0111 247 167 A7 § Ї   § ง §
1010 1000 250 168 A8 ¨ Ј   ¨ Ļ จ Ø Ẁ š
1010 1001 251 169 A9 © Š İ Š Љ   © Đ ฉ ©
1010 1010 252 170 AA ª Ş Ē Њ   ͺ × ª Š ช Ŗ Ẃ ª Ș
1010 1011 253 171 AB « Ť Ğ Ģ Ћ   « Ŧ ซ « ḋ «
1010 1100 254 172 AC ¬ Ź Ĵ Ŧ Ќ ، ¬ Ž ฌ ¬ Ỳ ¬ Ź
1010 1101 255 173 AD soft hyphen (SHY) ญ SHY
1010 1110 256 174 AE ® Ž   Ž Ў     ® Ū ฎ ® ź
1010 1111 257 175 AF ¯ Ż ¯ Џ   ― ¯ Ŋ ฏ Æ Ÿ ¯ Ż
1011 0000 260 176 B0 ° А   ° ฐ ° Ḟ °
1011 0001 261 177 B1 ± ą ħ ą Б   ± ą ฑ ± ḟ ±
1011 0010 262 178 B2 ² ˛ ² ˛ В   ² ē ฒ ² Ġ ² Č
1011 0011 263 179 B3 ³ ł ³ ŗ Г   ³ ģ ณ ³ ġ ³ ł
1011 0100 264 180 B4 ´ Д   ΄ ´ ī ด “ Ṁ Ž
1011 0101 265 181 B5 µ ľ µ ĩ Е   ΅ µ ĩ ต µ ṁ µ ”
1011 0110 266 182 B6 ś ĥ ļ Ж   Ά ķ ถ
1011 0111 267 183 B7 · ˇ · ˇ З   · ท · Ṗ ·
1011 1000 270 184 B8 ¸ И   Έ ¸ ļ ธ ø ẁ ž
1011 1001 271 185 B9 ¹ š ı š Й   Ή ¹ đ น ¹ ṗ ¹ č
1011 1010 272 186 BA º ş ē К   Ί ÷ º š บ ŗ ẃ º ș
1011 1011 273 187 BB » ť ğ ģ Л ؛ » ŧ ป » Ṡ »
1011 1100 274 188 BC ¼ ź ĵ ŧ М   Ό ¼ ž ผ ¼ ỳ Œ
1011 1101 275 189 BD ½ ˝ ½ Ŋ Н   ½ ― ฝ ½ Ẅ œ
1011 1110 276 190 BE ¾ ž   ž О   Ύ ¾ ū พ ¾ ẅ Ÿ
1011 1111 277 191 BF ¿ ż ŋ П ؟ Ώ   ¿ ŋ ฟ æ ṡ ¿ ż
1100 0000 300 192 C0 À Ŕ À Ā Р   ΐ   À Ā ภ Ą À
1100 0001 301 193 C1 Á С ء Α   Á ม Į Á
1100 0010 302 194 C2 Â Т آ Β   Â ย Ā Â
1100 0011 303 195 C3 Ã Ă   Ã У أ Γ   Ã ร Ć Ã Ă
1100 0100 304 196 C4 Ä Ф ؤ Δ   Ä ฤ Ä
1100 0101 305 197 C5 Å Ĺ Ċ Å Х إ Ε   Å ล Å Ć
1100 0110 306 198 C6 Æ Ć Ĉ Æ Ц ئ Ζ   Æ ฦ Ę Æ
1100 0111 307 199 C7 Ç Į Ч ا Η   Ç Į ว Ē Ç
1100 1000 310 200 C8 È Č È Č Ш ب Θ   È Č ศ Č È
1100 1001 311 201 C9 É Щ ة Ι   É ษ É
1100 1010 312 202 CA Ê Ę Ê Ę Ъ ت Κ   Ê Ę ส Ź Ê
1100 1011 313 203 CB Ë Ы ث Λ   Ë ห Ė Ë
1100 1100 314 204 CC Ì Ě Ì Ė Ь ج Μ   Ì Ė ฬ Ģ Ì
1100 1101 315 205 CD Í Э ح Ν   Í อ Ķ Í
1100 1110 316 206 CE Î Ю خ Ξ   Î ฮ Ī Î
1100 1111 317 207 CF Ï Ď Ï Ī Я د Ο   Ï ฯ Ļ Ï
1101 0000 320 208 D0 Ð Đ   Đ а ذ Π   Ğ Ð ะ Š Ŵ Ð
1101 0001 321 209 D1 Ñ Ń Ñ Ņ б ر Ρ   Ñ Ņ ั Ń Ñ Ń
1101 0010 322 210 D2 Ò Ň Ò Ō в ز     Ò Ō า Ņ Ò
1101 0011 323 211 D3 Ó Ķ г س Σ   Ó ำ Ó
1101 0100 324 212 D4 Ô д ش Τ   Ô ิ Ō Ô
1101 0101 325 213 D5 Õ Ő Ġ Õ е ص Υ   Õ ี Ő
1101 0110 326 214 D6 Ö ж ض Φ   Ö ึ Ö
1101 0111 327 215 D7 × з ط Χ   × Ũ ื × Ṫ × Ś
1101 1000 330 216 D8 Ø Ř Ĝ Ø и ظ Ψ   Ø ุ Ų Ø Ű
1101 1001 331 217 D9 Ù Ů Ù Ų й ع Ω   Ù Ų ู Ł Ù
1101 1010 332 218 DA Ú к غ Ϊ   Ú ฺ Ś Ú
1101 1011 333 219 DB Û Ű Û л   Ϋ   Û   Ū Û
1101 1100 334 220 DC Ü м   ά   Ü   Ü
1101 1101 335 221 DD Ý Ŭ Ũ н   έ   İ Ý   Ż Ý Ę
1101 1110 336 222 DE Þ Ţ Ŝ Ū о   ή   Ş Þ   Ž Ŷ Þ Ț
1101 1111 337 223 DF ß п   ί ‗ ß ฿ ß
1110 0000 340 224 E0 à ŕ à ā р ـ ΰ א à ā เ ą à
1110 0001 341 225 E1 á с ف α ב á แ į á
1110 0010 342 226 E2 â т ق β ג â โ ā â
1110 0011 343 227 E3 ã ă   ã у ك γ ד ã ใ ć ã ă
1110 0100 344 228 E4 ä ф ل δ ה ä ไ ä
1110 0101 345 229 E5 å ĺ ċ å х م ε ו å ๅ å ć
1110 0110 346 230 E6 æ ć ĉ æ ц ن ζ ז æ ๆ ę æ
1110 0111 347 231 E7 ç į ч ه η ח ç į ็ ē ç
1110 1000 350 232 E8 è č è č ш و θ ט è č ่ č è
1110 1001 351 233 E9 é щ ى ι י é ้ é
1110 1010 352 234 EA ê ę ê ę ъ ي κ ך ê ę ๊ ź ê
1110 1011 353 235 EB ë ы ً λ כ ë ๋ ė ë
1110 1100 354 236 EC ì ě ì ė ь ٌ μ ל ì ė ์ ģ ì
1110 1101 355 237 ED í э ٍ ν ם í ํ ķ í
1110 1110 356 238 EE î ю َ ξ מ î ๎ ī î
1110 1111 357 239 EF ï ď ï ī я ُ ο ן ï ๏ ļ ï
1111 0000 360 240 F0 ð đ   đ ȑ ِ π נ ğ ð ๐ š ŵ ð đ
1111 0001 361 241 F1 ñ ń ñ ņ ё ّ ρ ס ñ ņ ๑ ń ñ ń
1111 0010 362 242 F2 ò ň ò ō ђ ْ ς ע ò ō ๒ ņ ò
1111 0011 363 243 F3 ó ķ ѓ   σ ף ó ๓ ó
1111 0100 364 244 F4 ô є   τ פ ô ๔ ō ô
1111 0101 365 245 F5 õ ő ġ õ ѕ   υ ץ õ ๕ ő
1111 0110 366 246 F6 ö і   φ צ ö ๖ ö
1111 0111 367 247 F7 ÷ ї   χ ק ÷ ũ ๗ ÷ ṫ ÷ ś
1111 1000 370 248 F8 ø ř ĝ ø ј   ψ ר ø ๘ ų ø ű
1111 1001 371 249 F9 ù ů ù ų љ   ω ש ù ų ๙ ł ù
1111 1010 372 250 FA ú њ   ϊ ת ú ๚ ś ú
1111 1011 373 251 FB û ű û ћ   ϋ   û ๛ ū û
1111 1100 374 252 FC ü ќ   ό   ü   ü
1111 1101 375 253 FD ý ŭ ũ §   ύ LRM ı ý   ż ý ę
1111 1110 376 254 FE þ ţ ŝ ū ў   ώ RLM ş þ   ž ŷ þ ț
1111 1111 377 255 FF ÿ ˙ џ       ÿ ĸ   ’ ÿ

At position 0xA0 there's always the non breaking space and 0xAD is mostly the soft hyphen, which only shows at line breaks. Other empty fields are either unassigned or the system used isn't able to display them.

There are new additions as ISO/IEC 8859-7:2003 and ISO/IEC 8859-8:1999 versions. LRM stands for left-to-right mark (U+200E) and RLM stands for right-to-left mark (U+200F).

[edit] Relationship to Unicode and the UCS

Since 1991, the Unicode Consortium has been working with ISO to develop the Unicode Standard and ISO/IEC 10646: the Universal Character Set (UCS) in tandem. This pair of standards was created to unify the ISO 8859 character repertoire, among others, by assigning each character, initially, to a 16-bit code value, with some code values left unassigned. Over time, their models adapted to map characters to abstract numeric code points rather than fixed bit-width values, so that more code points and encoding methods could be supported.

Unicode and ISO/IEC 10646 currently assign about 100,000 characters to a code space consisting of over a million code points, and they define several standard encodings that are capable of representing every available code point. The standard encodings of Unicode and the UCS use sequences of one to four 8-bit code values (UTF-8), sequences of one or two 16-bit code values (UTF-16), or one 32-bit code value (UTF-32 or UCS-4). There is also an older encoding that uses one 16-bit code value (UCS-2), capable of representing one-seventeenth of the available code points. Of these encoding forms, only UTF-8's byte sequences are in a fixed order; the others are subject to platform-dependent byte ordering issues that may be addressed via special codes or indicated via out-of-band means.

Newer editions of ISO 8859 express characters in terms of their Unicode/UCS names and the U+nnnn notation, effectively causing each part of ISO 8859 to be a Unicode/UCS character encoding scheme that maps a very small subset of the UCS to single 8-bit bytes. The first 256 characters in Unicode and the UCS are identical to those in ISO-8859-1.

Single-byte character sets including the parts of ISO 8859 and derivatives of them were favored throughout the 1990s, having the advantages of being well-established and more easily implemented in software: the equation of one byte to one character is simple and adequate for most single-language applications, and there are no combining characters or variant forms.

As the relative cost, in computing resources, of using more than one byte per character began to diminish, programming languages and operating systems added native support for Unicode alongside their system of code pages. Windows NT was quite an early adopter of Unicode. However Unicode support in Windows 9x required linking with a special compatibility layer or restricting your design to a very small subset of the Windows API discouraging its use. As Unicode-enabled operating systems became more widespread, ISO 8859 and other legacy encodings became less popular. While remnants of ISO 8859 and single-byte character models remain entrenched in many operating systems, programming languages, data storage systems, networking applications, display hardware, and end-user application software, most modern computing applications use Unicode internally, and rely on conversion tables to map to and from other encodings, when necessary.

[edit] Development status

The ISO/IEC 8859 standard was maintained by ISO/IEC Joint Technical Committee 1, Subcommittee 2, Working Group 3 (ISO/IEC JTC 1/SC 2/WG 3). In June 2004, WG 3 disbanded, and maintenance duties were transferred to SC 2. The standard is not currently being updated, as the Subcommittee's only remaining working group, WG 2, is concentrating on development of ISO/IEC 10646.

[edit] References

  • ECMA standards, which in intent correspond exactly to the ISO/IEC 8859 character set standards, can be found at:
    • Standard ECMA-94: 8-Bit Single Byte Coded Graphic Character Sets - Latin Alphabets No. 1 to No. 4 2nd edition (June 1986)
    • Standard ECMA-113: 8-Bit Single-Byte Coded Graphic Character Sets - Latin/Cyrillic Alphabet 3rd edition (December 1999)
    • Standard ECMA-114: 8-Bit Single-Byte Coded Graphic Character Sets - Latin/Arabic Alphabet 2nd edition (December 2000)
    • Standard ECMA-118: 8-Bit Single-Byte Coded Graphic Character Sets - Latin/Greek Alphabet (December 1986)
    • Standard ECMA-121: 8-Bit Single-Byte Coded Graphic Character Sets - Latin/Hebrew Alphabet 2nd edition (December 2000)
    • Standard ECMA-128: 8-Bit Single-Byte Coded Graphic Character Sets - Latin Alphabet No. 5 2nd edition (December 1999)
    • Standard ECMA-144: 8-Bit Single-Byte Coded Character Sets - Latin Alphabet No. 6 3rd edition (December 2000)
  • ISO/IEC 8859-1 to Unicode mapping tables as plain text files are at the Unicode FTP site.