Programming with Unicode

Todo

uppercase/lowercase

Todo

MySQL and PostgreSQL

Todo

Filesystems (ext2 vs VFAT)

• 1. About this book
  • 1.1. License
  • 1.2. Thanks to
  • 1.3. Notations
• 2. Unicode nightmare
• 3. Definitions
  • 3.1. Character
  • 3.2. Glyph
  • 3.3. Code point
  • 3.4. Character set (charset)
  • 3.5. Character string
  • 3.6. Byte string
  • 3.7. UTF-8 encoded strings and UTF-16 character strings
  • 3.8. Encoding
  • 3.9. Encode a character string
  • 3.10. Decode a byte string
  • 3.11. Unicode: an Universal Character Set (UCS)
• 4. Unicode
  • 4.1. Categories
  • 4.2. Statistics
  • 4.3. Normalization
• 5. Charsets and encodings
  • 5.1. Encodings
  • 5.2. Popularity
  • 5.3. Encodings performances
  • 5.4. Examples
  • 5.5. Handle undecodable bytes and unencodable characters
    • 5.5.1. Undecodable byte sequences
    • 5.5.2. Unencodable characters
    • 5.5.3. Error handlers
    • 5.5.4. Replace unencodable characters by a similar glyph
    • 5.5.5. Escape the character
  • 5.6. Other charsets and encodings
• 6. Historical charsets and encodings
  • 6.1. ASCII
  • 6.2. ISO 8859 family
    • 6.2.1. ISO 8859-1
    • 6.2.2. cp1252
    • 6.2.3. ISO 8859-15
  • 6.3. CJK: asian encodings
    • 6.3.1. Chinese encodings
    • 6.3.2. Japanese encodings
    • 6.3.3. ISO 2022
    • 6.3.4. Extended Unix Code (EUC)
  • 6.4. Cyrillic
• 7. Unicode encodings
  • 7.1. UTF-8
  • 7.2. UCS-2, UCS-4, UTF-16 and UTF-32
  • 7.3. UTF-7
  • 7.4. Byte order marks (BOM)
  • 7.5. UTF-16 surrogate pairs
• 8. How to guess the encoding of a document?
  • 8.1. Is ASCII?
  • 8.2. Check for BOM markers
  • 8.3. Is UTF-8?
  • 8.4. Libraries
• 9. Good practices
  • 9.1. Rules
  • 9.2. Unicode support levels
  • 9.3. Test the Unicode support of a program
  • 9.4. Get the encoding of your inputs
  • 9.5. Switch from byte strings to character strings
• 10. Operating systems
  • 10.1. Windows
    • 10.1.1. Code pages
    • 10.1.2. Encode and decode functions
    • 10.1.3. Windows API: ANSI and wide versions
    • 10.1.4. Windows string types
    • 10.1.5. Filenames
    • 10.1.6. Windows console
    • 10.1.7. File mode
  • 10.2. Mac OS X
  • 10.3. Locales
    • 10.3.1. Locale categories
    • 10.3.2. The C locale
    • 10.3.3. Locale encoding
    • 10.3.4. Locale functions
  • 10.4. Filesystems (filenames)
    • 10.4.1. CD-ROM and DVD
    • 10.4.2. Microsoft: FAT and NTFS filesystems
    • 10.4.3. Apple: HFS and HFS+ filesystems
    • 10.4.4. Others
• 11. Programming languages
  • 11.1. C language
    • 11.1.1. Byte API (char)
    • 11.1.2. Byte string API (char*)
    • 11.1.3. Character API (wchar_t)
    • 11.1.4. Character string API (wchar_t*)
    • 11.1.5. printf functions family
  • 11.2. C++
  • 11.3. Python
    • 11.3.1. Python 2
    • 11.3.2. Python 3
    • 11.3.3. Differences between Python 2 and Python 3
    • 11.3.4. Codecs
    • 11.3.5. String methods
    • 11.3.6. Filesystem
    • 11.3.7. Modules
  • 11.4. PHP
  • 11.5. Perl
  • 11.6. Java
  • 11.7. Go and D
• 12. Libraries
  • 12.1. Qt library
    • 12.1.1. Character and string classes
    • 12.1.2. Codec
    • 12.1.3. Filesystem
  • 12.2. The glib library
    • 12.2.1. Character strings
    • 12.2.2. Codec functions
    • 12.2.3. Filename functions
  • 12.3. iconv library
  • 12.4. ICU libraries
  • 12.5. libunistring
• 13. Unicode issues
  • 13.1. Mojibake
  • 13.2. Security vulnerabilities
    • 13.2.1. Special characters
    • 13.2.2. Non-strict UTF-8 decoder: overlong byte sequences and surrogates
    • 13.2.3. Check byte strings before decoding them to character strings
• 14. See also