devutils/ATFQ/book ATFQ git

Part I · How it works

12.2 Locales: why sorting, dates and numbers change

alsolocaleslc_alllc_ctypelc_collatelc_numericlc_timelocale-genlocalectlcollationsetlocalecannot change localec.utf-8posix locale

TL;DR

  • A locale bundles cultural rules: how bytes become characters (LC_CTYPE), sort order (LC_COLLATE), number and date formats, message language. Set by LANG, overridden by LC_*, all overridden by LC_ALL.
  • The same command can give different results in another locale: sort order, 3,14 vs 3.14, month names.
  • In scripts, pin it: LC_ALL=C for byte order and plain ASCII rules, or C.UTF-8 to keep UTF-8.
locale                       # current settings
locale -a                    # locales installed here
LC_ALL=C sort file.txt       # sort by bytes, the same everywhere

On your machine

alice@laptopexample
$ atfq 12.2
  Session   LANG=en_US.UTF-8
  Text      read as UTF-8 (en_US.UTF-8)
  System    LANG=fr_FR.UTF-8 for new sessions
  Installed C.utf8, and those in locale-archive

An example: Alice's laptop, one of the test machines. In your terminal, this block shows your own machine, read live.

Pitfalls

Deep dive 2 questions

Why does sort order depend on the locale?

Human sorting rules differ: ignoring case and accents at first, then breaking ties; Swedish puts ä after z, German next to a. The C locale skips all that and compares bytes, which is fast and the same everywhere, but puts Zebra before apple. Two programs sorting in different locales can disagree, which breaks tools like join and comm that expect sorted input.

Where do locales come from?

Locale definitions are compiled into binary files, one per locale or all in /usr/lib/locale/locale-archive. Debian and Arch compile only those listed in /etc/locale.gen; Fedora ships them in language packs. C.UTF-8 is built into recent glibc and always available.

Commands
locale localectl sort
Paths
/etc/locale.conf /etc/default/locale /etc/locale.gen
See also
12.1 Bytes and characters: ASCII, Unicode and UTF-8
27.1 Where environment variables come from
28.9 sort, uniq and cut