devutils/ATFQ/book ATFQ git

Part I · How it works

12. How text is stored

  1. 12.1Bytes and characters: ASCII, Unicode and UTF-8A file holds bytes. An encoding says which bytes mean which characters. ASCII covers 128 (English letters, digits, punctuation).
  2. 12.2Locales: why sorting, dates and numbers changeA locale bundles cultural rules: how bytes become characters (LC_CTYPE), sort order (LC_COLLATE), number and date formats, message language. Set by LANG, overridden by LC_*, all overridden by LC_ALL.
  3. 12.3Line endings, BOMs and invisible charactersLines end with \n (LF) on Linux and macOS, \r\n (CRLF) on Windows. The extra \r shows as ^M and breaks scripts.