gfx/text_shaper.h

What shaping is, and why a glyph per character is not enough

Arabic letters take one of four forms according to their neighbours, and the font holds each form as its own glyph, chosen through its GSUB table. Lam followed by alef is one glyph, required. A vowel mark sits on its letter at a point the font's GPOS table gives, not after it. Latin fonts carry ligatures and contextual alternates the same way. Drawn a glyph per character, Arabic is a row of unjoined isolated letters with their marks floating beside them -- unreadable to anybody who reads Arabic.

What is here

GSUB and GPOS, every lookup type -- single, multiple, alternate, ligature, contextual and chained contextual in all three formats, extension, reverse chaining; single and pair positioning, cursive attachment, mark to base, to ligature and to mark -- with lookup flags, GDEF glyph classes, mark attachment classes and mark filtering sets. Features are chosen and applied in the stages HarfBuzz uses, so the output is HarfBuzz's: the tests compare glyph by glyph against it. Arabic (and the scripts that join like it) get the joining analysis that picks each letter's form; Thai and Lao their SARA AM; Hangul its syllables composed or decomposed to jamo; Devanagari, Bengali, Gurmukhi, Gujarati, Oriya, Tamil, Telugu, Kannada and Malayalam their syllables found, reordered -- reph, half forms, below and post forms, pre-base matras -- and shaped stage by stage, as are Khmer's and Myanmar's; Balinese, Javanese, Tibetan, Sinhala, Mongolian, N'Ko and the eighty-odd other scripts HarfBuzz sends to its Universal Shaping Engine go through that; Hebrew, Latin, Greek, Cyrillic and the rest get the default shaper. Unicode normalisation as HarfBuzz does it: decomposed where the font lacks a character, marks in canonical order, recomposed where it has the composite.

Not here: HarfBuzz's fallbacks for an Arabic font with no GSUB (shaping from the presentation forms) and for Syriac's stretched abbreviation mark, and the AAT tables (morx, kerx) of Apple's own fonts.

HarfBuzz

The shapers for Thai and Lao, Hangul, the Indic scripts, Khmer, Myanmar and the Universal Shaping Engine, the syllable grammars and the Indic and Universal categories follow HarfBuzz (harfbuzz.github.io) closely -- its algorithms, rebuilt here, and its grammars and tables, regenerated by scripts/unicode. HarfBuzz is under the "Old MIT" licence, whose notice follows:

Copyright (c) 2010-2022 Google, Inc.; 2015-2020 Ebrahim Byagowi;
2019,2020 Facebook, Inc.; 2012,2015 Mozilla Foundation; 2011 Codethink
Limited; 2008,2010 Nokia Corporation and/or its subsidiary(-ies); 2009
Keith Stribley; 2011 Martin Hosken and SIL International; 2007 Chris
Wilson; 2005,2006,2020,2021,2022,2023 Behdad Esfahbod;
2004,2007,2008,2009,2010,2013,2021,2022,2023 Red Hat, Inc.; 1998-2005
David Turner and Werner Lemberg; 2016 Igalia S.L.; 2022 Matthias
Clasen; 2018,2021 Khaled Hosny; 2018,2019,2020 Adobe, Inc; 2013-2015
Alexei Podtelezhnikov.

Permission is hereby granted, without written agreement and without
license or royalty fees, to use, copy, modify, and distribute this
software and its documentation for any purpose, provided that the
above copyright notice and the following two paragraphs appear in
all copies of this software.

IN NO EVENT SHALL THE COPYRIGHT HOLDER BE LIABLE TO ANY PARTY FOR
DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES
ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN
IF THE COPYRIGHT HOLDER HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
DAMAGE.

THE COPYRIGHT HOLDER SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING,
BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
FITNESS FOR A PARTICULAR PURPOSE.  THE SOFTWARE PROVIDED HEREUNDER IS
ON AN "AS IS" BASIS, AND THE COPYRIGHT HOLDER HAS NO OBLIGATION TO
PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.

Microsoft's list of invalid Indic clusters and its additions to the Indic categories (IndicShapingInvalidCluster.txt, IndicSyllabicCategory- and IndicPositionalCategory-Additional.txt), (c) Microsoft Corporation, are under the MIT licence: see scripts/unicode/ms-use/COPYING.