Occasional OOM while enumerating system fonts #3

I think it’s FOP that causes this. It sometimes runs out of memory while enumerating system fonts, presumably because there are quite a lot of them.

There doesn’t appear to be any way to tell it to only use specific fonts without hard-coding the path. Putting <auto-detect/> in the FOP config file seems to be the only way to get it to look in the system font directories (see https://xmlgraphics.apache.org/fop/2.1/fonts.html).

Possible solutions/workarounds:

• Write a JNI wrapper around librsvg — bleh. It also makes things less self-contained, although it isn’t really with the fonts already anyway. (See https://github.com/mkowsiak/jnicookbook.)
• Embed the fonts into this project’s resources and load them directly via a relative path. This can’t be done with the true official University fonts, however, as some of the them are proprietary (Minion Pro in particular). Crimson Pro is a reasonable alternative for Minion Pro, but that’s still going to be a few MB of binary font files in the repo — not ideal.
• Download the fonts on demand. There’s an old fonts plugin for Gradle (https://github.com/florent37/fonts) but it’s broken because the API URL changed. It might work well if the plugin can be fixed or re-written. The only roadblock is that Iosevka isn’t available through Google Fonts. This is probably the most workable solution overall, however.

In the meantime, I guess watch out for the java_pidXXXXX.hprof files and delete them when they appear.

For reference, the following works for XeTeX with fontspec, assuming that the TTF files have been downloaded to ../buildSrc/build/resources/main/fonts/ (relative to the lecture directory):

\setsansfont{OpenSans}[
    Path=../buildSrc/build/resources/main/fonts/,
    Extension=.ttf,
    UprightFont=*-Light,
    BoldFont=*-Regular,
    ItalicFont=*-LightItalic,
    BoldItalicFont=*-Italic,
    Scale=0.912,
]

\setmonofont{iosevka-fixed-ss03}[
    Path=../buildSrc/build/resources/main/fonts/,
    Extension=.ttf,
    UprightFont=*-light,
    BoldFont=*-regular,
    ItalicFont=*-lightitalic,
    BoldItalicFont=*-italic,
    Scale=MatchUppercase,
]

(If the TTF files are in ../buildSrc/src/main/resources/fonts/ then a gradle build in buildSrc will copy them into the build resources directory.)

This is using the fonts as downloaded directly from their respective repositories.

One issue is that not all of the fonts are required (this is particularly true of Iosevka, which has something like 54 different weights, styles, etc.). A solution might be to create a repository that includes just the files required and download from that.

For Iosevka, use the unhinted SS03 “fixed” version, as that doesn’t include ligatures by default and is much smaller overall (e.g., https://github.com/be5invis/Iosevka/releases/download/v17.1.0/ttf-unhinted-iosevka-fixed-ss03-17.1.0.zip).

Argh, FOP is such a POS when it comes to custom fonts.

• <auto-detect /> works, but occasionally the JVM runs out of memory because of the sheer number of system fonts, where “font” has it's proper meaning of “a specific variant of a type face” (e.g., Open Sans is at least 12 different fonts). It auto-detects something like 3,500 individual fonts on my system.

• Specifying embedded fonts inside a <renderer> element as specified in the documentation doesn’t work. The fonts aren’t even registered and I’m not entirely convinced the config is properly parsed. After much experimentation and debugging and poking around in the XSD for the config file, specifying the fonts in a top-level <fonts> element works.

• The default for <base> is not where the FOP config file is, rather it’s the “execution” base directory, usually either the root of the app or jar (but see below for Gradle). You therefore need to specify the full relative path to the fonts directory when loading a font file. This also works with a <directory> element, which simplifies things, but it does yell a lot about something that I can't quite determine:

  INFO: ignoring out of order or duplicate glyph index: 168
  WARNING: coverage set class table not yet supported

  Specifying the fonts individually doesn't cause this (?!).

• <base> and <font-base> seem to be ignored (at least for the top-level <fonts> element), e.g., even when <font-base> is the full relative path:

  java.io.FileNotFoundException: /Users/nstanger/tmp/pdftest/app/OpenSans-Regular.ttf (No such file or directory)

Unfortunately the whole thing breaks completely under a Gradle daemon, as the base path is set to the gradle daemon home ~/.gradle/daemon/<version> rather than the jar root. Running without a daemon sets the base path to the directory containing the current build.gradle, so you can at least set the relative path to point to ../buildSrc, but you have to run Gradle without a daemon, which seems surprisingly more difficult than it should be. This works (https://github.com/gradle/gradle/issues/11517#issuecomment-602632415), but is a pain to use:

GRADLE_OPTS="-XX:MaxMetaspaceSize=256m -XX:+HeapDumpOnOutOfMemoryError -Xmx512m" gradle --no-daemon <target>

Someone also came up with a genius trick of munging the config on the fly (https://stackoverflow.com/a/72807651), but that still doesn’t work under Gradle, because the getResource() call returns a URI that points inside the jar and thus isn’t a real path 🤬.

The real problem is that the FOP config file has no concept of the current execution environment, so if something like Gradle munges that, everything breaks in exciting ways.

At this stage I can see two alternatives:

• Hack custom FOP classes — bleh. (See https://stackoverflow.com/a/54888834 for an example.)
• Switch to iText 7 and either sacrifice bitmap output, or refactor that into a separate class.