gfx/jpeg.h

Why a plugin needs it after all

png.h said no to JPEG, and for the art a plugin is BUILT from it was right: flat colour and hard edges are what PNG is for and what JPEG is worst at. What changed is everything else that arrives. A sampler shows the cover of the library it loaded. A preset browser shows the photograph the sound designer attached. A skin made by a user is a photograph of wood or of brushed metal, and every one of those is a JPEG because a camera or a photo editor made it. Refusing them sends a plugin author to vendor a decoder, and the vendored one is the code nobody here has read.

What is read

• baseline and extended sequential (SOF0, SOF1) and progressive (SOF2) DCT, 8 bits per sample, Huffman coded -- every JPEG a camera, a phone, a browser or an image editor writes;
• any sampling factors from 1 to 4: 4:4:4, 4:2:2, 4:2:0, 4:4:0, 4:1:1 and the odd ones, interleaved or one component per scan;
• restart markers, at any interval, in either kind of scan;
• greyscale, YCbCr, RGB (Adobe transform 0, or components named R, G, B), CMYK and YCCK as Photoshop writes them -- inverted, which is how the browsers read every CMYK JPEG whatever its markers say;
• the EXIF orientation, applied by default -- a phone stores its portraits on their side and says so in a tag, and an image drawn without reading it is a picture of somebody lying down -- and reported either way.

Refused by name: arithmetic coding, lossless and hierarchical JPEG, 12-bit samples, and a height left for a DNL marker to give later. Each is rare enough that a decoder carrying it is mostly carrying risk.

The same pixels as libjpeg

What a user compares against is how the image looks everywhere else, and everywhere else is libjpeg. So the arithmetic is libjpeg's, not merely a correct DCT: its integer IDCT (jidctint.c, 13-bit constants, two passes, its range limiting), its "fancy" upsampling for 2:1 chroma -- a triangle filter, three parts the nearer sample to one part the further, with the rounding biases alternating so the image does not drift -- and its fixed-point YCbCr to RGB. A JPEG through this and through Pillow comes out the same to the byte on every fixture in tests/data/images; the test reports the largest difference and asserts it small rather than zero, so a libjpeg that changes its rounding one day does not fail the build.

Every length in the file is attacker-controlled

As in png.h. Three limits matter. The pixel count is capped before anything is allocated, as there. The header's block count is held against the file's size: every block costs at least one bit of DC data, so a 200-byte file claiming 8000x8000 is refused rather than given 400 MB of coefficients. And the scans are counted, because a progressive file is a list of passes over the whole image and nothing in the format stops it repeating one forever. Entropy data that runs out is not an error: what was decoded is kept, the rest is grey, and the result says so in warning -- the same as libjpeg, and what a half-downloaded photograph wants.