painterly

painterly (adjective): of a painting or its style, characterised by visible brushwork and the rendering of form through colour and tone rather than by line. This is a program that takes a photo and paints it that way, in your browser, while you watch.

Live: davemaynard.github.io/painterly

A big brush lays in the underpainting. Finer brushes come back only where the picture still disagrees with the photo. Every stroke runs along the image instead of across it, so brushwork follows fur, edges and the line of a roof. It finishes in 45 seconds at any size — 20 for the underpainting — then stops.

What it does

• Paints coarse to fine. Five brushes, each half the size of the last. Each one blurs the photo to its own scale, measures where the canvas is still wrong, and puts a stroke down only there. The likeness converges; it doesn't fill in at random.
• Strokes follow the picture. Each step of a stroke moves perpendicular to the image gradient. That is the whole difference between brushwork and a smear filter.
• Same photo, same seed, same painting. All randomness is seeded, so a painting is reproducible on any machine, the timeline can be scrubbed to any moment, and the tests can compare pixels.
• Time to done, not time in app. The painting plays on a fixed 45-second clock, the underpainting fast and the finest brush slow, and then it is finished. There is a Download button and nothing else to do.
• Or stop at the underpainting. The Underpainting style plans the first, biggest brush only, with longer strokes and a looser threshold, and keeps the abstraction: a few hundred ribbons of colour that are unmistakably the photo and nothing like it. It was meant as a stage on the way to a painting and turned out to be the picture people liked.
• Your own photo. Drop one on the picture, or pick a file. It is painted in the tab and never uploaded anywhere.
• On a phone too. The bench unfolds, the photo is planned smaller so the wait stays a couple of seconds, and the transport still clears the fold.

Numbers

Run npm run size to regenerate the first two.

Library, gzipped	7.4 KB
Demo page script and its worker, gzipped	11.0 KB
Runtime dependencies	0 (a Mersenne Twister is bundled, see NOTICE)
Planning a 1050 × 1400 photo	~1.5 s on an M4 Mac mini, ~108,000 strokes
A plan crossing back from the worker	15 MB packed into typed arrays, not 110 MB of objects
Longest frame CI will accept	60 ms painting, 100 ms skipping to the end

Use it as a library

npm install github:davemaynard/painterly

import {brushes, createPainter, plan, rasterFromImageData} from 'painterly';

const context = canvas.getContext('2d');
if (!context) throw new Error('no 2d context');

context.drawImage(photo, 0, 0, canvas.width, canvas.height);
const source = rasterFromImageData(context.getImageData(0, 0, canvas.width, canvas.height));

// Pure data: every stroke, in painting order.
const painting = plan(source, {seed: 7});

// The hand: paints any prefix of the plan onto a 2D context.
const painter = createPainter(context, painting, brushes.bristle);
painter.paintTo(painting.strokes.length);

plan() takes options for the brush radii, the error threshold, stroke length and curvature, and an onLayer callback that hears from it after each brush; the defaults are tuned for photos around 1400 px. Plan and Stroke are plain typed objects, so a plan can be serialised, replayed, or rendered by something other than the bundled painter. packPlan() and unpackPlan() turn a plan into a handful of typed arrays and back, exactly: the form to post from a worker or keep around, at a seventh of the memory.

How it's built

• src/image/ measures the photo: a float RGB raster, a linear-time Gaussian blur (three box blurs), luminance and a Sobel gradient. No canvas, no DOM.
• src/plan/ decides the strokes, and names the two styles. Aaron Hertzmann's Painterly Rendering with Curved Brush Strokes of Multiple Sizes (SIGGRAPH 1998), ported and credited in the module header, with three habits kept from the 2020 sketches: a canvas primed with the photo's dominant hue family, a shuffled stroke order so the hand looks human, and seeded randomness.
• src/paint/ is the hand: a Brush says how many bristles a stroke becomes, how translucent, how far each hair's colour and weight wander, and how ragged its start and finish are. bristle is the 2020 look, ribbon is this port's first brush and the underpainting's default, round is the pointillist sketch that never finished, flat is a house-painter's brush.
• src/demo/ is the page: photo picker, a layer-paced schedule, a player that paints under a per-frame budget and jumps between the brushes' stages from copies taken at the handovers, and a planner on its own thread that reports each brush as it goes and plans the other photos behind the one playing. The transport is a DVD player's rather than a scrubber on purpose: strokes go down over one another and cannot be lifted, so there is no playing backwards, and the moments worth returning to are where each brush begins. Back, play and on are three drawn glyphs, the same geometry as the carets on the selects; the stages beside them show where the painting has got to.
• processing/ holds the 2020 Processing sketches this grew out of, untouched.

TypeScript, Canvas 2D, no framework. tsup builds the library to dist/ and the page script and its worker straight into docs/, which GitHub Pages serves as is; CI rebuilds docs/ and fails on a diff.

Development

npm install
npm run check    # Biome and tsc
npm test         # the planner on synthetic images in Node, then the page in Chromium:
                 # determinism across loads, scrub-back equals paint-forward, phone width
npm run build
npm run gif      # re-record docs/demo.gif from the page (needs ffmpeg)
npm run still    # re-shoot docs/still.jpg, the photo-and-painting pair above
npm run size     # re-measure the Numbers table

Photos

The dog is mine. The others are from Unsplash, free to use under the Unsplash License and credited on the page: Jonny Gios, Cristina Anne Costello, Pietro De Grandi. All are shipped resized with their metadata stripped.

Who made it

Dave Maynard, a front-end engineer in Atlanta. The dog is mine; everything else is credited below.

License

MIT. The bundled Mersenne Twister is BSD-3-Clause; see NOTICE.