Using ffmpeg for Advent of Code visualisation

By Sijmen J. Mulder, 25 December 2022

TL;DR: turns out it's easy to generate video by piping raw frames to ffmpeg and doing so has been a tremendous help for me in debugging my solutions!

Falling sand (puzzle 14, code)

Background

I've had a great time again this year solving Advent of Code puzzles this year. For those unfamiliar, those are Christmas-themed programming puzzles that start out quite simple and get progressively more challenging.

This year alone, puzzles ranged from counting calories, stacking cargo crates and simulating a simple file system to navigating lava caves with the help of an elephant, playing Tetris and finding your way hrough blizzards. But half the fun is the enthousiastic Reddit community!

But it doesn't help that I'm a fairly sloppy programmer so I spend too much time debugging my solutions – often to find out I skimmed over something important in the problem description!

Logging, or printf debugging, is actually very useful here but especially with problems dealing with grids over time, like pathfinding or cellular automata like Game of Life, or with inherently visual problems like intersecting rectangles or cuboids, graphical output is much more useful.

Enter ffmpeg

Considering that I'm mainly using C for my solutions, there is no built-in way to draw graphics, like a canvas. I could use a windowing toolkit and a graphics library like cairo, but that seemed like a lot of work and I prefer video files because they're easy to share.

Enter ffmpeg. ffmpeg is a free (as in freedom) command-line video and audio processor but also comes with libraries like libavcodec.

Using a library is usually the most flexible or 'proper' option but I was looking for a quick fix, and it turns out piping data to ffmpeg is surprisingly useful and powerful.

For the examples here I'll use C, but the principles apply universally. The basic idea is this:

Spawn ffmpeg and redirect its input
Write raw frames to ffmpeg

Launching ffmpeg

char *argv[] = {
	"ffmpeg",
	"-loglevel", "warning",
	"-stats",
	"-f", "rawvideo",
	"-pixel_format", "rgb24",
	"-video_size", "1280x720",
	"-framerate", "30",
	"-i", "-",
	"-pix_fmt", "yuv420p",
	"output.mp4",
	NULL
};

int fds[2];
if (pipe(fds) == -1)
	err(1, "pipe");

switch (fork()) {
case -1:
	err(1, "fork");
case 0:
	dup2(fds[0], 0);
	close(fds[1]);
	execvp("ffmpeg", argv);
	err(1, "ffmpeg");
}

close(fds[0]);
FILE *ffmpeg = fdopen(fds[1], "w");

Explaining the arguments:

-loglevel warning enormously cuts down on ffmpeg's output.
-stats brings back encoding progress, which is helpful in seeing what's going on.
-f rawvideo tells ffmpeg the input will be raw frames, but it also needs to know the exact formats. That what -pixel_format, -video_size, and -framerate are for. Use whatever is suitable.
-pix_fmt yuv420p specifies the output pixel format because it turns out RGB isn't supported well across browsers and video players.

Generating frames

For the input dimensions we specified, we'd declare a frame like this:

static uint8_t frame[720][1280][3];

1280 rows of 720 pixels of 3 bytes each (red, green and blue). While we usually write coordinates as (x,y), raw images are usually represented as lines of pixels.

We could draw a yellow rectangle:

for (int y=50; y<150; y++)
for (int x=30; x<200; x++) {
	frame[y][x][0] = 255;
	frame[y][x][1] = 255;
	frame[y][x][2] = 0;
}

Since we specified 30 frames per second, to make this frame visible for one second we'd send it 30 times:

for (int i=0; i<30; i++)
	fwrite(frame, sizeof(frame), 1, ffmpeg);

A slightly more interesting example would be to make the rectangle shift across the video over a few seconds:

for (int t=0; t<300; t++) {
	memset(frame, 0, sizeof(frame));

	for (int y=50; y<150; y++)
	for (int x=30; x<200; x++) {
		frame[y+t][x+t][0] = 255;
		frame[y+t][x+t][1] = 255;
		frame[y+t][x+t][2] = 0;
	}

	fwrite(frame, sizeof(frame), 1, ffmpeg);
}

Don't forget to close the handle so ffmpeg can finish the file:

fclose(ffmpeg);

int status;
if (wait(&status) == -1)
	err(1, "wait");
if (status)
	errx(1, "ffmpeg exited with status %d\n", status);

That's all there is to it. ffmpeg will output something like this and you get to enjoy the fruit of your work:

$ ./sample
frame=    1 fps=0.0 q=0.0 size=       0kB time=00:00:00.00 ...
...
frame=  300 fps=300 q=-1.0 Lsize=      36kB time=00:00:09.90 ...

Putting it together

The code for the sample above is available on GitHub.

For Advent of Code, I wrote a very small library of functions to set up ffmpeg, draw rectangles, and full grids. They can be found in my Advent of code repository: vis.h and vis.c.

Conclusion and samples

I'm fairly happy with this solution. It's conceptually simple but, as we're dealing with simple raw frames, very flexible for future extension – anything is possible. The downside of such a hand-rolled low-tech approach is that you don't get much for free, but that's OK for me.

Finally, here are my visualisations for Advent of Code this year:

Warning: FLASHING! Also, spoilers for Advent of Code 2022.

Pathfinding (puzzle 12, code)

Falling sand (puzzle 14, code)

3D voxel object (puzzle 18, code)

Permutations on number array (puzzle 20, code)

Path on a cube surface (puzzle 22, code)

Spreading elves (puzzle 23, code)

Pathfinding through blizzards (puzzle 24, code)

Feel free to send any comments or suggestion to me at ik@sjmulder.nl.