A Short Introduction to Makefiles

Makefiles are really good at one thing, managing dependencies between files. In other words, make makes sure all files that depend on another file are updated when that file changes.

We tell make how to do this by declaring rules. A typical rule looks like this:

A Simple Makefile

There are three parts to this rule:

  • The target, bundle.js, before the colon (:).
  • The prerequisites (what the target depends on), jquery.js lib.js main.js, after the colon (:).
  • The command, cat $^ > $@, on the next line after a leading tab, (\t).

There are two "automatic" variables in this command.

  • $@ – filename representing the target, in this case bundle.js.
  • $^ – filenames representing the list of the prerequisites (with duplicates removed).

"Automatic" means that the variables are automatically populated with relevant filenames. This will make more sense when we get into patterns later.

Here are some more variables that are useful.

  • $< – filename representing the first prerequisite.
  • $? – filenames representing the list of the prerequisites that are newer than the target.
  • $* – filename representing the stem of the target, in the above case bundle.

The Make Manual contains the full list of automatic variables

Execution

Running make with the above Makefile results in the following execution.

make runs the first target it finds in the file if none is given on the command line. In this case it is the only target.

The second run didn't do anything since bundle.js is up to date. To be up to date means that the last-modified time of bundle.js is newer than any of its prerequisites' last-modified times. Simple but powerful! When we create new targets all we have to worry about is making sure that our targets know what files it depends on, and what files they depend on, and so on.

It is possible to enter many targets on the left of the colon (:). make will treat them as separate rules and the automatic variable will make sure that the correct files are built.

But, since make treats the rules as separate rules, it will only build the first of them, the default target.

If we want to build bundle2.js, we can do it by explicitly telling make to do it by giving the target as command line parameter.

To get make to build both targets at once, we need to add a new, .PHONY:, target.

Running make now results in (after removing bundle*)

A .PHONY: target is a target without a corresponding file for make to check last modified time on. This means the target will always be run, forcing make to check if all the target's prerequisites needs to be built. The .PHONY label is not strictly necessary. If it is left out, make will check to see if there is a file called bundles and since there isn't one it will build it anyway.

Here's an illustration:

Marking a target that doesn't represent a file as .PHONY: is easy to do and avoids annoying problems once your Makefile grows.

.PHONY: clean

Conventionally every Makefile contains a clean target to remove all the artifacts that are built. In the above case it would contain something like:

make clean will now clean out all files created by the Makefile.

Directories

Directories in make usually needs a bit of special treatment. Let's say we want the bundles above to end up in a build directory. The following Makefile illustrates a problem.

Running make illustrates the problem:

The directory is not automatically created by cat. There are three ways to solve this and one is better than the others.

  1. Add mkdir -p to all rules creating files in the directory.
  2. Add a prerequisite to create the directory on the bundles target.
  3. Add an ordering prerequisite (|) to the rules creating the files in the directory.

1. is not good because the directory will be created more than once, one for each bundle (this is why the -p is needed).
2. is not good because the build directory is not a prerequisite target for bundles.
3. is good because the build directory is clearly a prerequisite of the rule that creates the bundles in this directory.

The reason we have to use an ordering prerequisite instead of a normal prerequisite is that cat would fail otherwise. Here's the resulting good Makefile.

Patterns

Now, we know the basics of Makefiles. We can create rules with targets, prerequisites and commands that are run when needed. But, we have been working with named files all this time. This works fine for small examples like above, but when we have hundreds of files this quickly gets out of hand. Patterns to the rescue.

Let's say that we have a bunch of images that we would like to optimize by running them through an optimizer. The images are in the images/ directory and the optimized images are built into build/images. The naive (and not working) way to do this is shown below. (I'm faking optimize with a simple copy, cp.) The % sign is glob matched with the part of the filename that is not literal.

To see why this is not a viable Makefile, we try to run it with make.

What is going on here? Why is only one image copied and why is it copied as name build/images/*? The problem is that the target files don't exist yet and the * is interpreted literally. If we copy the files into the build directory and touch the source files, it works the way we want.

Here is the main rule to know about patterns. The target file list has to be created from the available source files. To do this we have help of a number of functions, including wildcard, shell, etc. shell will allow us to call anything that we can call from the shell This is very powerful!

How do we solve the above problem? We can do this by getting a list of source images and transforming this list into a list of target images. This is easily done.

The first line introduces both variables and functions.

Variables can be declared in a number of ways, but the :=-declaration is the simplest. It evaluates the value on the right and sets the value on the left to the result, like variables in most programming languages.

Functions are called with the $() construct, and wildcard is a function that evaluates a shell filename pattern and returns a list of filenames.

The full line above populates images with the .png files from the images directory.

The second line converts the source images into the target images. Variables are evaluated the same way as function calls, with the $() construct. By adding a colon-equals expression, a variable substitution reference, after the variable name we can substitute a pattern for another. Example

The third line tells make that our optimize target depends on all the targets existing. This makes sure that all the targets are built.

The fourth line sets up the targets `$(target_images) and its prerequisites with a static pattern rule. The pattern does the opposite of the variable substitution reference above, it deconstructs a single target into the source it depends on. The final part of the of this line is an order prerequisite on the rule to create the directory.

A Recipe for Creating Makefiles

  • Create a list of targets that you want to create from the sources. You have the full power of bash, python, awk, etc. at your disposal.
  • Create a static pattern rule to convert a single target into the source it can be created from.
  • Add order prerequisites to make sure directories are automatically created.
  • Add a callable target that depends on all the target files that you want to create.

Commented Example

Here's a more exotic example of what you can use make for. We have a directory of Javascript source files in lib. The corresponding test files are in test. There may be multiple directories below both lib and test. The testfiles are named like the source files with an added .spec after the stem of the filename.

We want to use a makefile to help us run only the tests that are relevant based on the files that are changed. To keep track of what tests have been run we're going to use marker files and touch them every time a test is run.

Now whenever you run make, it will run only the relevant tests.

Makefiles are really good at one thing: building only stale files. If that is our problem, we should give make a try.

Leave a Reply