Let us begin with a very basic example. Our project is going to be very simple: we have a simple data source, a CSV file. With this data, we need to measure the mean and standard deviation. Not a very exciting example, and it certainly won't win you a noble prize, but it will highlight how we might create our simple makefile to automate this process.

In this example, we have two starting files: (1) our data source – data.csv; and (2) our program with which we can import the data source and compute the mean and standard deviation, aptly named stats.py. This program needs no command line arguments, but when called it simply looks for data.csv, computes the statistics and writes a new file output.csv.

In the same directory, we shall create a new file called Makefile. The contents of our makefile are going look like this:

output.csv: data.csv stats.py
        python stats.py 

This simple example includes the majority of what you will be doing in makefiles. We have a target in this case output.csv. We can tell its a target because it is followed by a colon ':'. This is the command we will pass to make when we wish to execute our make file.

Next, we specify our (optional) dependencies. Our target depends on two files: data.csv and stats.py. This is simple a space delimited list of files after our target. Simply stated, we are telling make that in order to create our CSV file, both data.csv and stats.py must exist. But there's more: make is smart and by listing the dependencies, we are telling make that if either our data or our program changes then output.csv will need to be re-made. If they haven't change and output.csv already exists, make will tell us that nothing more needs to be done (it doesn't bother executing the program twice).

The next line tells make what to do in order to create our target, our rule. In this case we call python stats.py. Like python, makefiles are indentation delimited. But unlike python where this indentation can be done either with tabs or spaces (but definitely not both), makefiles are always indented with tabs.

To run this makefile, we type make output.csv into the command line. If everything is setup correctly, make will run python stats.py and our output.csv will be created.

Suppose we have two algorithms and we wish to generate some comparative metrics. Like the previous example, we have a single data source called data.csv, but this time, we have two python programs: one for each algorithm. Each of these programs will create its own CSV file output. The process flow will look like this:

In this case, we wish to execute both python programs to create both outputs.

output_1.csv: data.csv algorithm_1.py 
        python algorithm_1.py

output_2.csv: data.csv algorithm_2.py
        python algorithm_2.py 

This time we have created two targets, one for each of our output csv files. Each target has their own dependencies and the rule to create the different outputs.

In order to actually create the CSV files, we will type:

make output_1.csv
make output_2.csv

into the command line. make is of course happy to take these two requests, but there is more onerous on us to make sure to execute both commands. While this is okay for these two CSVs, it will become more laborious when we have more.

To overcome this issue, we can use a PHONY target. A phony target is one that won't exist but serves as a alias to provide make with a command we can type. In our previous example, both output_1.csv and output_2.csv were files that will exist after make has executed the two rules. With a phony target, however, make won't bother to look for the target's existence.

We can create a phony target with:

.PHONY: output
output: output_1.csv output_2.csv

Our phony target, named output, depends on both our CSV files. Now, we can get make to create both of these output files with a single command: make output.

Our previous examples have consisted of only one processing step, we take our input data, and produce an output. In these cases, keeping track of what is up-to-date and what still needs to be executed is easy enough. But when we introduce processing pipelines where the output of one program feeds into another, things can get a little more complicated.

We have introduced a significantly more complicated process. We use a c++ program squares.cpp to take all of the data in the CSV, square it, and save the intermediate version as squared_data.csv. Processes such as this may occur when the dataset is large enough that we apply computations in batches or jobs via high-performance computing. In these cases, it is better to keep the original data source and preserve its mutability.

With this squared data, our third algorithm – algorithm_3.py – is executed to produce output_3.csv. To automate this process, we will add the following rule to our makefile:

output_1.csv: data.csv algorithm_1.py
        python algorithm_1.py

output_2.csv: data.csv algorithm_2.py
        python algorithm_2.py

output_3.csv: data.csv squares.cpp
        g++ -i squares.cpp squares.o
        ./squares.o 
        python algorithm_3.py

.PHONY: outputs
outputs: output_1.csv output_2.csv output_3.csv

and all outputs can be made with make outputs.

Both algorithm 1 and 2 are the same as the previous examples, but algorithm 3 has more computation steps. First, we must ensure that our c++ code is compiled, then we must produce the squares_data.csv (done by ./squares in this example), and then finally run python algorithm_3.py to produce the results.

This is great, but what if algorithm 2 depended on both data.csv and squared_data.csv?

We could change our output_2.csv target to something like this:

output_2.csv: data.csv squared_data.csv algorithm_2.py 
        g++ -i squares.cpp squares.o 
        ./squares.o 
        python algorithm_2.py 

But we may notice that the compilation of the c++ program occurs twice. So, instead, lets make the executable a target within its own right:

squares.o: squares.cpp 
        g++ -i squares.cpp squares.o 

In addition, we can also create a target for the squared_data.csv as we only with to create it once.

squared_data.csv: squares.o 
        ./squares.o

and we'll amend our previous versions of targets output_2.csv and output_3.csv to depend on this executable already existing and being up-to-date.

output_2.csv: data.csv algorithm_2.py squared_data.csv
        python algorithm_2.py

output_3.csv: data.csv algorithm_3.py squared_data.csv
        python algorithm_3.py 

This way, both the compilation and creation of squared_data.csv happens once.

Our final makefile shall look like the following:

output_1.csv: data.csv algorithm_1.py
        python algorithm_1.py

output_2.csv: data.csv algorithm_2.py squared_data.csv
        python algorithm_2.py

output_3.csv: data.csv algorithm_3.py squared_data.csv 
        python algorithm_3.py

squares.o: squares.cpp 
        g++ -i squares.cpp squares.o

squared_data.csv: squares.o 
        ./squares.o

.PHONY: outputs 
outputs: output_1.csv output_2.csv output_3.csv

and all output CSV files can still be created with one single command: make outputs. If we wish to just execute one pathway or algorithm, we can just specify that particular target. For example, if we just wish to run algorithm 1, we can run make output_1.csv.

We can go further and improve our process of research. Now that we have our outputs from each of the algorithm, we may summarise them and produce a final CSV file to present in a report.

Our summarise.py takes all the results from each algorithm, provides some summary statistics, and outputs a single CSV file that is suitable for a report.

If we use latex, and the PGFplotstable package, we can also automate the process of getting these results into our report.

\documentclass{article}

\usepackage{pgfplotstable} % must use this package

\title{Report}

\begin{document}

% import our table from the CSV file
\begin{table}
\centering
\caption{My table}
\label{tab:my_table}
\pgfplotstableread[col sep=comma]{final_results.csv}\data
\pgfplotstypeset[
    % column options
]{\data}
\end{table}

\end{document}

Now, every time our results file changes, either because we added more algorithms, or we have modified the code or data, our report will always be up to date.

PGFPlotstable is a very useful package for importing, formatting, and even doing basic evaluations of data. Moreover, if you combine this package with the regular PGFplots, you data tables and plots can be automatically kept in sync with one another.

We can go further and add the compilation of this latex document to our makefile:

.PHONY: report
report: final_results.csv report.tex
        pdflatex report.tex

And compile it using make report. These is an added benefit here that we may also add the recompilation rules for if our document contains a bibliography:

.PHONY: report
report: final_results.csv report.tex references.bib
        pdflatex report.tex
        bibtex report.aux
        pdflatex report.tex
        pdflatex report.tex