makefile - multiple targets from one recipe and parallel execution

Question

I have a project which includes a code generator which generates several .c and .h files from one input file with just one invocation of the code generator. I have a rule which has the .c and .h files as multiple targets, the input file as the prerequisite, and the recipe is the invocation of the code generator. I then have further rules to compile and link the generated .c files.

This works fine with a -j factor of 1, but if I increase the j factor, I find I get multiple invocations of the code generator, up to the -j factor or the number of expected target files, whichever is smallest. This is bad because multiple invocations of the code generator can cause failures due to the generated code being written multiple times.

I'm not going to post my actual (large) code here, but I have been able to construct a small example which appears to demonstrate the same behavior.

The Makefile looks like this:

output.concat: output5 output4 output3 output2 output1
    cat $^ > $@

output1 output2 output3 output4 output5: input
    ./frob input

clean:
    rm -rf output*

Instead of a code generator, for this example I have written a simple shell script, frob which generates multiple output files from one input file:

#!/bin/bash

for i in {1..5}; do
    {
    echo "This is output${i}, generated from ${1}. input was:"
    cat ${1}
    } > output${i}
done

When I run this Makefile with non-unity -j factors, I get the following output:

$ make -j2 
./frob input
./frob input
cat output5 output4 output3 output2 output1 > output.concat
$

We see ./frob here gets invoked twice, which is bad. Is there some way I can construct this rule such that the recipe only gets invoked once, even with a non-unity -j factor?

I have considered changing the rule so that just one of the expected output files is the target, then adding another rule with no recipe such that its targets are the remaining expected output files, and the prerequisite is the first expected output file. But I'm not sure this would work, because I don't know if I can guarantee the order in which the files are generated, and thus may end up with circular dependencies.

Answer 1

This is how make is defined to work. A rule like this:

foo bar baz : boz ; $(BUILDIT)

is exactly equivalent, to make, to writing these three rules:

foo : boz ; $(BUILDIT)
bar : boz ; $(BUILDIT)
baz : boz ; $(BUILDIT)

There is no way (in GNU make) to define an explicit rule with the characteristics you want; that is that one invocation of the recipe will build all three targets.

However, if your output files and your input file share a common base, you CAN write a pattern rule like this:

%.foo %.bar %.baz : %.boz ; $(BUILDIT)

Strangely, for implicit rules with multiple targets GNU make assumes that a single invocation of the recipe WILL build all the targets, and it will behave exactly as you want.