automake-history: Dependencies for the User
2.3 Dependencies for the User
=============================
Description
-----------
The bugs associated with 'make dist', over time, became a real problem.
Packages using Automake were being built on a large number of platforms,
and were becoming increasingly complex. Broken dependencies were
distributed in "portable" 'Makefile.in's, leading to user complaints.
Also, the requirement for 'gcc' and GNU 'make' was a constant source of
bug reports. The next implementation of dependency tracking aimed to
remove these problems.
We realized that the only truly reliable way to automatically track
dependencies was to do it when the package itself was built. This meant
discovering a method portable to any version of make and any compiler.
Also, we wanted to preserve what we saw as the best point of the second
implementation: dependency computation as a side effect of compilation.
In the end we found that most modern make implementations support
some form of include directive. Also, we wrote a wrapper script that
let us abstract away differences between dependency tracking methods for
compilers. For instance, some compilers cannot generate dependencies as
a side effect of compilation. In this case we simply have the script
run the compiler twice. Currently our wrapper script ('depcomp') knows
about twelve different compilers (including a "compiler" that simply
invokes 'makedepend' and then the real compiler, which is assumed to be
a standard Unix-like C compiler with no way to do dependency tracking).
Bugs
----
* Running a wrapper script for each compilation slows down the build.
* Many users don't really care about precise dependencies.
* This implementation, like every other automatic dependency tracking
scheme in common use today (indeed, every one we've ever heard of),
suffers from the "duplicated new header" bug.
This bug occurs because dependency tracking tools, such as the
compiler, only generate dependencies on the successful opening of a
file, and not on every probe.
Suppose for instance that the compiler searches three directories
for a given header, and that the header is found in the third
directory. If the programmer erroneously adds a header file with
the same name to the first directory, then a clean rebuild from
scratch could fail (suppose the new header file is buggy), whereas
an incremental rebuild will succeed.
What has happened here is that people have a misunderstanding of
what a dependency is. Tool writers think a dependency encodes
information about which files were read by the compiler. However,
a dependency must actually encode information about what the
compiler tried to do.
This problem is not serious in practice. Programmers typically do
not use the same name for a header file twice in a given project.
(At least, not in C or C++. This problem may be more troublesome
in Java.) This problem is easy to fix, by modifying dependency
generators to record every probe, instead of every successful open.
* Since Automake generates dependencies as a side effect of
compilation, there is a bootstrapping problem when header files are
generated by running a program. The problem is that, the first
time the build is done, there is no way by default to know that the
headers are required, so make might try to run a compilation for
which the headers have not yet been built.
This was also a problem in the previous dependency tracking
implementation.
The current fix is to use 'BUILT_SOURCES' to list built headers
(⇒Sources (automake)Sources.). This causes them to be built
before any other build rules are run. This is unsatisfactory as a
general solution, however in practice it seems sufficient for most
actual programs.
This code is used since Automake 1.5.
In GCC 3.0, we managed to convince the maintainers to add special
command-line options to help Automake more efficiently do its job. We
hoped this would let us avoid the use of a wrapper script when
Automake's automatic dependency tracking was used with 'gcc'.
Unfortunately, this code doesn't quite do what we want. In
particular, it removes the dependency file if the compilation fails;
we'd prefer that it instead only touch the file in any way if the
compilation succeeds.
Nevertheless, since Automake 1.7, when a recent 'gcc' is detected at
'configure' time, we inline the dependency-generation code and do not
use the 'depcomp' wrapper script. This makes compilations faster for
those using this compiler (probably our primary user base). The
counterpart is that because we have to encode two compilation rules in
'Makefile' (with or without 'depcomp'), the produced 'Makefile's are
larger.
Info Catalog
automake-history: Dependencies As Side Effects
automake-history: Dependency Tracking Evolution
automake-history: Techniques for Dependencies