Debian Package rebuild

Rebuild of the Debian archive with clang

By Sylvestre Ledru (Debian, IRILL).

Presentation

This document presents the result of the rebuild of the Debian archive (the distribution) with clang, a C/C++ compiler.

clang is now ready to build software for production (either for C, C++ or Objective-C). This compiler is providing many more warnings and interesting errors than the gcc suite while not carrying the same legacy as gcc.
This rebuild has several goals. The first one is to prove (or not) that clang is a viable alternative. Second, building a software with different compilers improves the overall quality of code by providing different checks and alerts.

Rebuild results

The detailed list of errors with clang 13.0.0:
33859 packages have been rebuild. Among them, 994 (2.9 %) failed.
Most of the errors are explained with test cases.
Type of errorOccurrenceclang % / Debian %
Some headers could not be found 75 7.55% / 0.22% List of errors
Cannot be narrowed 69 6.94% / 0.2% List of errors
Linker error 63 6.34% / 0.19% List of errors
Testsuite is failing with clang 62 6.24% / 0.18% List of errors
Could not find objective C headers 59 5.94% / 0.17% List of errors
C++11 requires a space between literal and identifier - new in 3.8 50 5.03% / 0.15% List of errors
Missing symbols at link time 46 4.63% / 0.14% List of errors
non-void function should return a value 34 3.42% / 0.1% List of errors
Relocation error 27 2.72% / 0.08% List of errors
Unknown argument in the command line 18 1.81% / 0.05% List of errors
Expected declaration (#define missing?) 18 1.81% / 0.05% List of errors
Variable set but not used - new in 13.0.0 18 1.81% / 0.05% List of errors
No support of nested C function 16 1.61% / 0.05% List of errors
Unknown warning option 15 1.51% / 0.04% List of errors
Security: Format string is not a string literal 14 1.41% / 0.04% List of errors
Not allowed args fails the build 12 1.21% / 0.04% List of errors
Cannot find member in the struct 11 1.11% / 0.03% List of errors
Cannot use -o use multiple output 10 1.01% / 0.03% List of errors
Wrong main declaration 10 1.01% / 0.03% List of errors
Some files are gone in the process 10 1.01% / 0.03% List of errors
Redefinition failed 8 0.8% / 0.02% List of errors
No viable overloaded 7 0.7% / 0.02% List of errors
Unqualified lookup in templates 6 0.6% / 0.02% List of errors
Linker option unused 6 0.6% / 0.02% List of errors
Preprocessor differences 6 0.6% / 0.02% List of errors
Unsupported argument with another option 5 0.5% / 0.01% List of errors
Conflicting types 5 0.5% / 0.01% List of errors
Variable length array in structure won't be supported 5 0.5% / 0.01% List of errors
No matching function call 5 0.5% / 0.01% List of errors
No matching member or constructor call 5 0.5% / 0.01% List of errors
Void function should not return a value 5 0.5% / 0.01% List of errors
Missing exception specification 5 0.5% / 0.01% List of errors
Unknown Type Name 4 0.4% / 0.01% List of errors
Ambiguous declaration 4 0.4% / 0.01% List of errors
Argument unused caused failure 4 0.4% / 0.01% List of errors
Deprecated copy - new in 10.0 4 0.4% / 0.01% List of errors
Unsupported include directive within macro arg 4 0.4% / 0.01% List of errors
Source think that it is Mac Os X because of clang 4 0.4% / 0.01% List of errors
Multiple definition 3 0.3% / 0.01% List of errors
Use of old GNU field designator 3 0.3% / 0.01% List of errors
Parameter list without types not allowed 3 0.3% / 0.01% List of errors
Ignored or unknown attributes 3 0.3% / 0.01% List of errors
does not allow access declarations - new in 6.0 3 0.3% / 0.01% List of errors
Cannot pass non-trivial object of type 'xx' to variadic function - new in 6.0 3 0.3% / 0.01% List of errors
implicit conversions 3 0.3% / 0.01% List of errors
Unsupported option 2 0.2% / 0.01% List of errors
Equality comparison with extraneous parentheses 2 0.2% / 0.01% List of errors
Access to a protected member 2 0.2% / 0.01% List of errors
Member is not a structure or union 2 0.2% / 0.01% List of errors
Explicit Specialization after instantiation 2 0.2% / 0.01% List of errors
Could not find a library 2 0.2% / 0.01% List of errors
Potential usage of an uninitialized variable 2 0.2% / 0.01% List of errors
The build timeout 2 0.2% / 0.01% List of errors
Forward declaration of class cannot have a nested name specifier 2 0.2% / 0.01% List of errors
Hides overloaded virtual function 2 0.2% / 0.01% List of errors
Implicit function error 2 0.2% / 0.01% List of errors
-Wvarargs: undefined behavior with va_start - new in 3.9 2 0.2% / 0.01% List of errors
Optimization flag is not supported - new in 4.0 2 0.2% / 0.01% List of errors
qmake: failed to parse default search paths from compiler - new in 6.0 2 0.2% / 0.01% List of errors
Missing field initializer 2 0.2% / 0.01% List of errors
Moving a temporary object prevents copy elision 2 0.2% / 0.01% List of errors
Overrides a member function but is not marked override 2 0.2% / 0.01% List of errors
Unqualified lookup into dependent bases of class templates 1 0.1% / 0% List of errors
Configure failed 1 0.1% / 0% List of errors
Tautological comparison 1 0.1% / 0% List of errors
Unused parameter 1 0.1% / 0% List of errors
Unused function 1 0.1% / 0% List of errors
Cannot initialize a element 1 0.1% / 0% List of errors
Changes of default constructor 1 0.1% / 0% List of errors
Wrong C++ default declaration in a method 1 0.1% / 0% List of errors
Embedded directive has undefined behavior 1 0.1% / 0% List of errors
Header guard failure 1 0.1% / 0% List of errors
Absolute value error - new in 3.5.0 1 0.1% / 0% List of errors
Invalid instruction mnemonic 1 0.1% / 0% List of errors
Unknown/unexpected directive 1 0.1% / 0% List of errors
Initializer element not compile-time constant 1 0.1% / 0% List of errors
Tautological pointer comparison - new in 3.5.0 1 0.1% / 0% List of errors
Incorrect conversion - new in 3.6.0 1 0.1% / 0% List of errors
Explicitly assigning value of variable 1 0.1% / 0% List of errors
illegal storage class on function 1 0.1% / 0% List of errors
Not categorized192 19.32% / 0.57% List of errors

A maintainer view is also available. This shows per developers which packages should be fixed.

Errors can be caused by several reasons:
Of course, a bug can hide other bugs.
Bugs (with patches) will be reported in the Debian bug tracker. They will be tagged as wishlist.

Detailed results

The full list of all the results with logs are available at the following URL:

Status

The Debian archive is regularly rebuilt against the latest version of clang. This to achieve three goals:
1) See which packages get fixed
2) See the impact of new clang releases
3) Find bugs or gcc-compatibility issues in gcc itself



Annexes

Rebuild

The 2.9 and 3.0 rebuild itself have been done on a cluster called Grid 5000.
3.1, 3.2 and later rebuilds have been done on AWS, the Amazon cloud, sponsoring Debian.

Each packages in the Debian archive has been rebuild with the chroot described further.
For each packages failing to build from the source with clang, the package has been rebuild with a "normal" Debian sid chroot. If working, we considered that it was due to a clang bug. Otherwise, the package is not listed in this list.

Configuration of the chroot

The procedure for 2.9 & 3.0 was the following:
VERSION=4.6
cd /usr/bin
rm g++-$VERSION gcc-$VERSION cpp-$VERSION
ln -s clang++ g++-$VERSION
ln -s clang gcc-$VERSION
ln -s clang cpp-$VERSION
cd -

All packages of the archive (15658 sources packages) have been rebuild this chroot (even the one without any C or C++ code). Build dependencies used are the same as Debian. For example, that means that the libxml2 used to build a package will be the one from Debian, not the one built with clang.

From clang 3.1, with the EC2 Amazon cloud, an chroot-setup-commands is used for each sbuild to setup the environnement.
The init code is the following: 
apt-get update

echo "Install of clang"
apt-get update
apt-get install --yes --no-install-recommends clang -t unstable

echo "Replace gcc, g++ & cpp by clang"

cd /usr/bin
VERSIONS="4.8 4.7 4.6"
for VERSION in $VERSIONS; do
rm g++-$VERSION gcc-$VERSION cpp-$VERSION
ln -s clang++ g++-$VERSION
ln -s clang gcc-$VERSION
ln -s clang cpp-$VERSION
done

echo "Block the installation of new gcc version"
echo "gcc-4.6 hold"|dpkg --set-selections
echo "cpp-4.6 hold"|dpkg --set-selections
echo "g++-4.6 hold"|dpkg --set-selections
echo "gcc-4.7 hold"|dpkg --set-selections
echo "cpp-4.7 hold"|dpkg --set-selections
echo "g++-4.7 hold"|dpkg --set-selections

echo "Check if gcc, g++ & cpp are actually clang"
gcc --version|grep clang > /dev/null || exit 1
The same procedure has been used for the 3.2 rebuild except that the clang packages have been rebuild for unstable (instead of the experimental packages).

Acknowledgments

Many thanks to Lucas Nussbaum for his time and patience.
Stefano Zacchiroli for the review (among other things).
Thanks also to Grid 5000 for their infrastructure and Amazon for their credit to use AWS.