Published: Fri 06 January 2023
By C.D. Clark III
In Programming .
Conan's CMakeDeps
and CMakeToolchain
generators provide "transparent" integration with CMake, and support for
multi-config builds. By "transparent", we mean you can use Conan to supply the dependencies for a CMake project without modifying its CMakeLists.txt file.
By multi-config, we mean the ability to compile Debug or Release binaries in one build directory without rerunning CMake. There have been a series of CMake generators
that have lead up to this, cmake, cmake_paths, cmake_find_package, etc., but with Conan 2.0, only CMakeDeps and CMakeToolchain will remain.
The new generators work nicely, but sometimes they lead to strange errors that are difficult to figure out. The Conan documentation provides details on how
to use the generators, but it does not provide details on how they work . So, in this article I am going to detail how the two generators work to achieve
transparent CMake integration and multi-config builds, how to configure your recipe to work correctly with the generators, and how to use them when consuming
libraries..
Table of Contents
CMake's "Dependency Manager"
First we need to understand how CMake works without Conan. In the following examples, I will greatly simplify what CMake does when finding and using
dependencies, but the concepts are the same.
Finding and using dependencies
Let's say I have a small project that depends on Boost. A simple CMakeLists.txt would look like this:
cmake_minimum_required ( VERSION 3.16 )
project ( my_project )
find_package ( Boost COMPONENTS program_options REQUIRED )
add_executable ( example example.cpp )
target_link_libraries ( example Boost::boost Boost::program_options )
There are two steps for using a dependency in CMake. First, we have to find the dependency. This is what find_package(...) does. There is actually
a fairly complex series of steps taken to find the package (see the CMake documentation), but
basically, using "modern" CMake, find_package( <PKG> REQUIRED) looks for a file named <PKG>Config.cmake and sources it. So the package is considered found
if a file named <PKG>Config.cmake is found.
The second thing we have to do is link against the dependency, which in CMake terms is more than just the linker. Our example executable needs to know where
the boost headers are, any special compiler flags that are needed, and many other properties in addition to any library files that should be linked against.
This is what target_link_libraries(...) does. When we "link" example against the Boost targets Boost::boost and Boost::program_options, CMake takes care
of forwarding the settings that are required to use them to the example target.
One important thing to note here is that setting up the target properties
(Boost::boost and Boost::program_options in this example) is the
responsibility of <PKG>Config.cmake .
All find_package( <PKG> REQUIRED) does is find <PKG>Config.cmake and source it. A library can provide support for CMake by writing and installing its
own <PKG>Config.cmake
file (Note: technically Boost does not provide a BoostConfig.cmake file, CMake has to provide support for Boost itself, which it does).
This means that, in order to find a package, CMake has to be able to find the <PKG>Config.cmake file. It does this by searching a set of directories. Again,
the actual proceedure is quite complex, but for simplicity we can just say that CMake looks in a set of predefined system directories, plus any directories
listed in the CMAKE_PREFIX_PATH variable.
When dependencies aren't found
If a package is not found when find_package(...) is called, and the REQUIRED flag is set, then CMake will fail and report the error. For Boost, this looks
something like
-- The C compiler identification is GNU 9 .4.0
-- The CXX compiler identification is GNU 9 .4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at /usr/local/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:230 ( message) :
Could NOT find Boost ( missing: Boost_INCLUDE_DIR program_options)
Call Stack ( most recent call first) :
/usr/local/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:594 ( _FPHSA_FAILURE_MESSAGE)
/usr/local/share/cmake-3.23/Modules/FindBoost.cmake:2376 ( find_package_handle_standard_args)
CMakeLists.txt:5 ( find_package)
-- Configuring incomplete, errors occurred!
This error message is Boost-specific, because CMake knows about Boost and how to find it. But if we were looking for some other library that CMake does not
know about nativley, for example, if we had
find_package ( MissingLibrary REQUIRED )
then we the error message would look something like this
-- The C compiler identification is GNU 9 .4.0
-- The CXX compiler identification is GNU 9 .4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:5 ( find_package) :
By not providing "FindMissingLibrary.cmake" in CMAKE_MODULE_PATH this
project has asked CMake to find a package configuration file provided by
"MissingLibrary" , but CMake did not find one.
Could not find a package configuration file provided by "MissingLibrary"
with any of the following names:
MissingLibraryConfig.cmake
missinglibrary-config.cmake
Add the installation prefix of "MissingLibrary" to CMAKE_PREFIX_PATH or set
"MissingLibrary_DIR" to a directory containing one of the above files. If
"MissingLibrary" provides a separate development package or SDK, be sure it
has been installed.
-- Configuring incomplete, errors occurred!
Here, CMake is telling use that it coudl not find MissingLibraryConfig.cmake, and that we should add the directory containing
the file to CMAKE_PREFIX_PATH. Or, we could set MissingLibrary_DIR to the directory. Either way, CMake needs us to tell it where to find
the config file for the library.
When targets aren't defined
Once a <PKG>Config.cmake file is found, it is sourced. That means that pretty much anything could be put in there, but its responsibility is to
create the library's targets that the consumer will link against (typically, this file sources several other files to do the work). In the case of Boost, we are linking aginst the Boost::boost and Boost::program_options targets.
If a library's <PKG>Config.cmake file is found, but it does not define a target that we use, then we will get a different error. For example, say we have
cmake_minimum_required ( VERSION 3.16 )
project ( my_project )
find_package ( LibraryMissingTarget REQUIRED )
add_executable ( example example.cpp )
target_link_libraries ( example LibraryMissingTarget::LibraryMissingTarget )
and that there is a LibraryMissingTargetConfig.cmake file that CMake can find, but it does not define the target LibraryMissingTarget::LibraryMissingTarget. Then,
when we run CMake, we will get something like this
-- The C compiler identification is GNU 9 .4.0
-- The CXX compiler identification is GNU 9 .4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
CMake Error at CMakeLists.txt:8 ( target_link_libraries) :
Target "example" links to:
LibraryMissingTarget::LibraryMissingTarget
but the target was not found. Possible reasons include:
* There is a typo in the target name.
* A find_package call is missing for an IMPORTED target.
* An ALIAS target is missing.
-- Generating done
CMake Generate step failed. Build files cannot be regenerated correctly.
As a side note, the :: in the target name is important. If we had
target_link_libraries ( example LibraryMissingTarget )
instead, then the configure step would pass, but we would get an error when building (with cmake --build .)
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /home/cclark/Code/sync/projects/conan-examples/cmakedeps_and_cmaketoolchain_blog_examples/missing_target_2/my_project/build
[1/2] Building CXX object CMakeFiles/example.dir/example.cpp.o
[2/2] Linking CXX executable example
FAILED: example
: && /usr/bin/c++ CMakeFiles/example.dir/example.cpp.o -o example -lLibraryMissingTarget && :
/usr/bin/ld: cannot find -lLibraryMissingTarget
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
Apparently the :: tells CMake that this is a target name. Without it, CMake will assume that it is the name of a library that needs to be added to the linker
line. That is because target_link_libraries(...) is used for both transitive dependency tracking, and actual linking.
Transitive dependencies
Modern CMake's target system is nice because it keeps the details of how a dependency is used (adding header includes, library files, etc) hidden. Whereas before
you would have to manually manipulate the CMake variables for include directorys, compiler flags, etc, to use a dependency, now you just link against the target
provided by the dependency and it takes care of the rest.
The other benefit to CMake's target system is that it is transitive . This means that if your library uses Boost, you can do a find_package(Boost) and link
against it, and this will be forwarded to any consumers that link against your library automatically (you need to do some work to make sure that your dependencies are
found when a consumer does a find_package(...) for your library, but it is pretty straight forward). So, if you create a target MyLib::MyLib and link against Boost::boost publicly, then anyone that links against MyLib::MyLib will have Boost::boost added too.
CMake's multi-config generators
CMake supports multi-config generators, where both Debug and Release builds can be compiled form a single configuration. Visual Studio has supported this for years,
but recently (CMake 3.17) a Ninja Multi-Config generator was added that does the same thing.
Any developer that uses Visual Studio has grown accustom being able to switching between Debug and Release builds from inside the IDE. This
is an important part of the developer workflow and is an important feature for CMake to support
The challenge comes when dealing with dependencies. A Debug build of your application should link against Debug builds of its dependencies (so you can debug into them).
This means that both Debug and Release builds need to be installed, and findable. It is possible (and maybe even common) to install Debug and Release builds side-by-side,
but it is not universal.
I don't know how Visual Studio deals with this internally, but the Ninja Multi-Config generator uses generator expressions to switch between Debug and Release settings.
Generator expressions are evaluate at build time (i.e. cmake --build . --config Release), so you can do something like this
add_library ( MyTarget_DEBUG )
... configure MyTarget_DEBUG
add_library ( MyTarget_RELEASE )
... configure MyTarget_RELEASE
add_library ( MyTarget INTERFACE IMPORTED )
target_link_libraries ( MyTarget $ $< CONFIG:Debug > :MyTarget_DEBUG> $ $< CONFIG:Release > :MyTarget_RELEASE> )
Here we are creating separate targets for Debug and Release builds. We then create an interface target that our consumers will link against
and simply forward to the correct target based on the configuration, which is resolved at build time. This is not exactly how multi-config builds are achieved,
but it is the same idea.
Conan's "transparent" CMake support
The goal of Conan's transparent CMake support is to allow consumers to use Conan to provide dependences for their project, without editing their CMakeLists.txt file.
In the example above, we want to be able to use something like this
cmake_minimum_required ( VERSION 3.16 )
project ( my_project )
find_package ( Boost COMPONENTS program_options REQUIRED )
add_executable ( example example.cpp )
target_link_libraries ( example Boost::boost Boost::program_options )
regardless of whether or not we use Conan. In early versions of Conan, consuming dependencies provided by Conan in CMake would require something like this
cmake_minimum_required ( VERSION 3.16 )
project ( my_project )
include ( ${ CMAKE_BINARY_DIR } /conanbuildinfo.cmake )
conan_basic_setup ()
add_executable ( example example.cpp )
target_link_libraries ( example ${ CONAN_LIBS } )
Which meant that the application either had to fully commit to Conan, or write logic to handle both Conan and "vanilla" builds.
Conan's CMake integrations has evolved several times since then. It became clear that user's wanted the ability to use CMake with Conan in
the same way they would use it without Conan, and so the CMakeDeps and CMakeToolchain generators were created.
A word about "generators"
Both CMake and Conan have "generators". In CMake, generators write the files that will be consumed by your build tool. So the "Ninja" generator
will write build.ninja and the "Unix Makefiles" generator will write Makefile. There are Visual Studio generators that will write Visual Studio Solution files.
This is what makes CMake so useful and why it is often referred to as a "meta build system", you can generate files for all kinds of different build tools, on
different platforms, with a single CMakeLists.txt file. By the way, I recommend the "Ninja" generator, it is fast, and does parallel builds by default.
Conan generators also write files that will be consumed by your build system, but they are only concerned with telling your system how to find and use
your dependencies. So when using Conan with CMake, the Conan generators will write files that CMake uses to find and use the dependencies, and the CMake generator
will write files that your build tool uses to build the project.
CMakeDeps generator
The CMakeDeps generator writes <PKG>Config.cmake for dependencies. So, if you require boost/1.80.0 and eigen/3.3.7, then running
$ mkdir build
$ cd build
$ conan install .. -s build_type = Release
will create the files BoostConfig.cmake and Eigen3Config.cmake in the build/ directory. It will actually create several other files too, like BoostTargets.cmake,
Boost-Target-release.cmake, and Boost-release-x86_64-data.cmake. If you look inside BoostConfig.cmake, you see that it is actually pretty small.
That is because it sources BoostTargets.cmake to setup the targets (this is common practice with CMake). This file then sources Boost-Target-*.cmake.
This is where the multi-config support happens. If you install both Debug
and Release builds (conan install .. -s build_type=Debug && conan install ..
-s build_type=Release), then Conan will create files named
Boost-Target-debug.cmake and Boost-Target-release.cmake. Both of these
files will get sourced, and each will set up the targets for their respective
build type and add them to the top-level target (i.e. Boost::boost) using a
generator expression so that CMake can switch between them at build time.
Boost depends on a few other libraries, including zlib, so you will also find ZLIBConfig.cmake and friends in the build/ directory. These
files define the ZLIB::ZLIB target, and if you look in the Boost-Target-release.cmake file, you will see that ZLIB::ZLIB is listed as one of
Boost's dependency targets. So, just as we would do when writing our own <PKG>Config.cmake files, the Boost::* targets will link against the
targets of their dependencies, and users only need to link against the Boost target they are using. CMake will propagate the dependency settings.
So the CMakeDeps generator is responsible for writing the CMake <PKG>Config.cmake files. These define the targets that we will link against with
target_link_libraries(...), but we still need to find these files, and by default, CMake does not look in the current directory for <PKG>Config.cmake files.
This is where the CMakeToolchain generator comes in.
CMakeToolchain generator
The CMakeToolchain generator writes a CMake toolchain file named conan_toolchain.cmake. This file adds the build/ directory to CMAKE_PREFIX_PATH so that
CMake can find the generated <PKG>Config.cmake files with find_package(<PKG> ...). It actually does a bit more than that, but the point is that the toolchain
file tells CMake everything it needs to know to use the config files Conan generated for our dependencies. Using the toolchain file keeps us from having to specify a
giant list of CMake variables at the command line. Instead, we can do something like this
$ mkdir build
$ cd build
$ conan install .. -s build_type = Release
$ cmake .. -DCMAKE_TOOLCHAIN_FILE= conan_toolchain.cmake -DCMAKE_BUILD_TYPE= Release
Note that we have to specify the build type for CMake generators that don't support multi-config builds. If you are using Visual Studio, or the Ninja Multi-Config generator,
you can leave the last command line option off.
Recipe settings
The CMakeDeps and CMakeToolchain generators are intended to be used together. One takes care of configuring dependency settings to be consumed
with target_link_libraries(...) and the other takes care of making the dependencies discoverable with find_package(...). As we saw at the beginning of the post,
calling find_package(Boost) will look for a file named BoostConfig.cmake, which is written by Conan. However, conancenter package names are required to use
lowercase only, the package name for Boost 1.80.0 is boost/1.80.0. So how does Conan know to capitalize "Boost" for the config file? Well, package recipes can
set properties that tell the CMakeDeps generator what it should name the file. We can also tell it the name of the target it should define. This is what
really enables the "transparent" part of CMake integration.
The properties we need to set are cmake_file_name and cmake_target_name. These are set on the cpp_info object. So, in the case of boost, we could so something like this
def package_id ( self ):
self . cpp_info . set_property ( "cmake_file_name" , "Boost" )
self . cpp_info . set_property ( "cmake_target_name" , "Boost::boost" )
Now, I know that this is not what Boost's recipe actually looks like, it is much more complicated. But, the point is that setting the cmake_file_name property
tells Conan what <PKG> in <PKG>Config.cmake should be. By default it is the name of the package (i.e. boost). The cmake_target_name property tells Conan what the
target name it creates should be. By default it is just the package name repeated with a :: in between (i.e. boost::boost). Note that the target name
is not only used in the dependency's config file, it is also embedded in anything that uses the dependency. So if the upstream package uses CMake and defines
targets for its consumers, this should be made to match so that the dependency can be used without or without Conan.
This allows Conan to create its own versions of the CMake config files that will replicate the upstream library's own files.
There are other properties that can be set as well, see the Conan documentation , but if the library you are packaging uses "modern" CMake standard practices, these are probably the only two you need.
Note that is is actually possible to get Conan to use the <PKG>Config.cmake
file generated by the library itself. See this
issue
for details. If you do this with your own projects, you still need to make sure that you set the cmake_target_name property correctly, because again, it will get
embedded in any recipes that use the package as a dependency. This is something that people seem to want, but there are several reasons for not doing it. For one, it does
not allow for multi-config builds. If at all possible, you should give Conan the information it needs to generate CMake config files for you.
Troubleshooting
The new Conan generators are great, but they can give some strange errors that are difficult to understand when you first start using them. Here are a few...
CMake configure succeeds, but build fails with missing headers.
If the configure step (cmake ..) succeeds, but the build step (cmake --build .) fails with errors about not being able to find header or libraries, then it probably means your build types are mismatched. If you install the Debug version of a dependency but then try to build the Release version of your
library/application, then CMake will find your dependency (because the <PKG>Config.cmake files will be present). The dependencies target will even be defined,
but it will be linked to the Debug targets with a generator expression, so when you build in Release mode, the targets are effectively empty.
Errors on Windows
Boost Related
I am not a Windows development export, I only use Windows to test that a library I am working on will build there.
So there may be better (correct) ways to fix these issues, but this is what I found worked for me (after a lot of banging my head on the desk).
Autolinking
If you get a linker error about not being able to find a boost library while compiling a project on Windows that looks like this
LINK : fatal error LNK1104 : cannot open file ' libboost_program_options - vc142 - mt - x64 - 1 _72 . lib
then you might need to disable Boost's autolinking feature, which you can do by linking against the
Boost::disable_autolinking target.
target_link_libraries(MyTarget ... Boost::disable_autolinking ...)
Again, I don't know how the Boost autolinking feature works, so maybe there is a setting in Conan's Boost recipe that will make this work.
Dynamic Linking
If you are trying to use a shared Boost library (conan install .. -o boost:shared=True) and you get a linker error that looks like this
example.obj: error LNK2001: unresolved external symbol "...
then you need to enable dynamic linking for boost, which you can do by linkinking against the Boost::dynamic_linking targets.
target_link_libraries(MyTarget ... Boost::dynamic_linking ...)
Boost's header_only option and missing targets
The Boost recipe has a header_only option that you can use to keep from building/downloading the compiled parts of Boost, which you may want to do if
you only use the header libraries. However, when header_only is set to true, the recipe does not create targets for the compiled components, or the targets
that only affect the compiled components, such as Boost::disable_autolinking and Boost::dynamic_linking. So if you link against these somewhere in your
CMakeLists.txt, they will not be available if you do conan install .. -o boost:header_only=True, and you will get errors.
You should not be linking against these targets in the first place if you are not using any compiled components of Boost, but if you have some optional
part of you library that can be build when a Boost component is available, you just need to take care of handling the targets correctly.
_ITERATOR_DEBUG_LEVEL Linker errorsIf you get a linker error that looks like this
error LNK2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL': value '2' doesn't ...
this it means you have tried to build a different configuration (Debug vs. Release) than you installed with Conan. This seems to only
happen when using the static version of Boost. If you are using a multi-config generator (which if you are on Windows, you probably are)
then you may want to just always install both Debug and Release versions of your dependencies.
Missing shared library (.so and .dll) when running unit tests
If you link against a shared library (.so on Linux, .dll on Windows), then the library
will be loaded at runtime . This is handled differently on Linux and Windows. On Linux it kind of "just works", so if you are coming from Linux
to Windows, this can be confusing.
On Linux, when an executable tries to load a shared library, the system will search for the library in directories listed in the
LD_LIBRARY_PATH environment variable. However, there is also something called a RPATH that is stored in the executable, and on Linux,
the path to the shared libraries that were used at link time is stored in the executable. That means that you can run an executable that uses
shared libraries that live in your Conan cache on Linux. Obviously, if you copied the executable to another machine, this would break, but it works
for running unit tests that used shared libraries.
This does not work on Windows, at least not be default. I don't know what environment variable needs to be set on Windows (or if there even is one), but
the simplest thing to do is put the .dll in the same directory as the executable. Note that this does not work on Linux. You can use Conan's import mechanism to copy the .dll files from your dependencies into the build directory and then run
your unit tests from there.
See the Conan documentation about rpath for more details.
Conclusion
As usual, I wrote this post as a reference for myself, and hopefully someone else finds it useful. If your not using the new CMakeDeps/CMakeToolchain generators in
your project yet, give them a try. You should be able to use them on a project that does not use Conan yet without modifying your CMakeLists.txt file.