How do Dependency Managers Work?
Dependency managers like CocoaPods, Carthage or SwiftPM seem magic. But they also seem kind of awkward and complicated. Let’s take them apart into their constituent parts to make it easier to understand why they work the way they work.
Obtaining the (matching) Code
The first thing a dependency manager has to do is make sure that you get the right version of the code for the current version of your app. The simplest approach to this is what Git Submodules do: You define a certain version of the dependency’s code (in this case, a particular Git Commit) and write it into the current version of your app. Then, when you check out your app, that code gets checked out as well.
The most important part here is that the dependency’s code ends up at a fixed directory path, so your Makefile or Xcode project file or whatever can reliably reference it.
Both of these points are guaranteed by your .gitmodules
file: It specifies the directory
into which a dependency should be checked out, as well as the Git Hash representing a
particular version of the code. Every dependency manager has something roughly equivalent,
be it called Cartfile
, Podfile
, or Package.swift
.
Coalescing Duplicate Dependencies
The second thing a dependency manager needs to do is make sure that, if multiple
dependencies depend on the same sub-dependency, you only get one. Imagine your app was
called MyMusicPlayer.app
, and it consisted of two sub-components, MP3PlaybackLib
and
AACPlaybackLib
. And both of these dependencies relied on a MetadataLib
sub-dependency
for reading artist, song title, composer, album etc. from the files.
You initial instinct would be to just add the MetadataLib
submodule to both
MP3PlaybackLib
and AACPlaybackLib
. But then you’d get complaints from your compiler
about duplicate symbols.
So what most dependency managers do in this situation is that they enforce that the same
dependency is only downloaded once and shared by all submodules that need it. How they
usually do this is that they do not download dependencies into the same folder as the
component that depends on it, but into a central location. In the case of CocoaPods, that
means all dependencies are downloaded into a Pods
directory in the application’s folder.
Carthage has its Carthage
folder, and SwiftPM downloads all dependencies into its
Packages
directory.
This not only makes it easier for the dependency manager to detect duplicate dependencies,
it also means that dependencies know their other dependencies will be right next to them
in the file system. In fact, I worked on projects where that was how we handled
dependencies: Every app had a submodules
folder at the top level, and we put all
dependencies in there. All sub-modules (frameworks, libraries etc.) were set up so they
expected their dependencies next to them, and it was the app repository’s job to check out
the right versions so all of it would build.
A package manager just automates that work.
Resolving Conflicts
Another thing a dependency manager has to do is detect conflicts between modules, and, if possible, resolve them. How they do that is by allowing a range of versions instead of just one fixed version. That way, if you just make compatible changes, the dependency manager can automatically upgrade you to a newer version to make it match another component’s requirement.
They usually do this by collecting all the requirements for each module, and then matching available releases against those requirements, usually picking the newest match. usually that also implies that you can’t just reference Git branches like a submodule, but need to actually tag releases and use semantic versioning to be able to tell versions that are compatible with each other from ones that aren’t.
The advantage of this is that the dependency manager can see if your sub-components have
conflicting version requirements for their dependencies. Like, if MP3PlaybackLib
used
MetadataLib
1.0.0
but AACPlaybackLib
wants MetadataLib
2.0.0
, where incremementing
the first digit according to semantic versioning
indicates that the API has changed in an incompatible way, no version will match the
version range specified in the Podfile
/Cartfile
/Package.swift
, because the range is
1.x.x
for MP3PlaybackLib
but AACPlaybackLib
’s range is 2.x.x
, and nothing can be
both 1.x
and 2.x
.
Resolved Dependency List Files
Most dependency managers have a second version of the file listing the dependencies.
CocoaPods has Podfile.lock
, Carthage has Cartfile.resolved
and SwiftPM has
Package.resolved
. What are these for?
The normal dependency list file (Podfile
, Cartfile
, Package.swift
) contains a range
of version numbers. But you can’t build “a range of versions”. You have to pick one. Since
we want to be able to go back to a particular release of our app and build it again (e.g.
to fix a bug), we need to be able to remember (in Git or whatever version control we’re
using) what versions our dependency ranges were resolved to at the time.
That’s what the resolved dependency list file is for. It contains the concrete version of the dependency that will be checked out, and you can check that into your version control.
Integrating the Dependencies into Your Build
While not strictly part of dependency management (for example, Carthage doesn’t do this), many dependency managers also integrate with the app to save you from having to reference the dependencies they checked out in your app.
While SwiftPM support was just added into Xcode by Apple, dependency managers that are not maintained by a first party have to do their own work here. In the case of CocoaPods, this is a bit confusing at first, but actually turns out to be not that complicated.
What CocoaPods does is that it basically edits your project file and adds dependencies to it, assuming they are all frameworks and libraries. However, since CocoaPods and the developer both editing a project file would be a tad risky, they actually create a Workspace around your project. A Workspace is an umbrella around several projects that lets them see each other, and lets you override settings in them, either manually, or via xcconfig text files.
So what CocoaPods does is, it creates a workspace around your project. The only one who edits the Workspace is CocoaPods. You are free to edit your project file without any collisions. And from then on, you just need to remember to double-click the Workspace and not your project to build your app (otherwise you wouldn’t get the dependencies and your app wouldn’t build).
That’s it?
Yes. You see, the general functionality of a dependency manager consists of a few simple parts. Of course, creating your own and making it work reliably is harder, and there are numerous edge cases and additional features one might want. But at its core? It’s just a fairly straightforward solution to a problem people have been solving by hand for quite a while.
And you know how it is: Don’t spend time solving problems you don’t have, but once you have a problem, having an automated solution to it means you can avoid casual mistakes performing it by hand.