Over the past few weeks I’ve taken to submitting similar patches to any Go
project that I depend on, directly or indirectly (and some that I don’t).
These patches have all been relatively simple: basic support for Go Modules.
They mostly comprise a
go.mod file, a
go.sum file if required, and minor
tweaks to the Makefile or CI configuration.
I also submitted a few fixes to packages that had existing, but broken module
Here is an incomplete (probably) list of patches that I could find by skimming my browser history:
And here are a handful of issues and patches that report broken module support (but don’t add support for modules themselves), either directly or when importing the project as a dependency:
github.com/dimfeld/httptreemux: General confusion about the import compatibility rule.
github.com/drone/drone: Not importing via the canonical import path.
github.com/Jguer/yay: Did not specify a canonical import before supporting modules (breaking the build outside
GOPATHor when modules are on everywhere).
github.com/Jguer/yay: Incorrect module name for tagged major version.
github.com/ortuman/jackal: Incorrect use of semver tags.
github.com/writeas/writefreely: Did not change module name after fork of dependency.
From this I learned a few things (although some of it I, admittedly, already knew):
Tooling and the import compatibility rule
The tooling (or documentation if this tooling exists and I just couldn’t find
it?) is lacking.
For large projects that are above version one the import compatibility rule
makes supporting modules very time consuming.
Your life becomes a mix of running
sed(1) and then finding and fixing
changes that weren’t actually part of an import line.
Once those large projects are supporting modules, upgrading your project to use new major versions is also very time consuming. Contrary to the Go team’s claim that no one will use modules that change their version often, so the import compatibility rule is not a problem, sometimes you’re stuck with a service (I’m looking at you, Stripe) that can’t stop making breaking changes and doesn’t backport bug fixes. When this happens there’s nothing you can do about it, and it’s infuriatingly difficult to perform upgrades. Even if the breaking change itself was minor and easy to fix, you still have to update the import line every single place you used it and wait for any other dependencies you use to do the same.
The upgrade path from legacy dependency management tools like dep for modules that require use of the import compatibility rule is also very difficult or impossible. Modules did not attempt to be backwards compatible with the existing ecosystem, so supporting it in your post-v1 library means breaking support for consumers of your library who are using old tooling. Because of this, Stripe has decided to revert Go Modules support in their package. I was told several times by someone on the Go team (I’m paraphrasing) that: “people won’t use projects that change major versions frequently, so we don’t have to worry about the import compatibility rule”, but sometimes you’re using a service like Stripe whether you want to be or not, so you’re also stuck with their SDK and the design decisions they’ve made in the past.
Users with no prior experience with modules are likely to merge broken support.
The various combinations of semver tags on the repo, the import compatibility
rule, having or not having a mod file at all, etc. are numerous, confusing, and
not well documented (or hard to find if they are documented).
I saw far more broken module support than I saw correct module support on the
Common mistakes included not adding the major version to the module name where
required, not adding the major version to the import path and accidentally
importing older versions of the module from the newer version, leaving out
dependencies from build tags, leaving out dependencies entirely, and not
I’m putting this down to a documentation problem because I don’t think the Go
project leaders are likely to back down and change anything at this point, but
realistically it’s a complexity problem.
A few unmaintained packages are widely used across the small set of packages I
looked at (eg.
github.com/pmezard/go-difflib and possibly
github.com/BurntSushi/toml), but they’re not always clearly marked as
The dependency on
difflib was most commonly pulled in as an indirect
dependency by way of
github.com/stretchr/testify; if you use Testify not
only are you pulling in unnecessary dependencies, but you’re pulling in
Please take this risk into account before pulling in dependencies for minor
Luckily, authors appear to be more likely to recognize dependency bloat and move
to fix it when they have a list in front of them.
Several authors removed unnecessary testing dependencies, or started removing
small dependencies that didn’t need to be external, after I submitted a patch
with a large
go.mod file that made the problem obvious.
However, lots of small libraries I looked at had no dependencies at all; yay!
People accuse me of complaining about modules a lot, and while I don’t think they should have been released in Go 1.11 without a few changes, and shouldn’t be made official in Go 1.12, I actually really like them overall. As more projects are updated to support modules and the tooling improves, many of these problems are likely to go away or become easier to deal with.
For now, I recommend supporting modules in your project as soon as possible and filing an experience report about your upgrade. The benefits to modules far outweigh the cost, even for large projects. Being able to see a dependency list and known good versions of dependencies has already saved me from build and packaging issues several times; let’s get the Go ecosystem on a single standard packaging technology and then, if we can’t delay the full release, work out the kinks from there.
I previously left out patches for
Previously I also did not list any of the issues filed against projects using
modules incorrectly, or which would be broken by using modules.
Added a few more patches to the list and mentioned that at least one project has decided to revert module support.
Previously the date on these corrections was wrong. I have also added a few more missing links.