SamWhited|blog

Introducing Mux

Note: This post is about code.soquee.net/mux, an HTTP multiplexer for Go. This post applies to mux version v0.0.4. Because this version is pre-1.0, things may change by the time you read this. The most up-to-date version of the mux documentation can always be found at pkg.go.dev/code.soquee.net/mux.

There are a vast number of Go HTTP multiplexers on the market already, but every time I try a new one I find that it has one or more of the following problems:

To finally have a multiplexer that meets my requirements, I wrote the code.soquee.net/mux package. Though I wrote and open sourced it quite a while ago, I never really promoted it or tried to explain to anyone what it could do or why they might want to use it over the many alternatives, so here it goes.

Basic Usage

The mux package works in a broadly similar way to other Go HTTP multiplexers, except that it makes a trade off that involves having an extra level of indentation when creating the router to prevent routes from being added after the router is created. Creating a new router looks something like this:

serveMux := mux.New(
	mux.HandleFunc(http.MethodGet, "/profile", http.RedirectHandler("/profile/me", http.StatusPermanentRedirect)),
	mux.Handle(http.MethodGet, "/profile/me", profileHandler()),
	mux.Handle(http.MethodPost, "/logout", logoutHandler()),
)

Use of the functional options pattern lets your users know that they shouldn’t mutate the multiplexer after it has been returned and ensures that our handlers and patterns are grouped into an easy to read list.

Route parameters

The multiplexer also has a simple pattern language for matching “route parameters” which are typed, and optionally named, variables that can appear as any route component.

mux.Handle(http.MethodGet, "/users/{username string}", userHandler()),

In this example there is a single route parameter with the name “username” and the type “string” which matches any single path component such as “/users/geoff” or “/users/123”. Other valid types are more or less restrictive and match multiple components, or only match if the path component can be parsed into a Go type. As of this writing, the full list of types is:

Currently the path type must be the last comonent in the route and matches the remainder of the route (regardless of how long it is). This restriction may be lifted in the future.

There is also a special type: static. A static typed parameter must be named and does not store any value. In fact, /foo is just a shorthand for /{foo static}. You should never have to know this or use the latter syntax, it’s just an interesting implementation detail.

When a route is matched, the value of each named path parameter is stored on the request context. To retrieve the value of named path parameters from within a handler, the Param function can be used.

pinfo := mux.Param(req, "username")
fmt.Println("Got username:", pinfo.Raw)

This returns a ParamInfo struct containing the parsed (typed) and raw (string) value of the parameter, the name of the component that was matched, the type type of the route parameter (eg. “int” or “uint”), and the offset of the matched component in the path.

Safety

Consider the following routes:

mux.New(
	mux.Handle(http.MethodGet, "/users/{username string}", userHandler()),
	mux.Handle(http.MethodGet, "/users/new", newUserHandler()),
)

If you try to build the code in this example and then run the application, you’ll see that it panics on startup:

panic: conflicting type found, {new static} in route “users/new” conflicts with existing registration of {username string}

This is because the static route, /profile/new conflicts with the variable route parameter “username”. What happens in this system if a user registers or already has the username “new”? We can’t know which route should be selected, and we might end up with a user that cannot access their own profile, or the inability for new users to register because the registration page redirects them to an existing users profile!

Instead of risking it, mux disallows ambiguous routes using two rules:

  1. If a variable route parameter exists in the same position in two otherwise identical routes, they must have the same name and type (ie. no /{string} and /{int} or /{a string})
  2. Variable and static route components may not appear in the same position in otherwise identical routes (ie. no /{string} and /foo or /{foo static})

Put another way, no two routes from the following may be registered together:

/user/{a int}/new
/user/{b int}/edit
/user/{float}/edit
/user/{b string}/edit
/user/me

When I first used a multiplexer that had similar rules I thought they were cumbersome. However, after a while I noticed that my APIs and websites that used these rules often had much more logical and consistent routes, and no ambiguous routes slipping through code review. In the end, I decided to keep the rules for mux strict to keep myself honest and make products that use it better.

Normalization

Let’s imagine that we are building a web app that uses a GitHub like URL including a username and a repo. We register a route like the following:

mux.Handle(http.MethodGet, "/{user string}/{repo string}", repoHandler())

GitHub, unfortunately, allows any casing in the username meaning that there are multiple resources that serve the same information, /soquee and /Soquee are the same thing in GitHub’s mind.

We want to do better, so we use a case insensitive lookup in the database to find the canonical casing and then redirect to the correct place. To keep the examples simple, we’ll assume the canonical casing is always lowercase, instead of looking up the correct case in the database. Repos on the other hand are always case sensitive.

How do we handle this redirect? In most libraries, we’d either have to hard code the exact path component where the username is in our handler so that we can replace that component in the path and issue a redirect, or we’d have to just search and replace over the entire path for the username we want to replace. Both of these are potentially buggy. The first because we’ve introduced tight coupling in the code and if we make changes we might break our hard coded path. Alternatively, we’d need lots of boilerplate to generate the route and return the route and the path, which means we can’t see the route as a string literal in our list of routes. The second is buggy because the repo name might be the same as the username except for the case and we wouldn’t know which name to rewrite to the canonical case.

In other routers I’ve used, we can get the original route, iterate through its components, parse any path parameters and check if the names match, then find that same component in the path and replace it. This is a lot of work.

The mux package handles this for you using the WithParam function. This returns a new copy of the request with the given route parameters replaced on the context. Because a new request (with a new context) is returned, nothing needs to be mutated and any middleware already using the old request keeps its same view of what the route looks like. Anything using the new request can then call Path to construct the updated path.

The following handler normalizes just the username component of the path, resulting in a redirect to the canonical (in this case username case mapped) username:

mux.HandleFunc("GET", "/{username string}/{repo string}", func(w http.ResponseWriter, r *http.Request) {
	username := mux.Param(r, "username")
	normalized, err := precis.UsernameCaseMapped.String(username.Raw)
	if err != nil {
		…
	}

	// If the username was not canonical, redirect to the canonical username.
	if normalized != username.Raw {
		r = mux.WithParam(r, username.Name, normalized)
		newPath, err := mux.Path(r)
		if err != nil {
			…
		}
		http.Redirect(w, r, newPath, http.StatusPermanentRedirect)
		return
	}

	// Show the users profile.
	fmt.Fprintf(w, "Profile for the user %q", username.Raw)
})

Conclusion

You may notice that I didn’t cover speed in this article. That’s because I have not benchmarked mux in any meaningful way (microbenchmarks that ensure performance of individual parts of the code don’t tell the whole story and aren’t worth mentioning here). While I believe that it should compete with the fastest Go muxers out there, and have tried to take care to avoid expensive operations when looking up routes, speed just isn’t one of the priorities of mux. In general, even in the hot path of an HTTP API, mux should be fast enough for everyone’s needs, and if not, I accept patches!

I hope this has given you a good sense of what’s possible with mux. To recap, the main features are:

Soquee is an issue tracker written in Go and a collection of open source (BSD-2-Clause) Go modules written to support it. Open source code written for Soquee (and eventually the issue tracker itself) can be found at code.soquee.net.