Go Concurrency
You may have heard that Go has a concurrency model based on communicating
sequential processes 1, and you may have heard that this is somehow better
than other languages for some reason.
I hear this a lot, but I don’t often hear any discussion of the practical
differences between Go’s concurrency system and that of, say, Python, Rust, or
C#.
In some ways it’s obvious that the go
keyword is not the same as async
and
await
or yield
, but how do these differences actually affect the code you
write?
This post will be a discussion of what I consider the most important difference.
Let us begin by writing a library that includes an example generator from Python’s [PEP 525]. The async/await syntax used here was popularized by C# and is being adopted by Rust, so it’s safe to say that it’s quite popular and likely to be familiar to a large number of developers, even if you’re not familiar with Python specifically:
async def ticker(delay, to):
"""Yield numbers from 0 to `to` every `delay` seconds."""
for i in range(to):
yield i
await asyncio.sleep(delay)
Rewritten in Go, this might look something like this:
// ticker yields numbers from 0 to max on the returned channel, waiting for some
// duration between numbers.
func ticker(delay time.Duration, to uint) <-chan uint {
c := make(chan uint)
go func() {
for i := uint(0); i < to; i++ {
c <- i
<-time.After(delay)
}
close(c)
}()
return c
}
[Playground][Example 1]
This is the version of concurrency I see a lot of people writing in Go, it’s more or less a literal translation of the Python version, and it’s very verbose and not very flexible (as we’ll see in a moment). However, I’d also argue that it’s wrong. If this ticker function were included in a library, I would personally write it like this (I would also probably remove the delay in this particular case and let the user handle that in the callback, but let’s ignore that since it’s not the point of this post and this whole example is probably not something we’d ever actually write):
// ticker calls f with numbers from 0 to max, waiting for some duration between calls.
func ticker(delay time.Duration, to uint, f func(uint)) {
for i := uint(0); i < to; i++ {
f(i)
<-time.After(delay)
}
}
[Playground][Example 2]
This may seem rather self-defeating because what we’ve done is rewrite the function with no support for concurrency at all. Different worker coroutines, for example, can no longer pull the next value from the ticker when they are ready to do more work. If one of Go’s benefits is good concurrency, why would we write a function in a library without any support for concurrency?
The answer is quite simple: in Go, concurrency is an application level concern.
In Python the use of yield
, async
, and await
keywords means that functions
are either concurrency friendly, or they’re not.
You end up with the red/blue function problem described by Bob Nystrom in
[What Color is Your Function?].
This means that if you want a function in your library to be compatible with
asyncio (a popular Python concurrency library), you must write your function
to return futures (or use the syntactic sugar that async/await provides).
However, no such restriction is placed on Go (with a tradeoff that you have less
control over where the event loop or logical processor is preempted).
Because concurrency in Go is primarily managed using the go
keyword, which is
a part of function calls and not function declarations, we can move all
concurrency to the call site and make using concurrency the application authors
problem decision.
If I (the application author) want to call our ticker with no concurrency, I
make a normal function call.
If I want to distribute its values to various worker coroutines, I call the
function inside a goroutine and map it onto my applications concurrency model
myself.
I could also do this with the concurrent version of course, but now I have a
verbose, slower, concurrent function in the library and I have verbose, slow,
synchronization code in the application and none of it’s necessary since I’m not
using the function in a concurrent manner anyways.
func main() {
// Not concurrent
ticker(1*time.Second, 2, func(i uint) {
log.Println(i)
})
// Useless goroutines (but we could be doing something useful here!)
c := make(chan uint)
go func() {
for i := range c {
log.Println(i)
}
close(c)
}()
ticker(1*time.Second, 2, func(i uint) {
c <- i
})
}
[Playground][Example 3]
All this is really just a repeat of what Bob Nystrom says in his aforementioned post, and it’s really a lot of words just to say: If you find yourself using channels or goroutines inside a library, please reconsider your API. It is rarely necessary, and often detrimental.
-
Hoare, C. A. R. (1978). “Communicating sequential processes”. Communications of the ACM. 21 (8): 666–677. [
doi:10.1145/359576.359585
]. [doi:10.1145/359576.359585
]: https://doi.org/10.1145%2F359576.359585 [PEP 525]: https://www.python.org/dev/peps/pep-0525/ [Example 1]: https://play.golang.org/p/P_K8Y83IEip [Example 2]: https://play.golang.org/p/j2MJNiWT7XC [What Color is Your Function?]: https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ [Example 3]: https://play.golang.org/p/jA7lt6qnSLK ↩︎