Cleaning up Async Without Swift 5.5

It’s finally here! After seven years of callbacks and completion handlers, we now have a way to cleanly and correctly make asynchronous calls in Swift without endless nesting. It’s async/await and it’s awesome.

Sadly, these new features require runtime support. Which means, at least for the time being, async is iOS 15-/macOS 12-only.

For those of us supporting older deployment targets, this can be a bit of a let down. But not all hope is lost! We can build clean, flattened-out async handling on our own. And maybe learn a thing or two about the nature of asynchrony along the way?

async/await

Let’s start with a description of the problem and how Swift 5.5 solves it.

Given a series of dependent, asynchronous calls, we currently have to write them in a bunch of nested callbacks:

func fetchPortrait(
  user: UUID, 
  completion: (Image)->Void
) { 
  fetchAvatar(user: user) { url in
    fetchImage(url: url) { image in
      completion(image)
    } 
  }
}

This works, but execution jumps all over the place spatially and temporally making the code hard to reason about and prone to errors. We would like to specify our async calls procedurally, instead. One right after the other, no nesting.

Swift 5.5 solves this with async/await:

func fetchPortrait(user: UUID) async -> Image {
  let url = await fetchAvatar(user: user)
  let image = await fetchImage(url: url)
  return image
}

How are they doing that? Swift.org sheds some illumination:

An asynchronous function … is a special kind of function that can be suspended while it’s partway through execution.

Long time readers will recognize this as a coroutine. A coroutine is ideal for handling asynchronous work because we can suspend it just after it begins its async job, then resume it again right where it left off when the job completes and we’re ready to continue.

Coroutines on the Cheap

This flow of events might sound familiar. Let’s look a little closer look at the execution of a standard “completion handler” function:

func prettyHTML(
  url: URL, 
  completion: (HTML)->Void
) {
  getPage(url: url) { html in
    // Do work to format the HTML...
    completion(html)
  }
}

In this example, prettyHTML starts getting some HTML via getPage… and then returns. Some time later, the closure we passed to getPage is called, and work is done to pretty up the HTML.

In other words, prettyHTML is essentially “suspended while it’s partway through execution.” And then, at some later time, its execution is resumed again from that very spot it left off.

Yup. Completions handlers are the poor man’s coroutine. They just require that we bundle up all the state needed to “resume the coroutine” in a closure. It’s that closure that leads to the nesting and non-linear execution flow.

So we already have the building blocks for decent async handling. What we really need is a better way to manage those completion closures.

CPS Strike

So let’s zoom in on our completion handlers. Imagine the simple function:

func fetchAvatar(user: UUID) -> URL

If we want to make this asynchronous by rewriting it as a “cheap coroutine” with completion handler, what do we need to do? Rather than returning the URL we’ll move it to the argument of a new closure we pass in as a parameter:

func fetchAvatar(user: UUID, complete: (URL)->Void)

There’s a name for rewriting functions this way — it’s called Continuation Passing Style or CPS. And the closure we pass when we do this is called a continuation. Which makes sense — it’s the thing we call to “continue” where we left off.

Let’s make a type to represent a continuation:

typealias Continuation<Ret> = (Ret) -> Void

Our CPS version of fetchAvatar could then look like:

func fetchAvatar(user: UUID, c: Continuation<URL>)

In fact, if we rewrite the signature of all the functions from our async example, above, we might notice something striking:

func fetchPortrait(
  user: UUID, 
  c: Continuation<Image>
)
func fetchAvatar(user: UUID, c: Continuation<URL>)
func fetchImage(url: URL, c: Continuation<Image>)

They all take a Continuation type — the only difference is the generic type representing the value they’ll eventually be called with. If we had a way to transform a Continuation<URL> to a Continuation<Image> , we could chain these together!

In other words, given a Continuation<URL> we want a way to apply a transformation (T)->Continuation<U> that results in a new Continuation<Image>. This is a flatMap. And flatMaps are the domain of Monads.

There’s a Monad for That

A Continuation Monad wraps a Continuation. That is, it’s what happens when you render a Continuation, itself, in continuation passing style.

If a Continuation, conceptually, rewrites a value as a function that takes said value as its parameter, then a continuation of a Continuation is a Continuation rewritten as a function that takes said Continuation as its parameter.

Deep breath.

Okay, let’s to try to illustrate this in code:

// A plain old Value:
Value

// Continuation of Value:
(Value)->Void

// Continuation of Continuation of Value:
((Value)->Void)->Void

Defining types for these abstractions as we go will help keep things clear. ContinuationMonad is a continuation of a Continuation:

typealias ContinuationMonad<Value> = 
          (@escaping Continuation<Value>) -> Void

We can create one of these monads with a function that takes the value we eventually want to pass on to the continuation. We can name this function anything we like. init, makeMonad, unit… Khanlou’s post on monads covers a lot of various names traditionally used for monadic operations.

Here, we’re going to name it async:

func async<Value>(_ wrappedValue: Value) 
     -> ContinuationMonad<Value> {
  return { continuation in
    continuation(wrappedValue)
  }
}

This returns a function that, when called with a continuation, calls the continuation with the wrappedValue passed in on creation. In other words, this returns a ContinuationMonad.

Flat the Planet

Or course, our whole goal is to be able to transform between different types of continuations. In the context of a ContinuationMonad, we want to be able to, given a value T from the source monad, create a new monad with a transformed value U. Which is to say:

typealias Transform<T,U> = 
               (T) -> ContinuationMonad<U>

We apply this transform with our friend, flatMap. But the implementation of a flatMap over a continuation of a Continuation is a little intense:

func flatMap<T,U>(
  _ monad: @escaping ContinuationMonad<T>, 
  _ transform: @escaping Transform<T,U>
) -> ContinuationMonad<U> {
  return { continuation in
    monad { wrappedValue in
      transform(wrappedValue)(continuation)
    }
  }
}

Given a ContinuationMonad and a Transform, we return a function that, when called with a Continuation, will:

If this causes headaches, we are, at least, in very good company. My best advice is to throw it into a playground and step over it a couple or hundred times. Enlightenment will follow in its own measure.

Once we get over the raw mechanics, the important thing for us to recognize is none of these transformations are actually applied at the time we call flatMap. We return a thing that will call the thing that will apply the transformations when the continuation is passed in. In other words, everything is still asynchronous.

Spicy Curry

With the ContinuationMonad to guide us, our fetchPortrait implementation can create a new monad wrapping the given UUID, then flatMap it into our fetchAvatar operation:

// We’ll make this better. Promise!
func fetchPortrait(
  user: UUID, 
  c: @escaping Continuation<String>
) {
  flatMap(
    async(user),
    { aUser in 
      { aContinuation in 
        fetchAvatar(user: t, c: u) 
      } 
    }
  )
  //...
}

Yikes! What happened in the second half of our flatMap?

Well, remember the flatMap takes a Transform that looks like :

// Transform’s definition:
(T)->ContinuationMonad<U>

or, if we break it down a few steps further:

// Substitute ContinuationMonad’s definition:
(T)->(Continuation<U>)->Void

// Substitute Continuation’s definition:
(T)->((U)->Void)->Void

That’s a tricky shape to fill. And our fetchAvatar’s signature doesn’t match it:

(UUID, (URL)->Void)->Void

Though, actually, if we pull that first UUID argument off of the front, creating new function that takes only the UUID, returning a function that takes what remains of the original parameters… that fits the shape we need exactly:

// What we need:
(T)->((U)->Void)->Void

// fetchAvatar after pulling first argument 
// into its own function:
(UUID)->((URL)->Void)->Void

There’s a functional programming concept called “currying” that does precisely this. The details are probably worth a blog post on their own, and the Point•Free folk have a really good series on this if you want to know more.

But for now, suffice it to say that, given this function:

public func curry<T, U, Z>(
  _ ƒ: @escaping (T, U)->Z
) -> (T) -> (U) -> Z {
  { t in { u in ƒ(t, u) } }
}

and some function that takes two arguments like:

func example(_ i: Int, _ d: Double) -> String {
  "\(i):\(d)"
}

both these expressions are equivalent:

example(42, 33.3)        //> "42:33.3"
curry(example)(42)(33.3) //> "42:33.3"

Which lets us rewrite our messy flatMap above as:

func fetchPortrait(
  user: UUID, 
  c: @escaping Continuation<String>
) {
  flatMap(
    flatMap(
      async(user), 
      curry(fetchAvatar)
    ),
    curry(fetchImage)
  )
}

Much better!

Operator Operation

But hold the phone… the whole point of this was to get rid of nesting in our callbacks, and here we are nesting a bunch of flatMaps instead? How is this an improvement?

In short, there’s a big difference between nested closure implementations (and their captured environments) and nesting identical functions with identical signatures; not the least of which is: we can chain redundant function calls by introducing an operator.

When it comes to declaring an operator, there can be a lot of sturm and drang over the choice of symbol. Haskell uses >>= for a flatMap. That’s good enough for me:

precedencegroup FlatMapOperator { 
  associativity: left 
}
infix operator >>=: FlatMapOperator

func >>=<T,U> (
  lhs: @escaping ContinuationMonad<T>, 
  rhs: @escaping ContinuationMonadTransform<T,U>
) -> ContinuationMonad<U> {
  return flatMap(lhs, rhs)
}

With this, we can rewrite our monad chain, above, as:

async(user)
>>= curry(fetchAvatar)
>>= curry(fetchImage)

Which is feeling pretty nice. Of course, this returns a ContinuationMonad which, as we mentioned above, doesn’t actually do anything on its own. To get it jump into action, we need to give it a continuation.

We could do that with a trailing closure:

(async(user)
 >>= curry(fetchAvatar)
 >>= curry(fetchImage)) { image in
  logImage(image)
}

But more often than not, we’ll have a parent continuation we’ll want to resume when the monad chain completes. Because it’s already a continuation, we don’t need to make a new closure to hold it. We can just apply it to the preceding function.

To make this look pretty, we’ll add a backwards application operator. Haskell uses $ for this, which is problematic in Swift for a number of reasons. So we’re going to steal F#’s <| instead:

precedencegroup BackApplication {
    associativity: left
}
infix operator <|: BackApplication

public func <| <T, U>(ƒ: (T)->U, t: T) -> U {
  return ƒ(t)
}

Now we can chain async operations on our monad with >>= and apply a continuation (which will be called, asynchronously, with the result of the whole monad chain) with <|:

async(user)
>>= curry(fetchAvatar)
>>= curry(fetchImage)
<| logImage

1,800 Words Later…

Not so long ago, we set out on our quest with nothing but a nested mess of async spaghetti:

func fetchPortrait(
  user: UUID, 
  completion: (Image)->Void
) { 
  fetchAvatar(user: user) { url in
    fetchImage(url: url) { image in
      completion(image)
    } 
  }
}

But with a little understanding of continuations (and a handful of monads and operators on loan from functional languages), we’ve ended up with:

func fetchPortrait(
  user: UUID,
  completion: Continuation<Image>
) {
  async(user)
  >>= curry(fetchAvatar)
  >>= curry(fetchImage)
  <| completion
}

Not bad! Makes me feel like there’s plenty of gas left in the tank of the ol’ runtime to get us through these golden years of iOS 12–14 support.

This was a long one with a lot of scattered code, so I’ve pulled everything together into a gist. You ought to be able to paste it right into a Playground and get results — at least in Xcode 12.