Optional? Considered Harmful?

No, the Optional type in Swift, denoted by a question mark ?, is not harmful when used properly.

Optional is, however, easy to misunderstand and mis-apply, particularly for people new to programming, or new to Swift and arriving from other languages. I think the question mark syntax scares people by connoting the unknown and ominous. Many people associate that question mark with the big red compilation errors they get when they first start programming in Swift.

Xcode tries to be helpful: if you press the “Fix” button it adds the nil-coalescing operator ??, and then expects you to provide a default value. Maybe you provided an empty string "" for each of these placeholders, or the number 0, just to get your app to compile—and then you run it to find a screen full of empty fields.

The alternative Fix-It that Xcode generally offers is force-unwrapping with a ! operator, and that way lies crashes. There are so many StackOverflow questions about Unexpectedly found nil errors that they’ve collated all the answers onto a single page.

This behaviour might seem excessively pedantic or un-ergonomic, particularly if you’re coming to Swift from a language like JavaScript. Surely, according to the literature, it’s supposed to be a “joy to write”?

In short: it’s a short-term pain for a long-term gain. With the Optional type Swift is preventing null referencing, a common kind of programmer error where you try to use something that’s declared, but not actually there. In almost all cases, this will save you time debugging in the future. Elsewhere on the Swift about page, you’ll find (emphasis mine):

Swift is safe. Undefined behavior is the enemy of safety, and it’s best to catch mistakes before software goes into production. Swift makes the obvious path the safest.¹

Null referencing can take all kinds of forms, but typically happens because the thing you’re trying to use has been de-initialised (gone out of memory), or because it never existed in the first place. Sir Tony Hoare (of Quicksort fame) invented the null reference in ALGOL W in 1965, which he valiantly took the fall for in a 2009 talk at QCon where he described it as his “billion dollar mistake”. In the talk, he explains that Fortran programmers refused to use the Fortran-to-ALGOL transpiler because their programs would immediately produce a subscript error; rather than resolving the issue, these people just wanted the programs to run. In Sir Tony’s words: “Fortran programmers preferred to… experience disaster rather than check all their subscripts.”

The kind of problems Optional protects you from

In some other programming languages, null referencing is a common source of runtime errors. For instance, in Java, the code below compiles, but if you tried to use any of the variables, you’d get a NullPointerException at runtime. Modern IDEs like IntelliJ IDEA might show a warning, but it won’t stop this from compiling, and running, and crashing:

// You can declare a variable in Java without assigning anything to it.
String name;

// You can also explicitly set variables to `null`.
String quest = null;

// You can also set something to `null` after it's initially been assigned to something else.
String favouriteColour = "blue";
favouriteColour = null;

System.out.println(favouriteColour.toUpperCase()); // produces a NullPointerException

This is why Java projects are often riddled with this kind of construction:

if (name == null) {
    throw new NoNameException();
}

// do something useful

A NullPointerException is the closest thing in the JVM ecosystem to Swift’s Unexpectedly found nil error.

How Swift banishes null referencing with Optional

Swift does have a nil value, a special value which you can only use in a context that has been declared Optional. You can write out the full Optional type definition with angle brackets, but Swift also lets you write the type name followed by a question mark ? as shorthand.

// These two declarations are equivalent
var favouriteColour: Optional<String> = "blue"
var favouriteColour: String? = "blue"

In statically-typed programming languages like Swift, the type system is a kind of labelling—for the compiler, but also for programmers looking at your code in future (including you.) You can see declaring something as Optional as the equivalent of you telling a future programmer (maybe yourself):

“There might not be a value here. You need to account for what happens if there is nothing, because that’s a valid and plausible state for the program to be in.”

The upshot of this is that the error we saw in the Java code above are impossible under normal circumstances in Swift. The compiler is strict and will stop you² from declaring a constant or variable without giving it a value. It will also stop you from assigning nil to a constant or variable that has not been declared Optional. The equivalent of that code wouldn’t even compile, let alone run!

let name: String  // Won't compile unless you assign a value to `name` in the same scope, before it is used
let quest: String = nil  // won't compile because the types don't match

var favouriteColour = "blue"
favouriteColour = nil  // also won't compile because the types don't match

print("My favourite colour is \(favouriteColour)") // Will never execute, because the code won't compile :)

The type theory behind Optional

Type theorists call this an option type, or a maybe type. It’s a special case of the either type, which is also a kind of monad.³ In short, an Optional can be in one of two states:

• Contains something, which must be a value of the specified type
• Contains nothing, in which case it is nil

Swift provides a great way to model mutually exclusive cases as an enum,⁴ so we can think about Optional like this. Because enums in Swift can have associated values, you can even store the wrapped value on the “something is here” case. Here’s the Optional type, as represented by an enum:

/// This type is a message... and part of a system of messages...
/// pay attention to it!
/// Sending this message was important to us.
/// We considered ourselves to be a powerful culture.
@frozen public enum Optional<Wrapped> : ExpressibleByNilLiteral {

    /// * This is not a place of honor
    /// * No highly esteemed value is stored here
    /// * This message is a warning about nothingness
    /// * This place is best left shunned and uninhabited
    case none
    
    /// * This **IS** a place of honor
    /// * A highly esteemed value **IS** stored here
    /// * The value can be unwrapped and used with gay abandon
    case some(Wrapped)
}

Surprise—this kind of enum is exactly what an Optional is under the hood in Swift. You can see it yourself by command-clicking on it in Xcode, or going to where it’s defined in the Swift source code.⁵ The question mark notation is just bit of syntactic sugar that makes it less cumbersome to write.

Underneath, an Optional is just an enum like any other. In Swift, unlike in Objective-C, nil is not a pointer or a reference—it’s just a special value (called a “literal”) that represents Optional.none.

This gives Optional a number of interesting properties:

1. You can use them with switch statements and expressions. In Swift, switch statements have to be exhaustive. This offers an additional level of safety, guaranteeing all cases are accounted for.⁶
2. This is extra powerful when combined with pattern matching. You can match on optionals which contain an enum; you can match on enums with optional associated values.
3. There’s some more syntactic sugar: you can omit the .some from your case declarations when you have an enum wrapped inside an Optional. You can effectively write as if Optional.none is an additional case in your inner enum.

This allows us to switch on an enum with a associated optional values and be confident that we’ve handled every single case that could possibly arise.

A worked example of how this makes undefined behaviour impossible

Let’s imagine I want to represent various items in my media library: books, TV shows, and movies. We could represent this with a big flat type festooned with optionals:

struct ItemFlat {
    var title: String
    var kind: Kind?
    
    // MARK: Books only
    var authors: [String]?
    var publisher: String?
    var pages: Int?
    
    // MARK: Movies & TV shows
    var director: String?
    var actors: String?
    var producers: [String]?
    
    // MARK: TV shows only
    var numberOfEpisodes: Int?
    
    // MARK: Movies only
    var runtime: Int?
    
    enum Kind {
        case book
        case tvShow
        case movie
    }
}

This is not ideal for a number of reasons:

• If I inspect a single FlatItem instance and find that authors is nil, I don’t know if that’s because the FlatItem is a TV show or movie (which can’t have an author) or if it is a book, but has an anonymous author.
• The compiler allows me to assign values that are not relevant to that kind of media. For instance, I could give a book a numberOfEpisodes. I could mitigate against this by adding some documentation, but this gives more for people to read.
• When I come to use this type, at the call site I would need to write logic to check what media kind I’m dealing with, then extract the fields relevant to that kind of media, and then I would need to unwrap or nil coalesce the values because they’re all Optional.

But because Swift allows us to write enums with associated values, I could instead write something like this:

struct ItemBackedByEnum {
    var title: String
    var metadata: Metadata?
    
    enum Metadata {
        case book(authors: [String]?, publisher: String, pages: Int)
        case tvShow(director: String, actors: [String], producers: [String], numberOfEpisodes: Int)
        case movie(director: String, actors: [String], producers: [String], runtime: Int)
    }
}

We can write code that switches on the optional Metadata type—an enum nested within an enum—and this allows us to guarantee we have handled every possible case that could arise:

let credits = switch myItemBackedByEnum.metadata {
case let .book(.some(authors), publisher, pages):
    "Written by \(authors.joined(separator: " & ")), published by \(publisher), \(pages) pages"
case let .book(.none, publisher, pages):
    "Written by an anonymous author, published by \(publisher), \(pages) pages"
case let .tvShow(director, actors, producers, episodes):
    "Directed by \(director), starring \(actors.joined(separator: " & ")), produced by \(producers.joined(separator: " & ")), \(episodes) episodes"
case let .movie(director, actors, producers, runtime):
    "Directed by \(director), starring \(actors.joined(separator: " & ")), produced by \(producers.joined(separator: " & ")), \(runtime) minutes long"
case .none:
    nil
}

This offers a number of advantages:

• It allows me to specify what properties each kind of media can have. This makes our previous example (e.g. of a book having a numberOfEpisodes) impossible.
• I can tell the difference between a book with no author (where its author associated value is nil) and another type of media that has no author because it’s not a book.
• I have not had to use a single force-unwrap ! or nil-coalescing operator ??. Once you start to build your data models in this way, you will start noticing the difference. These language features allow you to write code that is more expressive, tidier at the point of use, and makes undefined behaviour impossible.

Dealing with big types with lots of optional fields

Of course, much of the time, you will be handling data from various sources—be those imported files, or from remote sources such as APIs—where ‘large type with lots of Optionals’ is just how the data is encoded, for example as a JSON file or in a database table.

For this, it can be helpful to write a transitional ‘raw’ type that represents how your data arrives in your app. Then you can write a richer, more ergonomic type which you will use as your data model, and abstracts away the complexities of decoding the JSON. Your goal here is to minimise the amount of time you’re handling your raw type.

The key thing to remember here is to check the contract of the API you’re calling. If your response is missing some non-optional fields, or has some fields set to null that should be present given some other condition or invariant, then the API is returning junk and you should throw an error. Don’t be afraid of throwing errors if something is wrong—it’s better than defaulting to an empty string and then getting a screen full of blank fields!

Some Optional hygiene tips and antipatterns

Handling cases where an Optional touches the UI with nil coalescing

Generally, I only use the nil coalescing operator ?? to assign a default value in my UI code—for instance, if I’m populating a form with some optional values, but the control I’m using to display the value doesn’t accept Optionals. For instance, here’s a line of SwiftUI where I define an optional field (serialNumber) in a form in Unspool:

TextField(UIStringKey.rollMetadata(.serialNumber).key,
          text: $model.camera.serialNumber ?? "")

When you come to save your data back to your model, be careful that you don’t save empty/placeholder values back to the model—this means you’d lose the rich type information and be saying you “have” a serial number, even if it’s an empty string. Ideally, you want to avoid the state where your Optional has a value, but it’s junk.

I like to handle this particular case with Strings by adding a short extension to all my Swift projects:

extension String {
    var nilIfEmpty: String? {
        self.isEmpty ? nil : self
    }
}

Problem: Things defined as Optional when they’re not

Solution: Start by just removing question marks and seeing where the compiler complains. If you start tugging on this thread and find that all you need to do is remove the question mark from the type definition, then your value never needed to be Optional in the first place.

Returning nil when an error occurs rather than throwing

Solution: Determine if it is valid—i.e. expected behaviour—for the function or value not to have a result. In this case, it is acceptable to return nil—e.g. if you have a field in an Address type for houseName (not everyone’s house/building has a name.)

If the absence of a value is because of an error, then you should actually handle that error, or throw it upwards. You can also look at using the Result type instead, which will either contain a successful result or a throwable error—this is useful when it’s a “valid” state for your program for an error to have occurred.

Optionalception! (i.e. optionals that contain other optionals)

If you come across a situation where a type’s annotation ends in ??, this can be not especially ergonomic and lead to confusion about how to handle it.

Consider this code to find the author of the first in a list of poems:

struct Poem {
    var author: String? // nil means the poem's author is anonymous
    var body: String    
}

// define poems list

var allPoemAuthors = poemsList.map { $0.author }
var firstPoemAuthor: String?? = allPoemAuthors.first

Because the type of author is Optional<String>, the type of allPoemAuthors is Array<Optional<String>>. And because Array.first can return nil if the array is empty, there is a semantic difference between these two cases:

• the outer Optional is nil (there are no values in the poems list)
• the inner Optional is nil (i.e. there is a first value in poemsList, but its author value is nil)

There are a number of strategies to avoid this situation. You could instead try using optional chaining to drill down into the property of poemsList.first (in our case this would actually save us time since we’re only interested in the first one):

var firstPoemAuthor: String? = poemsList.first?.author

On the other hand, if you do want to filter a collection of Optionals and get the non-nil values, this is also possible:

var poemAuthors = poemsList.filter { $0.author != nil }.map { $0.author! }

But there is an even better way to do this: compactMap(_:) allows you to return the non-nil results of a transformation, with no need to force unwrap or filter first:

var poemAuthors = poemsList.compactMap { $0.author }

To do this on an optional that is not in a sequence, you can use flatMap(_:), to transform an Optional using a transformation that might also return Optional:

var poemAuthor = firstPoem.flatMap { $0.author }

As highlighted in Paul Hudson’s post on Hacking With Swift on this topic, common situations where this might apply are type conversions (e.g. between String and numeric types), or when you’re performing another transformation that might legitimately fail.

(This is based on a lightning talk I gave at iOSDevUK in 2023. With thanks to Lena Stöxen, Connor Habibi, Robert Hillary, and Rob Whittaker for letting me bounce ideas off them, and to Dr Paul Cadman for setting the standard. Apologies to Chris Lattner, C.A.R. Hoare, and the late Edsger W. Dijkstra.)