Functional error-handling with stack traces

17 Dec 2022software-development error-handling

Over the time I have developed a go-to approach for error-handling in my pure functional Scala projects.

I've seen developers thinking that exceptions and their benefits don't play well together with a (pure) functional programming style. But this is a wrong assumption. Functional error-handling and exceptions can be combined to get the best of both worlds and here I'll show how.
In my examples I use the ZIO library, but it works similar with Cats-Effect or other async/effect libraries.

Why exceptions?

Some developers use an error-handling approach that relies on custom defined classes/types which are unrelated to Exceptions. They return those errors using a Result type such as Scala's builtin Either. While this works, it commonly has the severe drawback of losing the ability to inspect the stack trace when debugging an error.

Having a meaningful stack trace can make the difference between reading the logs for a minute or having to dig into a problem for days. They are just too valuable not to use them, so I always use exceptions for my error-types.

I start with a common exception-class for the whole project. It gets a unique name, is easy to create and spot in logs / stack traces.

class GeneralException(message: String, cause: Throwable) extends Exception(message, cause)
object GeneralException {
  def apply(message: String, cause: Throwable = null) = new GeneralException(message, cause)
}

Using it is simple.

if(condition) {
  throw GeneralException("boom")
}

Or, in a pure functional style using ZIO as the Result type:

if(condition) {
  ZIO.fail(GeneralException("boom"))
} else {
  ZIO.succeed(42)
}

It is generally much better to define and use semantically meaningful error-types. But everyone is lazy sometimes. In those cases, using GeneralException directly is still much better than relying on Java's builtin Exception.

The next step is therefore to specific errors for all the things that can go wrong. Those help to make code easier to read and also allow to catch and handle only specific errors while passing along others.

sealed trait Error extends GeneralException

case class UserNotFound(id: UserId, cause: Throwable = null)
  extends GeneralException(s"User with id $id not found", cause) with Error

case class SendingEmailFailed(email: Email, cause: Throwable = null)
  extends GeneralException(s"Sending email to $email failed", cause) with Error

case class SendingSmsFailed(phoneNumber: PhoneNumber, cause: Throwable = null)
  extends GeneralException(s"Sending SMS to $phoneNumber failed", cause) with Error

case class SendingNotificationFailed(id: UserId, cause: Throwable = null)
  extends GeneralException(s"Notifying user with id $id failed", cause) with Error

In this example I am defining just a flat list of errors, but this can totally become a complex hierarchy with multiple common error-traits. You might have noticed that the cause parameter is set to null by default - something we usually try to avoid in the Scala world. However, in this case I make an exception for improved ergonomics. Error-handling is really one of those things that need to be as easy as possible. Otherwise, people tend to take shortcuts.

Using it in code

Here is how it looks when I write my business logic using different methods that throw the errors above.

def notifyUser(userId: UserId, message: Text) = {

  val notificationResult = for {
    user <- getUser(userId)
    notificationSentTimestamp <- if (user.preferences.prefersEmail) {
      sendEmail(user.email, message)
    } else {
      sendSms(user.phoneNumber, message)
    }
  } yield notificationSentTimestamp

  notificationResult.mapError(e => SendingNotificationFailed(userId, e))
}

I find this pure-functional monadic approach of error-handling quite elegant.

In the example, getUser might fail with a UserNotFound-error and sendEmail and sendSms might fail with SendingEmailFailed and SendingSmsFailed respectively.

The different actions are executed and if an error occurred then I turn it into a SendingNotificationFailed-error. Wrapping errors is optional. If I were to remove this line, the resulting error-type would just be a combination of all possible errors that can occur. But it's often nicer to hide the underlying errors and make the result more meaningful - and the method-signature shorter.

So what exactly is the return type of the notifyUser function?
If I keep the last line then it is:

def notifyUser(userId: UserId, message: Text): IO[SendingNotificationFailed, NotificationSentTimestamp]

If I remove it becomes:

def notifyUser(userId: UserId, message: Text): IO[UserNotFound | SendingEmailFailed | SendingSmsFailed, NotificationSentTimestamp]

If I do not add an explicit return type to the method, it will be inferred for me by the compiler, including all possible errors. That means, when adding more functionality later (e.g. additionally sending a push notification to the smartphone) or changing the error-handling, I won't have to refactor the whole chain of methods that call each other.
This is very useful, because a small change can otherwise force to refactor dozens of method signatures.

Note that IDEs like IntelliJ can still show the return type inline, so even if it is not explicitly annotated, it is still visible and can serve as documentation.

The resulting logs

Using this approach proves me with a very nice stack trace when I log an error. Most developers in the JVM world should be familiar with that.
Let's assume that my call to notifyUser failed because the SMS could not be sent. When looking into the logs, I will see something like this:

Exception in thread "main" SendingNotificationFailed: Notifying user with id xxx failed
    at MyProject.method3(MyProject.scala:19)
    at MyProject.method2(MyProject.scala:12)
    at MyProject.method1(MyProject.scala:8)
Caused by: SendingSmsFailed: Sending SMS to 123456789 failed
    at MyProject.method5(Sms.scala:143)
    at MyProject.method4(Sms.scala:77)
Caused by: ConnectionError: Could not connect to SMS provider XXX
    at org.example.SmsProvider.connect(Foo.scala:123)
    ... 2 more

And so on. It's a nice chain of errors which makes it very easy to understand both the high-level impact and the root cause.

API errors

If this is some internal process in my application then this kind of error-logging can already be good enough.

But usually I need to expose errors to a third party such as an external system or an end-user. In that case I typically don't want to expose internal technical details.
I also want to keep my API stable. Therefore, I define another layer of error-types which are describing what my API exposes.

sealed trait ApiError

case class ApiUserNotFound(id: UserId, cause: Throwable = null)
  extends GeneralException(s"The user with id $id was not found", cause) extends ApiError

case class ApiSendingNotificationFailed(id: UserId, cause: Throwable = null)
  extends GeneralException(s"Notifying user with id $id was not successfull", cause) extends ApiError

My internal errors are created at some point during the execution and before returning a response, I turn them into API errors.

def toApiError(error: Error): ApiError = error match {
  case e @ UserNotFound(id, _) => ApiUserNotFound(id, e)
  case e @ SendingNotificationFailed(id, _) => ApiSendingNotificationFailed(id, e)
  // ... and so on
}

This looks a bit redundant but has an advantage: I can now modify my internal errors in any way I want - if I modify them in a way that would impact my API then the compiler will make me aware of it because transforming internal errors into api errors will fail to compile.

Adding references and codes

On top of that, I find it useful to add two more things: error-references and error-codes.
Error-references are unique identifiers that allow me to find the error in the logs.
Error-codes are a well-defined list where each entry describe the type of error so that an external system can react to specific errors easily.

Error codes

Let's start with error codes. I'm extending each of my ApiError types with a specific error-code:

type ErrorCode = String // for readability

sealed trait ApiError

case class ApiUserNotFound(id: UserId, cause: Throwable = null, errorCode: ErrorCode = "EC_UserNotFound")
  extends GeneralException(s"The user with id $id was not found", cause) extends ApiError

case class ApiSendingNotificationFailed(id: UserId, cause: Throwable = null, errorCode: ErrorCode = "EC_NotificationError")
  extends GeneralException(s"Notifying user with id $id was not successfull", cause) extends ApiError

I have used different formats here - numbers, strings, whatever fits my needs, but if I have the choice I prefer to add a prefix and use easy to read names.
To keep the example simple I'm using strings and default-arguments here, but optimally an enum is used here to ensure that error-codes are never reused by accident.
The ValueEnum of Enumeratum can be used to do so.

Error references

Error-codes are nice to indicate the type of error, but they won't make it very easy to debug a problem when getting an error report from a user. To make my life easier in such situations I add unique references whenever an error happens.

For that, I extend the original GeneralException class:

type ErrorReference = String

class GeneralException(message: String, cause: Throwable, errorReference: ErrorReference = GeneralException.createReference())
  extends Exception(s"$message | error-reference: $errorReference", cause)

object GeneralException {
  def apply(message: String, cause: Throwable = null, errorReference: ErrorReference = GeneralException.createReference()) = new GeneralException(message, cause, errorReference)
  def createReference(): ErrorReference = "ER_" + scala.util.Random.alphanumeric.take(12).mkString
}

Be aware that this code is not pure functional and is executes an effect at the time of the error-creation (generating a random reference). Again I believe that this is a good trade-off for the ease of use.
Depending on the system it is worth to spend some time to come up with an identifier format that e.g. includes information about time and hence is sortable and has fewer collisions... just in case there are a lot of errors - I hope there aren't :^).

Also note that the error-reference is automatically appended to the message, so it is guaranteed to appear in the stack trace when an error is logged.

I usually add this reference to my API error-types and then expose it to the end-user in case they want to report an error.

Conclusion

A few best practices are violated here, including class inheritance, usage of null and non-pure-functional code to create references on error-creation.
But overall, with only a little extra effort I get the best of both worlds: clean error-handling for both reading and usage, as well as comprehensive stack traces that make debugging easy.

Next: Context-dependant code is better than its reputation ►

Functional error-handling with stack traces

Why exceptions? #

Using it in code #

The resulting logs #

API errors #

Adding references and codes #

Error codes #

Error references #

Conclusion #