This article is part of A Guide to Express API Validation.
When you’re creating an Express API it can be difficult to know how to handle error cases and send consistent error responses. It becomes even more complicated if you want to send helpful error responses with extra details about what went wrong.
You know these extra details are needed because they’ll also be super helpful for debugging requests to your API, but before you know it, you find yourself designing your own error response format. It all feels awkward, and like it’s probably something you shouldn’t be doing, but what alternative is there?
Thankfully, there’s an awesome alternative, and you can find it in the ‘Problem Details for HTTP APIs’ specification (RFC7807). Don’t worry though, I don’t expect you to go and read the whole RFC (Request for Comments) document. I know that RFCs aren’t always the easiest of reads, but I think the ideas in this one are so good that I’ve done the RFC reading for you and pulled out all of the good stuff that can help you with formatting your API error responses.
In this article we’ll explore the Problem Details specification and how it can help you build better APIs. By learning how to apply this well-defined and structured approach, your struggles with creating API error responses will be a thing of the past.
Jump links
Introducing the ‘Problem Details for HTTP APIs’ specification
The aim of the problem details specification is to define a common error format which you can use for the error responses from your API. This avoids having to invent your own error response format or, even worse, attempting to redefine the meaning of existing HTTP status codes. Seriously, don’t do this! The meaning of HTTP status codes are well documented and commonly understood for a reason.
The status codes defined in the HTTP specification are very useful, and often provide enough context to the client as to what went wrong, but they don’t always convey enough information about an error to be helpful.
Take for example the status code 422 (Unprocessable Entity) – as defined in the HTTP specification, it tells a client that the server understood the request body and its structure, but was unable to process it. However, that alone doesn’t tell the client specifically what was wrong with the JSON that was sent in the request body. Problem details can help you solve this problem.
The specification describes a problem detail as "a way to carry machine-readable details of errors in a HTTP response". Let’s take a look at how the problem details specification defines them.
Problem types and Problem details objects
The problem details specification defines what a "problem type" and a "problem details object" are, and their relationship:
Problem type – A problem type definition must include a type URI (typically a URL), a short title to describe it and the HTTP status code for it to be used with.
If required, the definition can also specify additional properties to be included on problem details objects which use this type e.g. balance and accounts in the example above. These additional properties are referred to as "extensions" by the specification.
The type URI is effectively the namespace for the problem type definition. If the definition changes, the type should also change.
You should avoid defining a new problem type when the response HTTP status code provides enough context by itself. The specification gives the following example: "a ‘write access disallowed’ problem is probably unnecessary, since a 403 Forbidden status code in response to a PUT request is self-explanatory".
Problem details object – An object which includes the type, title and status properties for a problem type. This object represents a specific occurrence of that problem type. It can optionally contain a detail property – a human-readable explanation specific to this occurrence of the problem – and an instance property – a URI reference that identifies the specific occurrence of the problem.
A problem details object should include values for any extensions specified by the problem type definition.
Problem detail objects can be formatted as XML or JSON. For the purpose of this article we’ll be using JSON formatted problem details.
Example problem details response
The response body in this example contains a problem details object of the type https://example.com/probs/out-of-credit:
HTTP/1.1 403 Forbidden
Content-Type: application/problem+json
Content-Language: en
{
"type": "https://example.com/probs/out-of-credit",
"title": "You do not have enough credit.",
"detail": "Your current balance is 30, but that costs 50.",
"instance": "/account/12345/msgs/abc",
"balance": 30,
"accounts": ["/account/12345", "/account/67890"]
}— Source: RFC7807 – Problem Details for HTTP APIs.
Note how the example response above contains the header Content-Type: application/problem+json. This is the media type for JSON problem details which is defined by the problem details specification. Clients can use the Content-Type header in a response to determine what is contained in the response body. This allows them to handle different types of response bodies in different ways.
Any response containing a problem details object must also contain the Content-Type: application/problem+json header.
More details, clearer problems
Including problem details in the response body allows the client to derive more information about what went wrong, and gives it a better chance of being able to handle the error appropriately. Every problem details object must have a type property. The client can then use the value of the type to determine the specific type of problem which occurred.
In the example problem details object above (Example 3.1), the problem can be identified as an "out of credit" problem when the client checks the value of the type field: https://example.com/probs/out-of-credit
The type for a problem can be specific to your API, or you can potentially reuse existing ones if you wish.
Breakdown of a problem details object
To better understand the properties which make up a problem details object, let’s break it down and look at each property. Let’s start with our example problem details object:
{
"type": "https://example.com/probs/out-of-credit",
"title": "You do not have enough credit.",
"detail": "Your current balance is 30, but that costs 50.",
"instance": "/account/12345/msgs/abc",
"balance": 30,
"accounts": ["/account/12345", "/account/67890"]
}Now let’s go through this line by line:
"type": "https://example.com/probs/out-of-credit",The type URI for the problem type being used by this problem details object. The specification encourages that this is a real URL which provides human-readable documentation in HTML format. The client should use the value of this field as the primary identifier for the problem.
"title": "You do not have enough credit.",The title defined by the problem type.
"status": 403,The HTTP status code defined by the problem type. Should be the same as the status code sent in the response from the API.
As intermediaries between the client and the server (e.g. a proxy or a cache) might modify the response status code, this value can be used by the client to determine the original status code of the response. Also useful in situations where the response body is the only available part of the response e.g. in logs.
"detail": "Your current balance is 30, but that costs 50.",A human-readable explanation of the problem. It should focus on helping the client correct the problem. Machine-readable information should be added in extensions (see below). Specific to this occurrence of the problem.
"instance": "/account/12345/msgs/abc",A URI reference for the specific problem occurrence. Typically a URL, optionally containing more information. Specific to this occurrence of the problem.
"balance": 30,
"accounts": ["/account/12345", "/account/67890"]Extensions specified by the problem type. Specific to this occurrence of the problem.
The type, title and status – as defined by a problem type – should be the same for every occurrence of the problem.
Note: As with any response you send from your API, you should be careful when creating problem details objects that you don’t expose any of the implementation details of your application, as this can make it potentially vulnerable to attack.
How to send problem details responses with Express
Now that we’ve covered the concepts and conventions of problem details, we can write some code. This code will allow us to send problem details error responses from our Express API.
Note: This article uses ECMAScript (ES) module syntax. ES modules are well supported in the v12 and v14 releases of Node.js.
For Node.js 12.x, I recommend using v12.20.0 or higher. For Node.js 14.x, I recommend using v14.13.0 or higher. These releases of Node.js have stable ES module support, as well as improved compatibility with the older style CommonJS modules, which are still widely used.
Define problem types and map them to JavaScript error classes
In this code we’re going to define two different problem types and map them to JavaScript error classes – in this case, ones that are provided by the http-errors library. We’ll use these problem types later on when we create an error handler middleware.
// src/middleware/problem-details-response.js
import createHttpError from "http-errors";
const defaultProblemDetails = {
/**
* This is the only URI reserved as a problem type in the
* problem details spec. It indicates that the problem has
* no additional semantics beyond that of the HTTP status code.
*/
type: "about:blank",
status: 500,
};
const problemTypes = [
{
matchErrorClass: createHttpError.BadRequest,
details: {
type: "https://example-api.com/problem/invalid-user-id",
title: "User ID must be a number",
status: 400,
},
},
{
matchErrorClass: createHttpError.Forbidden,
details: {
type: "https://example-api.com/problem/user-locked",
title: "User has been locked",
status: 403,
},
},
];Look up the problem details for an error
Now let’s create a function which, when passed an error object, will look through our array of problemTypes for one which has been mapped to the type of error it has received:
// src/middleware/problem-details-response.js
/**
* Get the problem details which have been defined for an error.
*
* @param {Error} error
* @return {Object} - Problem details (type, title, status)
*/
function getProblemDetailsForError(error) {
const problemType = problemTypes.find((problemType) => {
/**
* Test if the error object is an instance of the error
* class specified by the problem type.
*
* @see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/instanceof
*/
return error instanceof problemType.matchErrorClass;
});
if (!problemType) {
/**
* A problem type hasn't been defined for the type of error
* this function has received so return fallback problem details.
*/
return defaultProblemDetails;
}
return problemType.details;
}Create an error handler to send a problem details response
This error handler middleware is going to call the getProblemDetailsByError() function which we just defined, and then send the problem details that it returns as a response body, along with the correct HTTP status code and Content-Type header:
// src/middleware/problem-details-response.js
/**
* Send an error response using the problem details format.
*
* @see https://tools.ietf.org/html/rfc7807
*
* @param {Error} error
* @param {Object} request - Express request object
* @param {Object} response - Express response object
* @param {Function} next - Express callback function
*/
function problemDetailsResponseMiddleware(
error,
request,
response,
next
) {
/**
* If response headers have already been sent,
* delegate to the default Express error handler.
*/
if (response.headersSent) {
return next(error);
}
const problemDetails = getProblemDetailsForError(error);
/**
* If the problem details don't contain an HTTP status code,
* let's check the error object for a status code. If the
* error object doesn't have one then we'll fall back to a
* generic 500 (Internal Server Error) status code.
*/
if (!problemDetails.status) {
problemDetails.status = error.statusCode || 500;
}
/**
* Set the correct media type for a response containing a
* JSON formatted problem details object.
*
* @see https://tools.ietf.org/html/rfc7807#section-3
*/
response.set("Content-Type", "application/problem+json");
/**
* Set the response status code and a JSON formatted body
* containing the problem details.
*/
response.status(problemDetails.status).json(problemDetails);
/**
* Ensure any remaining middleware are run.
*/
next();
};
export default problemDetailsResponseMiddleware;Use the problem details response error handler
Our error handling middlware will be run when an error object is passed to a next() Express function. When the next() function is called with with an error object, it automatically stops calling all regular middleware for the current request. It then starts calling any error handler middleware which has been configured.
It’s time to pull everything together. Here is a complete example Express API application, configured to use our problem details error handler middleware:
// src/server.js
import express from "express";
import createHttpError from "http-errors";
import problemDetailsResponseMiddleware from "./middleware/problem-details-response.js";
/**
* Express configuration and routes
*/
const PORT = 3000;
const app = express();
/**
* In a real application this would run a query against a
* database, but for this example it's returning a `Promise`
* which randomly either resolves with an example user object
* or rejects with an error.
*/
function getUserData() {
return new Promise((resolve, reject) => {
const randomlyFail = Math.random() < 0.5;
if (randomlyFail) {
reject(
"An error occurred while attempting to run the database query."
);
} else {
resolve({
id: 1234,
first_name: "Bobo",
is_locked: true,
});
}
});
}
/**
* This route demonstrates:
*
* - Creating an error when the user ID in the URL is not numeric.
* - Creating an error when the (faked) user object from the database
* is locked.
* - Catching a (randomly faked) database error (see `getUserData()`
* function above).
* - Passing all error objects to the `next()` callback so our problem
* details response error handler can take care of them.
*/
app.get("/user/:user_id", (request, response, next) => {
const userIdIsNumeric = !isNaN(request.params.user_id);
if (!userIdIsNumeric) {
const error = new createHttpError.BadRequest();
return next(error);
}
getUserData()
.then((user) => {
if (user.is_locked) {
const error = new createHttpError.Forbidden();
return next(error);
}
response.json(user);
})
.catch(next);
});
app.use(problemDetailsResponseMiddleware);
app.listen(PORT, () =>
console.log(`Example app listening at http://localhost:${PORT}`)
);Example problem details error responses
Here are the error responses that are produced by the code that we’ve just put together:
< HTTP/1.1 400 Bad Request
< Content-Type: application/problem+json; charset=utf-8
< Content-Length: 106
{
"type": "https://example-api.com/problem/invalid-user-id",
"title": "User ID must be a number",
"status": 400
}< HTTP/1.1 403 Forbidden
< Content-Type: application/problem+json; charset=utf-8
< Content-Length: 98
{
"type": "https://example-api.com/problem/user-locked",
"title": "User has been locked",
"status": 403
}< HTTP/1.1 500 Internal Server Error
< Content-Type: application/problem+json; charset=utf-8
< Content-Length: 35
{
"type": "about:blank",
"status": 500
}Just look at those beautiful structured error responses! ✨
Next steps
Now that you've learnt all about the clarity that problem details can bring to your error responses, I hope you're excited to start using them in your own APIs!
Want to learn more about how you can build robust APIs with Express? Take a look at some of my other articles:
Overwhelmed trying to implement validation in your Express API?
I've written a book that pulls together all of the validation concepts and techniques that I've shared through articles here on this blog. It combines them into a complete API validation strategy that you can immediately start applying in your Express applications.
"Concise and a pleasure to read. This book has clarified concepts for me that I couldn't get right even after years of consulting the Express documentation."
— Carles Andres, Lead Engineer at Tellimer
The book is packed with practical code examples and includes a handy JSON Schema cheat sheet.