Migrating from Express to Fastify, Part 2

In Part 1, we looked at the features of the Fastify Node.js Web framework compared to Express.js. In Part 2, we'll work through migrating an example Express.js application to Fastify.

The example application we'll migrate is called node-express-realworld-example-app, which is an Express/MongoDB-based implementation of the RealWorld example app backend. RealWorld is a specification of a simple blogging website, so the backend component involves managing users, articles, and comments; it's a good demonstration project since it's just large enough to be a decent representation of a production project.

The code for this example can be found here; at each step in the migration, there'll be a link to follow along with the complete set of changes.

Incremental migration using @fastify/express

Any code refactor is best done incrementally, so that it can be tested every step along the way. Fastify provides a core plugin called @fastify/express which makes this possible during an Express-to-Fastify migration. It can be used in a few different ways, but the feature that's useful in this example is its ability to load an entire Express application inside of a Fastify application.

With this capability, an Express application can be migrated to Fastify in a three-step process:

1. Replace the Express server component with Fastify, using @fastify/express to load the existing Express router
2. Migrate routes from Express to Fastify, one by one
3. Remove Express, associated plugins, and @fastify/express from the project

An overview of the example project

The initial state of the project can be browsed here. It's structured in a straightforward way:

• config/: A few modules for configuring environment variables and the passport library used for authentication
• models/: Contains the Mongoose models used for interfacing with MongoDB
• public/: A directory for static assets
• routes/: Modules defining the API endpoints of the application
• tests/: Contains Newman-based end-to-end automated testing
• app.js: The main entrypoint where the Express server is started

The first step in the migration will be to change app.js such that it starts a Fastify server, and using the @fastify/express plugin in order to load the existing Express application. The end result is an application that works no differently than before, but it'll be ready for Fastify routes to be added which co-exist with the Express ones.

Step 1: Replacing the server component

First, a few new packages need to be installed:

npm install --save fastify @fastify/express fastify-plugin @fastify/formbody

Note that fastify-plugin is a utility we'll use for defining Fastify plugins, and @fastify/formbody is for an Express-related workaround we'll discuss later.

Next, we'll want to move the existing Express routes from routes/ into a new routes/legacy/ directory, updating require() statements accordingly. This leaves the routes/ directory ready to accept new Fastify routes in step 2.

With those preparations complete, it's time to replace the Express server in app.js with a Fastify server. Currently, what that module is doing is:

Instantiating an Express object...

var express = require('express');
var bodyParser = require('body-parser');
var cors = require('cors');
var mongoose = require('mongoose');
/* ... */

var isProduction = process.env.NODE_ENV === 'production';

var app = express();

...configuring it, applying middleware, and setting up a database connection...

app.use(cors());

app.use(require('morgan')('dev'));
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());

/* ... */

if(isProduction){
  mongoose.connect(process.env.MONGODB_URI);
} else {
  mongoose.connect('mongodb://localhost/conduit');
  mongoose.set('debug', true);
}

require('./models/User');
require('./models/Article');
require('./models/Comment');
require('./config/passport');

app.use(require('./routes'));

/* ... */

...and finally, starting the server:

var server = app.listen( process.env.PORT || 3000, function(){
  console.log('Listening on port ' + server.address().port);
});

Fastify works similarly, but it has a plugin system which provides a flexible way to extend a Fastify server's capabilities. One property that plugins can enforce is encapsulation: ensuring that the capabilities added by a plugin can only be used within that plugin.

In this first step where we intend to temporarily load an Express application within a Fastify application, encapsulation is a great way to ensure that none of the Fastify-based code can inadvertently use functionality added by the Express application, so it makes sense to create a Fastify plugin for it. For now the entirety of the application will exist in there, but we'll change that incrementally later.

The Fastify version of app.js is similar in structure, defining a build() function which sets up the database and loads plugins, and then running it and calling a .listen() method to start the server:

var isProduction = process.env.NODE_ENV === 'production';

async function build() {
  const mongoose = require('mongoose');

  if(isProduction){
    mongoose.connect(process.env.MONGODB_URI);
  } else {
    mongoose.connect('mongodb://localhost/conduit');
    mongoose.set('debug', true);
  }

  require('./models/User');
  require('./models/Article');
  require('./models/Comment');
  require('./config/passport');

  // Create global app object
  const app = require('fastify')({
    logger: true,
  });
  await app.register(require('@fastify/formbody'));
  await app.register(require('./routes/legacy'), {isProduction});
  return app;
}

build()
  .then(app => app.listen({port: 3000}))
  .then((address) => {
    console.log('Listening on ' + address);
  }).catch(console.log);

The Express application configuration has been moved to a plugin defined in routes/legacy/index.js, which is registered in the Fastify app above. It looks like this:

const express = require('express'),
      session = require('express-session'),
      fp = require('fastify-plugin'),
      cors = require('cors'),
      errorhandler = require('errorhandler');

module.exports = fp(async function (fastify, opts) {
  await fastify.register(require('@fastify/express'), {
    // run express after `fastify-formbody` logic
    expressHook: 'preHandler',
  });

  fastify.use(cors());

  fastify.use(require('morgan')('dev'));

  /* ... (the same .use() calls that used to be in app.js) */

  const router = require('express').Router();
  router.use('/api', require('./api'));

  fastify.use(router);
});

For the most part it contains the same Express configuration that previously existed in app.js, except that it's contained in a Fastify plugin, and the configuration is applied to the fastify instance provided to the plugin. The plugin registers @fastify/express as a sub-plugin, which is what makes all that Express configuration work seamlessly.

There's one messy bit, which is the business with @fastify/formbody and the expressHook: 'preHandler' configuration line. There's a known incompatibility with the Express body-parser library, and the workaround is to immediately remove body-parser and substitute it with the @fastify/formbody plugin which works similarly. This was the only hangup like this that I encountered; otherwise the application runs just like before, using Fastify as the server.

Click here to view the full set of changes up to this point.

Step 2: Migrating a route to Fastify

Prerequisite #1: Implementing authentication decorators for Fastify routes

In the Express application, authentication is applied to a route by adding an auth.optional or auth.required middleware. Before migrating a route to Fastify, we'll need an equivalent way to add authentication requirements to a route.

In Fastify, this can be accomplished via Hooks and Decorators. For organizational purposes, this can be set up as a Fastify plugin, plugins/auth.js:

npm install --save @fastify/jwt

const fp = require("fastify-plugin");
const secret = require('../config').secret;

module.exports = fp(async function (fastify, opts) {
  await fastify.register(require("@fastify/jwt"), {
    // @fastify/jwt puts the token data on request.user by default; use
    // request.payload to match the existing application
    decoratorName: 'payload',
    secret,
    verify: {
      // The application uses "Token" in the Authorization header rather than
      // the more standard "Bearer", so add custom code to parse that
      extractToken: (request) => {
        const parts = request.headers.authorization.split(' ');
        if (parts.length !== 2) {
          throw new BadRequestError();
        }

        const [scheme, token] = parts;

        if (!/^Token$/i.test(scheme)) {
          throw new BadRequestError()
        }

        return token;
      },
    },
  });
  // Add a decorator so that routes can reference this via `fastify.authenticated`
  await fastify.decorate("authenticated", async function(request, reply) {
    try {
      await request.jwtVerify();
    } catch (err) {
      reply.send(err);
    }
  });
  // Add a decorator so that routes can reference this via `fastify.authenticatedOptional`
  await fastify.decorate("authenticatedOptional", async function(request, reply) {
    try {
      await request.jwtVerify();
    } catch {}
  });
});

To load this plugin, app.js will need to register it:

  await app.register(require('@fastify/formbody'));
  await app.register(require('./plugins/auth'));  // <-- Register the plugin
  await app.register(require('./routes/legacy'), {isProduction});
  return app;

With these decorators added, a route can now add a onRequest: [fastify.authenticated] or a onRequest: [fastify.authenticatedOptional] hook to handle authentication and to make user information available in the request.payload field.

Prerequisite #2: Setting up route plugins

Adding a route to a Fastify instance is no different from adding any other piece of functionality to a Fastify instance, which is what plugins are designed to help organize. As such, routes can be defined as a nested tree of plugins. The now-legacy articles.js module is as good a place as any to start converting routes, so to set up an equivalent module for the Fastify routes, we'll:

1. Import and register a routes plugin in app.js:

     await app.register(require('@fastify/formbody'));
     await app.register(require('./plugins/auth'));
     await app.register(require('./routes'));  // <-- Register the plugin
     await app.register(require('./routes/legacy'), {isProduction});
     return app;
   

2. In routes/index.js, import and register an api plugin:

   module.exports = async function (fastify, opts) {
     await fastify.register(require('./api'), {prefix: "/api"});
   };
   

   
   (Note how this plugin is registered with a "/api" prefix, so that all routes within will automatically have this prefixed to their route path)
3. In routes/api/index.js, import and register the articles plugin that will contain our migrated routes:

   module.exports = async function (fastify, opts) {
     await fastify.register(require('./articles'), {prefix: "/articles"});
   }
   

   
   (Note again the automatic "/articles" prefix which will be applied to those routes)
4. Finally, routes/api/articles.js can start as an empty plugin, ready for us to define routes within:

   module.exports = async function (fastify, opts) {
   }
   

Migrating a route

With the Fastify articles plugin ready to accept new routes, let's look at the first Express route, GET /api/articles, piece by piece:

routes/legacy/api/articles.js:

/* ... */

router.get('/', auth.optional, function(req, res, next) {
  /* ... */
});

Fastify does have a .get() shorthand like this, but I recommend getting used to using the full .route() declaration since I find it more readable when other options are defined:

routes/api/articles.js:

  /* ... */

  fastify.route({
    method: 'GET',
    url: '/',
    onRequest: [fastify.authenticatedOptional],
    handler: async function(request, reply) {
      /* ... */
    },
  });

  /* ... */

Note the onRequest hook which uses the authentication hook and decorator we defined previously in the auth plugin.

Next, the Express route checks for a number of query parameters and fetches some database objects:

routes/legacy/api/articles.js:

  /* ... */

  var query = {};
  var limit = 20;
  var offset = 0;

  if(typeof req.query.limit !== 'undefined'){
    limit = req.query.limit;
  }

  if(typeof req.query.offset !== 'undefined'){
    offset = req.query.offset;
  }

  if( typeof req.query.tag !== 'undefined' ){
    query.tagList = {"$in" : [req.query.tag]};
  }

  Promise.all([
    req.query.author ? User.findOne({username: req.query.author}) : null,
    req.query.favorited ? User.findOne({username: req.query.favorited}) : null
  ]).then(function(results){
    /* ... */
  });

  /* ... */

This is a place where beyond just replicating the Express route's behaviour, the Fastify route can easily improve upon it by adding more thorough schema validation for these parameters:

routes/api/articles.js:

  fastify.route({
    method: 'GET',
    url: '/',
    onRequest: [fastify.authenticatedOptional],
    schema: {   // <--- `schema` field added
      query: {  // <--- query param validation added (also works for body, params, headers)
        type: "object",
        properties: {
          limit: {type: "number", minimum: 1},
          offset: {type: "number", minimum: 0},
          tag: {type: "string", minLength: 1},
          author: {type: "string", minLength: 1},
          favorited: {type: "string", minLength: 1},
        },
        required: [],
      },
    },
    handler: async function(request, reply) {
      /* ... */
    },
  });

  /* ... */

Otherwise, the body of this route can be migrated largely unchanged, except with the req variable renamed to request (a purely stylistic choice but one which seems to be the convention in Fastify). The only other difference is where the response is returned; in Express res.json() is called, but Fastify is JSON-enabled by default and so a JavaScript object can just be returned directly.

The rest of the routes in this module can be migrated similarly, with one exception: some rely on an Express utility called .param(), for which Fastify doesn't have a direct equivalent. This utility is used in order to automatically fetch an Article object from the database for any route which has a :article query parameter, and make it available to the route handler as req.article:

routes/legacy/api/articles.js:

router.param('article', function(req, res, next, slug) {
  Article.findOne({ slug: slug})
    .populate('author')
    .then(function (article) {
      if (!article) { return res.sendStatus(404); }

      req.article = article;

      return next();
    }).catch(next);
});

It's possible to do the same in Fastify using the more generic "hooks" system:

fastify.addHook('onRequest', async (request, reply) => {
  if ('article' in request.params) {
    request.article = await Article
      .findOne({ slug: request.params.article})
      .populate('author');

    if (!request.article) {
        reply.code(401).send();
        return reply;
    }
  }
});

Note that for style reasons however, I opted to remove hooks like these and perform these database requests within the route handlers themselves; this makes the code more direct and easy to reason about, and also allows us to query for the Article in parallel with other database queries.

Click here to view the rest of the partial conversion to Fastify. This process can continue piece by piece until the last of the Express routes are migrated.

Step 3: Removing Express and related plugins

Once the migration is complete, the registration of the routes/legacy plugin in app.js can be removed, the modules in the legacy/ directory can be deleted, and all Express-related plugins can be removed from package.json (including @fastify/express). What's left is an application that has been fully migrated to Fastify, while remaining fully functional at every step along the way.

Click here to view the fully-migrated codebase.

Bonus: Adding integration tests

This project currently relies on end-to-end tests that execute against a full running instance of the application. It's valuable to have some tests like these, but most of them would be better off as integration tests, such that test cases are independent of one another and they have the ability to set up data before they run.

To use Fastify's built-in testing utilities to this end, a few minor things in this project need to be refactored:

1. The Fastify instance creation needs to be sparated from the place where the it's used to start up the server itself; this is because the test utilities work on the Fastify app itself without starting up a real Webserver.
   app.js is currently doing both:

   async function build() {
     /* ... */
   
     const app = require('fastify')({logger: true});
   
     /* ... */
   
     return app;
   }
   
   build()
     .then(app => app.listen({port: 3000}))
     .then((address) => {
       console.log('Listening on ' + address);
     }).catch(console.log);
   

   
   A solution is to have app.js export its build() function, and then create a separate server.js file to move the build() invocation into (updating the package.json scripts so that they invoke node ./server.js instead of node ./app.js). The behaviour is the same, but now the Fastify app can be imported in tests.
2. In in-memory MongoDB database needs to be configured, so that the integration tests can create one. This can be done easily with the mongodb-memory-server package:

   const {MongoMemoryServer} = require('mongodb-memory-server');
   
   mongod = await MongoMemoryServer.create();
   process.env.MONGODB_URI = mongod.getUri();
   
   /* Do testing... */
   
   mongod.stop();
   

In this case, Node-Tap was used as the test runner, but any test runner will do. An annotated example of one of these tests is shown below, and the full set of changes (including additional test case examples) can be found here.

tests/users.test.js:

const {MongoMemoryServer} = require('mongodb-memory-server');
const {test} = require('tap');

const build = require('../app')
const {deleteAll} = require('../models');
const User = require('../models/User');

// Add a test case for /users endpoints. In Node-Tap, test cases can be
// nested arbitrarily.
test('/users', async t => {
  let mongod, app;

  t.before(async () => {
    // Create a new in-memory MongoDB instance before the tests run,
    // and build the Fastify application
    mongod = await MongoMemoryServer.create();
    process.env.NODE_ENV = 'test';
    process.env.MONGODB_URI = mongod.getUri();
    app = await build();
  });
  t.beforeEach(async t => {
    // Before each test, call a utility which wipes all content from the
    // database, so that test cases are fully independent.
    await deleteAll();
  });
  t.teardown(async () => {
    // Shut down the app and stop the database when tests complete.
    await app.close();
    await mongod.stop();
  });

  // Define a test for POST /users
  await t.test('POST /users', async t => {
    // Here is the Fastify inject() utility that's being leveraged for testing:
    // A POST request is being simulated without a Webserver active, but it
    // otherwise functions the same way that a true POST request would. The
    // return value is a Promise that resolves to the Response object.
    const response = await app.inject({
      method: 'POST',
      url: '/api/users',
      headers: {
        'Content-Type': 'application/json',
        'X-Requested-With': '',
      },
      payload: {
        user: {
          email: 'john@jacob.com',
          password: 'johnnyjacob',
          username: 'johnjacob',
        },
      },
    })

    // With the Response in hand, whatever necessary test assertions can
    // be made.
    t.equal(response.statusCode, 200, 'returns a status code of 200');
    const body = response.json();
    t.hasProp(body, 'user', 'Response has "user" property"');
    t.hasProp(body.user, 'email', 'User has "email" property"');
    t.hasProp(body.user, 'username', 'User has "username" property"');
    t.hasProp(body.user, 'token', 'User has "token" property"');
  });
});

Conclusions

Completing this migration took me several hours, some of which was ramping up on @fastify/express and making changes to the app structure, but a lot of which was working through each migrated API endpoint. Consequently, I think the time it takes to complete a migration is going to scale linearly with the size of a project.

While I do think Fastify is a better choice than Express as a lightweight Web framework for Node.js, the cost of migrating relative to the benefits makes it such that I would only recommend doing it in specific scenarios, for example for small or early-stage projects, or when a major refactor is already going to take place. That said, I found the process itself to be smooth and reliable, and with good pre-existing test coverage I feel it's about as safe to do as any major refactor can be.

I hope this example is helpful for anyone considering making the switch to Fastify; the full source code is available on GitHub here.