Archives Search Feed About

Speed up NestJS test executions with Jest

October 1, 2022 ☼ NestJS ☼ Jest ☼ Node.js

No one likes (slow) tests… Right?

In this blog post I will show you how you can improve your tests speed execution and give you some pointers where to look if you want to dive deeper and squeeze every ounce of performance.

The following advices are being applied to Unit Tests. They could also be applied to E2E or integration of course, but sometimes your project setup might require too specific adjustments that will be difficult to cover in this blog post.

Tech Stack

I’ve personally applied the following techniques to a Node.js Microservice written in NestJS.

Why my tests are slow?

There could be several culprits but the main offender, at least in my case, was ts-jest.

ts-jest is a TypeScript preprocessor with source map support for Jest that lets you use Jest to test projects written in TypeScript.

It supports all features of TypeScript including type-checking (this will be important later).

Another offender is the quantity of memory that your tests consume.

Let’s take a look at each on of them individually and see how we can improve the speed execution.

Show me your Jest config

File: jest-unit.js

/** @type {import('jest').Config} */

const config = {
  moduleFileExtensions: ["js", "json", "ts"],
  rootDir: "..",
  testEnvironment: "node",
  testRegex: ".unit.spec.ts$",
  transform: {
    "^.+\\.(t|j)s$": "ts-jest",
  },
  moduleNameMapper: {
    "^src/(.*)$": "<rootDir>/src/$1",
  },
  collectCoverageFrom: ["<rootDir>/src/**/*.(t|j)s"],
  coverageDirectory: "../coverage",
};

module.exports = config;

TS-JEST

It’s an amazing package that does a lot of the heavy lifting and your tests 99% of the times runs without issues.

I’m sure you are wondering why is it slow? The issue lies in transpiling Typescript files and type-checking.

Solution 1

The naive way to do it would be disable the type-checking and compile each file in isolation:

In jest-unit.js:

globals: {
    'ts-jest': {
      isolatedModules: true,
    },
  },

More info here.

Results

Before:

Test Suites: 16 passed, 16 total
Tests:       1 todo, 79 passed, 80 total
Snapshots:   0 total
Time:        11.184 s

After:

Test Suites: 16 passed, 16 total
Tests:       1 todo, 79 passed, 80 total
Snapshots:   0 total
Time:        4.851 s

I don’t think disabling type-checking in our test files is a good idea honestly. But it works.

Solution 2

Use another compiler. Enters SWC.

SWC is an extensible Rust-based platform for the next generation of fast developer tools. SWC can be used for both compilation and bundling.

Add a .swrc file to the root of your project and install the ts-jest drop-in replacement: npm i -D @swc/core @swc/jest

File: .swrc:

{
  "sourceMaps": "inline",
  "module": {
    "type": "commonjs",
    "strict": true
  },
  "jsc": {
    "target": "es2020",
    "parser": {
      "syntax": "typescript",
      "decorators": true
    },
    "transform": {
      "legacyDecorator": true,
      "decoratorMetadata": true
    },
    "keepClassNames": true
  }
}

File: jest-unit.js

transform: {
    "^.+\\.(t|j)s?$": ["@swc/jest"],
  },

Results

Before:

Test Suites: 16 passed, 16 total
Tests:       1 todo, 79 passed, 80 total
Snapshots:   0 total
Time:        11.184 s

After:

Test Suites: 16 passed, 16 total
Tests:       1 todo, 79 passed, 80 total
Snapshots:   0 total
Time:        3.243 s

Wow. Problem solved right? Not quite.

SWC doesn’t play too well with NestJS and I couldn’t find a way to make it work with my test setup (mainly the integration tests). Unit Tests were running fine but basically I didn’t want to use two different setups for unit / integration.

Also, SWC [doesn’t do type-checking])(https://github.com/swc-project/swc/issues/571).

Type checking helps you develop with confidence, prevent mistakes, and enables smoother refactoring of large codebases (including tests).

Here you can find a repo with an example in case you are interested.

Solution 3

Tweak maxWorkers option until you find the right one for you.

I’d start with maxWorkers: '50%' and commit that to git so anyone can benefit from it.

I personally override that option locally for my setup and set it to 5.

You can find out the correct setup based on how many cores are available in your machine:

Run system_profiler SPHardwareDataType (Mac):

Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: MacBookPro15,1
      Processor Name: 6-Core Intel Core i7
      Processor Speed: 2,6 GHz
      Number of Processors: 1
      Total Number of Cores: 6
      L2 Cache: 768 KB
      L3 Cache: 6 MB

Results

Before:

Test Suites: 16 passed, 16 total
Tests:       1 todo, 79 passed, 80 total
Snapshots:   0 total
Time:        11.184 s

After:

Test Suites: 16 passed, 16 total
Tests:       1 todo, 79 passed, 80 total
Snapshots:   0 total
Time:        8.81 s

We’re using this solution in the end. Has the best trade offs and the complexity remains low.

Bonus

When running tests in a CI Environment using the --runInBand helps in an environment with limited resources. See this SO answer.

Do you have a memory Leak ?

File: package.json:

"test:unit": "jest --config ./test/jest-unit.js",

Run npm run test:unit -- --detectLeaks:

EXPERIMENTAL FEATURE!
    Your test suite is leaking memory. Please ensure all references are cleaned.

    There is a number of things that can leak memory:
      - Async operations that have not finished (e.g. fs.readFile).
      - Timers not properly mocked (e.g. setInterval, setTimeout).
      - Keeping references to the global scope.

Have a look at this well written article on the topic.

Close everything after each test

Make sure you that you always clean app any open connection or long running timers.

E.g.:

describe("YourFeatureHttpController", () => {
  let app: INestApplication;
  let psqlClient: PostgresClient;

  // Example of a DB Pool connection
  const pool: Knex = knex({
    client: "pg",
    connection: JSON.parse(process.env.SERVICES).postgresql[0].credentials.uri,
  });

  beforeAll(async () => {
    const moduleRef = await Test.createTestingModule({
      imports: [AppModule],
    }).compile();

    app = moduleRef.createNestApplication();
    app.useGlobalPipes(new ValidationPipe());
    app.get(AppModule).configure = function (consumer: MiddlewareConsumer) {
      consumer.apply((req, res, next) => {
        next();
      });
    };

    psqlClient = new PostgresClient(pool);

    await app.init();
  });

  afterAll(async () => {
    await app.close();
    await psqlClient.getPool().destroy();
  });

  // Your tests...
});

It’s not easy to solve leaking tests honestly. But as they say: “hard times make good developers”.

:)

That’s all folks!

If you have any suggestions, questions, corrections or if you want to add anything please DM or tweet me: @zanonnicola