FullStack Labs

Please Upgrade Your Browser.

Unfortunately, Internet Explorer is an outdated browser and we do not currently support it. To have the best browsing experience, please upgrade to Microsoft Edge, Google Chrome or Safari.
Upgrade

Parallelizing Your Tests in CircleCI and Collecting Coverage for CodeClimate

Written by 
Aaron Dewey
,
Senior Software Engineer
Parallelizing Your Tests in CircleCI and Collecting Coverage for CodeClimate
blog post background
Why Coinbase’s Mobile App Is 100 Percent Built with React Native
Evaluation of New Ruby on Rails Features
Dealing with Unexpected and Difficult Client Changes

This article will show you how to use workspaces and parallelism to reduce your test times in CircleCI, and will also explain how to upload your parallelized coverage results to CodeClimate.

Table of contents

When you are trying to hit a minimum of 90% test coverage, as we do here at FullStack Labs, you're bound to run into some lengthy test runs in your CircleCI Jobs. Optimizing your tests with parallelism and workspaces is achievable in a few key steps. First, build your project and persist your mounted directory, then run your tests using parallel jobs, and finally persist any artifacts you may want from the output.

Build & Test

To build and persist your application to the workspace, you will use persist_to_workspace. Below is an example that builds a typical Node.js application.

	
build:
    steps:
      - run:
          name: Install Node Modules
          command: npm install
      - persist_to_workspace:
          root: ~/repo
          paths:
            - ./*
	

NOTE: In this example, we persist all of the data in the repo directory. This is our working_directory. You can choose to persist individual files using the paths configuration.

In your test job, you need to set the parallelism flag to the max number of parallel jobs you would like to have. Then, attach your workspace using attach_workspace and run your tests.

	
test:
    parallelism: 4
    steps:
      - attach_workspace:
          at: ~/repo
      - run:
          name: Run Tests
          command: npm run test:ci
	

Next, configure your tests by figuring out your best use case when splitting files. In one of our projects, we chose to create our own partitions using CIRCLE_NODE_INDEX and CIRCLE_NODE_TOTAL as FusionJS and our custom configuration did not play well with the slice of files.

After you’ve pushed your changes up you can see the separate jobs passing and failing as needed showing you their statuses and output.

CircleCI job output showing three failed jobs and one success

As a bonus, you can split up your testing and lint jobs to have them fail separately without having to install and build any prerequisites since you can attach_workspace.

	
lint:
    steps:
      - attach_workspace:
          at: ~/repo
      - run:
          name: Run Lint
          command: npm run lint
	

Collecting Coverage

Now that your tests are parallelized you will need to collect coverage using a similar method you selected for splitting your files. To do this, we need to install the cc-test-reporter and configure the builds. Adding the Install Test Reporter and Report Before Build steps to our build job. See Configuring Test Coverage on CodeClimate’s website for configuration and links to your specific build of the test reporter.

	
build:
    steps:
      - run:
          name: Install Test Reporter
          command: |
            wget -O ./cc-test-reporter https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64
            chmod +x ./cc-test-reporter
      - run:
          name: Report Before Build
          command: ./cc-test-reporter before-build
      - run:
          name: Install Node Modules
          command: npm install
      - persist_to_workspace:
          root: ~/repo
          paths:
            - ./*
	

Then we can add a Format Test Coverage step to our test job, including another persist_to_workspace for the tmp directory. We use the cc-test-reporter to format-coverage of our lcov.info output for this test partition.

NOTE: We are using the $CIRCLE_NODE_INDEX to define our coverage partitions. This may vary depending on how you decide to split your files as described above.

	
 test:
    parallelism: 4
    steps:
      - attach_workspace:
          at: ~/repo
      - run:
          name: Run Tests
          command: npm run test:ci
      - run:
          name: Format Test Coverage
          command: ./cc-test-reporter format-coverage -t lcov -o tmp/codeclimate.$CIRCLE_NODE_INDEX.json coverage/lcov.info
      - persist_to_workspace:
          root: ~/repo
          paths:
            - tmp
	

Next, we can add a reporting job that will sum our coverage and upload it to CodeClimate. Again, using the cc-test-reporter to sum-coverage of all of our test runs, then uploading them by defining the input JSON.

	
report:
    steps:
      - attach_workspace:
          at: ~/repo
      - run:
          name: Upload Coverage To Code Climate
          command: |
            ./cc-test-reporter sum-coverage tmp/codeclimate.*.json -o tmp/codeclimate.total.json
            ./cc-test-reporter upload-coverage -i tmp/codeclimate.total.json
	

Finally, we can see all of the individual jobs on a pull request and the CodeClimate coverage results.

Github showing build, lint, test, and report jobs passing along with combined code coverage

Here at FullStack Labs, we produce high-quality solutions for our clients that they love! If you like what I posted above and would like to join us, please visit our Careers page. If you would like to reach us for any other reason, please contact us. We look forward to hearing from you!

Aaron Dewey
Written by
Aaron Dewey
Aaron Dewey

As a Senior Software Engineer at FullStack Labs, I am focused on delivering custom software solutions using technologies like React.js and Node.js. I have 5 years of experience leading teams and delivering performant code to frontend and backend code bases. As an avid learner and reverse engineer, I am eager to grow and take on any challenges that are presented to me. In my free time I code and watch horror movies with my wife.

FullStack Labs Icon

Let's Talk!

We’d love to learn more about your project. Contact us below for a free consultation with our CEO.
Projects start at $50,000.

company name
name
email
phone
Type of project
How did you hear about us?
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.