Update: 28.04.2022.

I’ve replaced the WP base unit test case with the test case provided by Yoast’s WP Test Utils test case. This cuts down on some needles search/replacing.

Thanks to Juliette Reinders Folmer for mentioning this repo to me

If you’ve stalked me on Twitter, you might have noticed I talk about Pest every once in a while. Pest is a PHP testing framework built by Nuno Maduro. If you’ve ever worked with Jest, you’ll be very familiar with Pest.
It’s a superset of PHPUnit, the go-to PHP testing framework, with a more readable syntax. It’s a joy to work with.

I’ve been using Pest for some time now, writing unit tests for the backend part of Eightshift Development kit. Writing unit tests is super simple, as you are writing tests in isolation. So you can write them for WordPress.
Writing tests is not something that is widespread in the WordPress community. That’s not to say it doesn’t exist, it’s just that not that many developers bother to write one. And there are libraries that will allow you to write tests for WordPress: wp-browser is my go-to library if I need a complete test suite (unit, integration, and end-to-end tests). Another library that is a must-have for WordPress unit tests is Brain Monkey. Or you can just scaffold plugin or theme integration tests using WP-CLI’s scaffold command.

The options do exist, but writing test is not easy. The first shock when writing tests is realizing that your code is not testable. That’s understandable. Especially in WordPress, where many developers start out from zero, and they create their first plugin or theme just by copy-pasting examples on the internet. The problem is often that these examples are sometimes quite old. I’m not an exception to this rule. Most of my tutorials are from 2016 or 2017. Not only has my knowledge about WP and PHP increased in that time, but WordPress as a framework advanced a ton.
I was one of those people who never bothered to write automated tests. Until a client asked me for a 90% code coverage before a production release. It was a fun month ?

Most developers in the WordPress space don’t see value in writing tests. “I can see if a menu works on the frontend, I don’t need automated tests for that”. And I can understand the sentiment. Especially if you’re a freelancer and need to finish a feature for a client. A client (usually) doesn’t care about tests. They don’t care about clean code. I’ve written about that before.

The testing mindset

It’s not until you need to create some complex architecture in your project where you create custom post types, custom REST routes, and connect to third-party APIs, that you see the benefit of automated tests.

So, as I’ve mentioned when working on Eightshift Development kit I wanted to make sure that the things we developed and outsource to the world, work as intended. You can use the dev kit as a scaffold, utilizing tons of cool ‘modern’ PHP features like dependency injection container, autowiring, and useful template design patterns to extend. My priority was making sure that the features we put in that library work.
And I really didn’t want to be the one testing if anything broke after we add a new feature manually.

So I started setting up unit tests. I started adding tests back in 2019. The tests were minimal, and I used WP-CLI to scaffold tests. It worked, but it wasn’t a nice developer experience. I heard about Pest somewhere in 2020. And I was intrigued. But to me, Pest kinda looked more Laravel-oriented. After all, Nuno Maduro is known for his amazing Laravel work. I didn’t know how I could use Pest in my WordPress projects. Especially when it comes to setting up integration tests with WordPress.
It wasn’t until early 2021 that I said: you know what, let’s at least try to write some unit tests for the dev kit’s backend library. It was a perfect candidate for that: it mostly consists of testable backend logic, and it has WP-CLI commands that will output something even without WordPress running somewhere in the background. We can easily test for that. So I’ve set it up, and the response from my team at the time was amazing. The syntax is so nice and easy to read. The DX (developer experience) is amazing.

For instance, a part of tests for checking if the command for creating custom post types works looks like this:

<?php

namespace Tests\Unit\CustomPostType;

use EightshiftLibs\CustomPostType\PostTypeCli;

use EightshiftLibs\Exception\InvalidNouns;

use function Tests\deleteCliOutput;
use function Tests\setupMocks;
use function Tests\mock;

/**
 * Mock before tests.
 */
beforeEach(function () {
    setupMocks();
    $wpCliMock = mock('alias:WP_CLI');

    $wpCliMock
        ->shouldReceive('success')
        ->andReturnArg(0);

    $wpCliMock
        ->shouldReceive('error')
        ->andReturnArg(0);

    $this->cpt = new PostTypeCli('boilerplate');
});

/**
 * Cleanup after tests.
 */
afterEach(function () {
    $output = \dirname(__FILE__, 3) . '/cliOutput';

    deleteCliOutput($output);
});


test('Custom post type CLI command will correctly copy the Custom post type class with defaults', function () {
    $cpt = $this->cpt;
    $cpt([], $cpt->getDevelopArgs([]));

    // Check the output dir if the generated method is correctly generated.
    $generatedCPT = \file_get_contents(\dirname(__FILE__, 3) . '/cliOutput/src/CustomPostType/ProductPostType.php');

    $this->assertStringContainsString('class ProductPostType extends AbstractPostType', $generatedCPT);
    $this->assertStringContainsString('admin-settings', $generatedCPT);
    $this->assertStringNotContainsString('dashicons-analytics', $generatedCPT);
});


test('Custom post type CLI command will correctly copy the Custom post type class with set arguments', function () {
    $cpt = $this->cpt;
    $cpt([], [
        'label' => 'Book',
        'slug' => 'book',
        'rewrite_url' => 'book',
        'rest_endpoint_slug' => 'books',
        'capability' => 'post',
        'menu_position' => 50,
        'menu_icon' => 'dashicons-book',
    ]);

    // Check the output dir if the generated method is correctly generated.
    $generatedCPT = \file_get_contents(\dirname(__FILE__, 3) . '/cliOutput/src/CustomPostType/BookPostType.php');

    $this->assertStringContainsString('class BookPostType extends AbstractPostType', $generatedCPT);
    $this->assertStringContainsString('Book', $generatedCPT);
    $this->assertStringContainsString('book', $generatedCPT);
    $this->assertStringContainsString('book', $generatedCPT);
    $this->assertStringContainsString('books', $generatedCPT);
    $this->assertStringContainsString('post', $generatedCPT);
    $this->assertStringContainsString('50', $generatedCPT);
    $this->assertStringContainsString('dashicons-book', $generatedCPT);
    $this->assertStringNotContainsString('dashicons-analytics', $generatedCPT);
});


test('Custom post type CLI documentation is correct', function () {
    $cpt = $this->cpt;

    $documentation = $cpt->getDoc();

    $key = 'shortdesc';

    $this->assertIsArray($documentation);
    $this->assertArrayHasKey($key, $documentation);
    $this->assertArrayHasKey('synopsis', $documentation);
    $this->assertSame('Generates custom post type class file.', $documentation[$key]);
});Code language: PHP (php)

First, in the beforeEach and afterEach methods (which are just a wrapper around setUp and tearDown fixtures in PHPUnit), we set up some general mocks (some Brain Monkey WP specific replacements for core functions). We also use Mockery to mock the WP-CLI success and error methods (we’re not interested in them at the moment) and initialize our class that will create a custom post type using $this->cpt = new PostTypeCli('boilerplate');. Then, we can then check the directory where the new file should be created and if that file contains strings that we’ve replaced them with. And we can also check if the PostTypeCli class (a WP-CLI invokable command class) contains correct documentation. That way, if we change it, we know that something has changed and should be documented in the changelog because our tests will fail.

In the above example, we’ve used PHPUnit assertions, but you can write the tests using Pest’s own expectation API. Expectations are a wrapper around assertions that make writing tests expectation even more readable. We could write one of the tests like:

test('Custom post type CLI command will correctly copy the Custom post type class with defaults', function () {
    $cpt = $this->cpt;
    $cpt([], $cpt->getDevelopArgs([]));

    $dir = \dirname(__FILE__, 4) . '/cliOutput/src/CustomPostType/';
    $file = $dir . 'ProductPostType.php';

    expect($dir)->toBeReadableDirectory();

    // Check the output dir if the generated method is correctly generated.
    $generatedCPT = \file_get_contents($file);

    expect($generatedCPT)
        ->toContain('class ProductPostType extends AbstractPostType')
        ->toContain('admin-settings')
        ->not->toContain('dashicons-analytics');
});Code language: PHP (php)

I’ve added an extra expectation to check if the directory was created and is readable. The syntax is really great, no?

Integration test setup

The reason most developers love Laravel is its DX. From the great and in-depth documentation to its clean syntax, useful commands, and nice API. From the developer’s point of view, it’s a joy to work with (we won’t go into details about some architectural decisions that I’m not the biggest fan of). Nuno working with Laravel naturally meant that Pest is going to integrate with Laravel super seamlessly. As a mostly WordPress developer, I’m super jealous of Laravel for that.

For instance, this is what the phpunit.xml looks like for one of the Laravel projects I’m working on:

<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="./vendor/phpunit/phpunit/phpunit.xsd"
         bootstrap="vendor/autoload.php"
         colors="true"
>
    <testsuites>
        <testsuite name="Feature">
            <directory suffix="Test.php">tests/Feature</directory>
        </testsuite>
    </testsuites>
    <coverage processUncoveredFiles="true">
        <include>
            <directory suffix=".php">./app</directory>
        </include>
        <report>
            <clover outputFile="tests/coverage/clover.xml"/>
            <html outputDirectory="tests/coverage/html" lowUpperBound="50" highLowerBound="90"/>
        </report>
    </coverage>
    <php>
        <server name="APP_ENV" value="testing"/>
        <server name="BCRYPT_ROUNDS" value="4"/>
        <server name="CACHE_DRIVER" value="array"/>
        <server name="DB_CONNECTION" value="sqlite"/>
        <server name="DB_DATABASE" value=":memory:"/>
        <server name="MAIL_MAILER" value="array"/>
        <server name="QUEUE_CONNECTION" value="sync"/>
        <server name="SESSION_DRIVER" value="array"/>
        <server name="TELESCOPE_ENABLED" value="false"/>
    </php>
</phpunit>Code language: HTML, XML (xml)

You can set up integration tests, that use an in-memory database (meaning you don’t have to set up a separate database for tests) in no time.

One thing that I was missing in my libs was integration tests. The code I wrote for the libs works isolated. When I run the WP-CLI command I know that the file will be created. But is this file correct? If I use the REST API endpoint creation command that will create a new endpoint in WordPress, is this endpoint actually going to be registered in WordPress? Without integration tests, I cannot know that. Well, I can, by actually manually setting everything up, then seeing in Postman if I have the custom route. But that’s a time-consuming process. With automated integration tests, I’d have a WordPress instance, I’d run my code against it and I could see in my REST APIs response that a new endpoint was created.

Is it possible to set up an integration test suite in WordPress using Pest?

First steps first

When I started to work on this feature what I did was basically a reverse-engineering process of the original WP-CLI scaffolding suite. I have set it up on a mock plugin to see how exactly is the core WordPress handling setting up tests.

When you scaffold tests in your plugin, you’re going to end up having several new files in your plugin:

• phpunit.xml.dist – configuration file for PHPUnit.
• .travis.yml – configuration file for Travis CI. I don’t use Travis anymore, they had some changes in their service, and since all my code is on GitHub I use GitHub Actions.
• bin/install-wp-tests.sh – configures the WordPress test suite and a test database.
• tests/bootstrap.php – a file that makes the current plugin active when running the test suite.
• tests/test-sample.php – sample file containing the actual tests.
• .phpcs.xml.dist – collection of PHP_CodeSniffer rules.

Here, the ‘interesting’ part is the install-wp-tests.sh script. That script will download and install, in a temporary folder, WordPress core (/tmp/wordpress) and WordPress integration test files (/tmp/wordpress-tests-lib). The integration tests files contain the necessary data for running the tests: wp-tests-config.php file that defines the database connection information (and some other constants), but also the basic test case class.
By default Pest will (under the hood) bind the it and test methods to PHPUnit\Framework\TestCase. The way you can replace it is by using uses function (no pun intended ?).

So in order for our tests to run we need to:

1. Change the phpunit.xml (or .dist) file to include a new bootstrap file, and add the necessary server variables.
2. Create a new bootstrap file where we’ll set up tests for integration testing.
3. Replace underlying test cases for integration tests.
4. Download WordPress and tests folders
5. Setup database

Modifying files

In my case, I already had unit tests inside the tests folder. In hindsight, I should have done this a year earlier. It’s always a good idea to separate the test types into different folders. It’s easier to split running separate test types that way. My root folder looks like this:

phpunit.xml
tests/
    Integration/
    Unit/
    Helpers.php
    Pest.php
    bootstrap.phpCode language: HTML, XML (xml)

Pest.php is a file where we make modifications to our Pest test suites.

<?php

uses()->group('integration')->in('Integration');
uses()->group('unit')->in('Unit');

uses(\WP_UnitTestCase::class)->in('Integration');Code language: PHP (php)

Here we grouped different tests in folders to different groups and we bound the WP core WP_UnitTestCase class to any test running in integration test suites.

But how do we get this test case file? Luckily for us, WordPress uses git for core development (although it’s not a Composer package on Packagist). We can pull it into our project, but we don’t want it to live in the vendor folder. It’s a bit cleaner if we could move it to a separate folder. Say wp folder in our project’s root folder. We won’t commit it anyhow (add that folder to .gitignore). In your composer.json file you’ll need to add the following:

{
    ...
    "repositories": [
        {
            "type": "vcs",
            "url": "https://github.com/WordPress/wordpress-develop"
        }
    ],
    "require-dev": {
        ...
        "aaemnnosttv/wp-sqlite-db": "^1.2",
        "brain/monkey": "^2.6",
        "koodimonni/composer-dropin-installer": "^1.4",
        "mnsami/composer-custom-directory-installer": "^2.0",
        "pestphp/pest": "^1.2",
        "php-stubs/wordpress-stubs": "^5.9",
        "phpunit/phpunit": "^9.5",
        "roave/security-advisories": "dev-master",
        "wordpress/wordpress": "^5.9",
        "yoast/phpunit-polyfills": "^1.0",
        "yoast/wp-test-utils": "^1.0"
    },
    "extra": {
        "installer-paths": {
            "./wp/": ["wordpress/wordpress"]
        },
        "dropin-paths": {
            "wp/src/wp-content/": ["package:aaemnnosttv/wp-sqlite-db:src/db.php"]
        }
    },
    ...
}Code language: JSON / JSON with Comments (json)

A couple of things are happening here. First, we added wordpress-develop as a repository so that we can pull the develop version to our project (develop version has WP core and tests, so it’s a win-win for us). Then we used koodimonni/composer-dropin-installer to move the wordpress-develop to wp folder. Because I want to use an in-memory database, to avoid having to set up a separate MySQL database, and to be able to run the tests on the CI a bit more easily, we are using aaemnnosttv/wp-sqlite-db package. That package provides a drop-in for using SQLite with WordPress. By using mnsami/composer-custom-directory-installer we can move the drop-in file to the correct place (wp/src/wp-content/ folder).
Yoast’s unit test polyfill and test utils packages will help us make tests run on multiple WP versions and on different PHPUnit versions, as well as provide a base test class for the integration tests that have a lot of useful additions from the Brain Monkey package.

Let’s modify phpunit.xml file:

<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="./vendor/phpunit/phpunit/phpunit.xsd"
         bootstrap="tests/bootstrap.php"
         colors="true"
>
    <testsuites>
        <testsuite name="Unit Test Suite">
            <directory>./tests/Unit/</directory>
        </testsuite>
        <testsuite name="Integration Test Suite">
            <directory>./tests/Integration/</directory>
        </testsuite>
    </testsuites>
    <php>
        <env name="TEST" value="true" force="true" />
        <server name="DB_NAME" value="es_libs_tests"/>
        <server name="DB_USER" value="root"/>
        <server name="DB_PASSWORD" value=""/>
        <server name="DB_HOST" value="localhost"/>
    </php>
    <coverage processUncoveredFiles="true">
        <include>
            <directory suffix=".php">./src</directory>
        </include>
        <exclude>
            <file>./src/Build/BuildExample.php</file>
            <file>./src/Menu/BemMenuWalker.php</file>
            <file>./src/ConfigProject/ConfigProjectExample.php</file>
        </exclude>
        <report>
            <clover outputFile="tests/coverage/clover.xml"/>
            <html outputDirectory="tests/coverage/html" lowUpperBound="50" highLowerBound="90"/>
        </report>
    </coverage>
</phpunit>Code language: HTML, XML (xml)

We’ve split the test suites based on the folders, and used a different bootstrap.php file for tests. bootstrap.php file will get loaded before tests are executed. It’s usually used for autoloading certain files, clearing and setting up a database before the test run, etc.

Bootstrapping the tests

Before diving into the bootstrap.php file, let’s go back to the Pest.php file. Because we would need to manually load the WP_UnitTestCase if we’d like to use it (and would have to meddle with the WP’s native bootstrap file), with the help of the WP Test Utils package we can just use Yoast\WPTestUtils\BrainMonkey\TestCase as a base test case for integration tests instead:

<?php

use Yoast\WPTestUtils\BrainMonkey\TestCase;

uses()->group('integration')->in('Integration');
uses()->group('unit')->in('Unit');

uses(TestCase::class)->in('Integration');Code language: PHP (php)

Now that we have this setup, we can go and look at our bootstrap.php file:

<?php

// Autoload everything for unit tests.
require_once dirname(__FILE__, 2) . '/vendor/autoload.php';

/**
 * Include core bootstrap for an integration test suite
 *
 * This will only work if you run the tests from the command line.
 * Running the tests from IDE such as PhpStorm will require you to
 * add additional argument to the test run command if you want to run
 * integration tests.
 */
if (isset($GLOBALS['argv']) && isset($GLOBALS['argv'][1]) && strpos($GLOBALS['argv'][1], 'integration') !== false) {
    if (!file_exists(dirname(__FILE__, 2) . '/wp/tests/phpunit/wp-tests-config.php')) {
        // We need to set up core config details and test details
        copy(dirname(__FILE__, 2) . '/wp/wp-tests-config-sample.php', dirname(__FILE__, 2) . '/wp/tests/phpunit/wp-tests-config.php');

        // Change certain constants from the test's config file.
        $testConfigPath = dirname(__FILE__, 2) . '/wp/tests/phpunit/wp-tests-config.php';
        $testConfigContents = file_get_contents($testConfigPath);

        $testConfigContents = str_replace("dirname( __FILE__ ) . '/src/'", "dirname(__FILE__, 3) . '/src/'", $testConfigContents);
        $testConfigContents = str_replace("youremptytestdbnamehere", $_SERVER['DB_NAME'], $testConfigContents);
        $testConfigContents = str_replace("yourusernamehere", $_SERVER['DB_USER'], $testConfigContents);
        $testConfigContents = str_replace("yourpasswordhere", $_SERVER['DB_PASSWORD'], $testConfigContents);
        $testConfigContents = str_replace("localhost", $_SERVER['DB_HOST'], $testConfigContents);

        file_put_contents($testConfigPath, $testConfigContents);
    }

    // Give access to tests_add_filter() function.
    require_once dirname(__FILE__, 2) . '/wp/tests/phpunit/includes/functions.php';

    /**
     * Register mock theme.
     */
    function _register_theme()
    {
        $themeDir = dirname(__FILE__, 2);
        $currentTheme = basename($themeDir);
        $themeToot = dirname($themeDir);

        add_filter('theme_root', function () use ($themeToot) {
            return $themeToot;
        });

        register_theme_directory($themeToot);

        add_filter('pre_option_template', function () use ($currentTheme) {
            return $currentTheme;
        });

        add_filter('pre_option_stylesheet', function () use ($currentTheme) {
            return $currentTheme;
        });
    }

    tests_add_filter( 'muplugins_loaded', '_register_theme' );

    require_once dirname(__FILE__, 2) . '/wp/tests/phpunit/includes/bootstrap.php';
}Code language: PHP (php)

Let’s break this down.

Because this file will be loaded for the unit and integration test suite we need to differentiate when to load which part. We don’t need to load WP core if we are running unit tests. We need the basic Pest files, which are all nicely autoloaded by Composer which is why we need the require_once dirname(__FILE__, 2) . '/vendor/autoload.php'; file.

We’re going to run the tests from a command line using vendor/bin/pest --group=integration command. This is the tricky part, especially if you want to run tests from an IDE, like PhpStorm. The only way (to my knowledge) to find out if we’re running unit or integration tests in bootstrap.php file is by checking the argv key from the $GLOBALS array (command-line arguments). That is why we have conditional checks. Now, when you run the tests from IDE (I’m using PhpStorm) the command is different, so the integration argument will be missing. That is unless you change the execution script a bit. But we can live with that.

The bootstrap file is largely based on the install-wp-tests.sh script and the WP-CLI generated tests/bootstrap.php file.

In the initial run, we need to set up wp-tests-config.php file. Luckily for us, the core comes with a template that we can use and just search-replace parts that we need. In our phpunit.xml we provided the server variables for the database, so we can use that to search-replace the database credentials. After that, we load the test functions and ‘register’ our mock theme. This can be modified for plugins from WP-CLI’s scaffold plugin command.

To verify if this is working for you just type die(\wp_get_theme()); after the require_once $basicBootstrapPath; line, and run the tests using vendor/bin/pest --group=integration command. In my case, I got eightshift-libs because I’m running the tests from the library.

Final test

Despite the subtitle, it’s not the final test, actually, it’s the first, but it’s the final test to see if everything is working as intended. In your Integration folder, create a new test case. We’ll be running a test to see if WordPress REST API is working properly. When you ping your WordPress site /wp-json/wp/v2 (with pretty permalinks turned on), you get a list of all your registered routes. The test if REST API works looks like this:

<?php

namespace Tests\Integration;

beforeEach(function () {
    parent::setUp();

    // Set up a REST server instance.
    global $wp_rest_server;

    $this->server = $wp_rest_server = new \WP_REST_Server();
    do_action('rest_api_init');
});

/**
 * Cleanup after tests.
 */
afterEach(function () {
    global $wp_rest_server;
    $wp_rest_server = null;

    parent::tearDown();
});

test('Rest API endpoints work', function () {
    $routes = $this->server->get_routes();

    expect($routes)
        ->toBeArray()
        ->toHaveKey('/wp/v2/posts');
});Code language: PHP (php)

First, we’ve set up the REST server instance, then we’ve initiated REST API with do_action('rest_api_init');. In the cleanup, we want to clean globals, so that they don’t pollute other tests. Our test is super simple: get all the routes, and see if one of the default keys exists. If it does, that means that the WP core is working.

The next steps for testing would be to try to add your custom route and test if it’s registered in the same way as here. Or any other functionality for that matter.

Caveats and conclusions

In this article, I’ve shown you how I managed to get Pest running WordPress integration tests. This is just a beginning for me, and I still need to test this out myself, but I wanted to share my ideas, and workflow of how I added it. Pest is a great tool and with this, my aim is to get more people on board with automated testing in WordPress.

While this setup works (it works on my computer ?), there are some caveats that you need to be aware of when figuring out whether you’d like to use Pest in your setup.
As of writing this article, there is an ongoing bug inside PHPUnit that prevents Pest from adding a feature that would allow you to run your tests in a separate process (in isolation).

Why would you need that you may ask? Well, unfortunately, WordPress is notorious for relying on and (ab)using globals. That means that if you change the global state in one of your tests, that state will carry on in other test cases. We can clean up after ourselves in tests (and should) but sometimes that’s not enough. You may need to have some constants defined so that a part of your code works, but once you do that, you cannot redefine them later on. This is where running tests in isolated processes come in handy. If you have a code that looks something like this:

if(defined('SOME_CONSTANT') && SOME_CONSTANT === 'foo') {
    // execute some logic if the constant is 'foo'.
} else {
    // execute some logic if the constant is not 'foo'.
}Code language: PHP (php)

Running tests with that constant will only cover one branch of your code (true statement or false statement).

This is just one of the examples of the usage of running tests in a separate process. The issue is opened about this, and I’ve made a PR that (hopefully) would address this, so if this issue is fixed we may have this feature in Pest in the future (if not for the lack of trying).

Another thing to note is that future versions of Pest will only work with PHP 8.1+. WordPress, for better or worse, still supports PHP 5.6 and is unclear when they’ll up the minimum version to PHP 7. There is ongoing massive work to make WordPress compatible with PHP 8+. That is a difficult process that is taking time because of the huge impact it can have on 40+ percent of the web. So if you are working on lower PHP versions, or need to support them, Pest is not aimed at you. In that case, I’d recommend using wp-browser.

Still, despite all the obstacles, I really hope this article will inspire people to use automated tests more in their projects.

Thank you all for reading, and I’ll see you all around ??.