How to write automation tests for LaunchDarkly Feature Flags with Cypress.io

Feature toggles (also known as feature flags) are invaluable tools when developing and deploying new features. They allow developers to release new functionality to production while keeping it hidden from end users. This approach replaces multiple code branches with a single trunk branch, simplifying version management even when features are incomplete or still in progress.

However, testing a system with multiple active feature flags can introduce significant challenges. With dozens of flags, each serving different variations, there is a substantial level of indeterminism. This is problematic for automation testing, where consistency and reliability are essential. If tests behave unpredictably, they can become flaky and unreliable.

To ensure robust and stable automation tests in a system using LaunchDarkly feature flags, you can follow several strategies:

1. Test Features in Isolation: Ensure that each test runs independently of others to avoid interference. This helps maintain predictable results across multiple test executions.
2. Create Targeted Test Data: Use helpers to create test data (such as users) that are tied to specific flag variations. This ensures that you know exactly which variation is being served during each test, helping you predict the expected outcomes. Remember to remove any created data before the next test run to ensure a clean testing environment. Consider whether setting the appropriate flag status should be added to the Cypress ‘before’ or ‘after’ hook.
3. Organize Tests by Flag: Tag tests that depend on temporary feature flags. This makes it easier to disable or remove tests associated with deprecated flags when they are archived.
4. Ongoing Maintenance: Regularly monitor which flags are being deprecated or removed and adjust your test suite accordingly. This prevents your tests from becoming outdated.
5. Test Fallback Values: When LaunchDarkly is unavailable (for example, due to network issues), ensure that all flags in your application fall back to predefined values. Testing these fallback behaviors is critical to avoid unexpected behavior in production as all features will use the fallback value at the same time.

Although testing systems with feature flags might seem complex, using tools like Cypress.io makes the process relatively straightforward. Let’s walk through an example of testing a simple React application with LaunchDarkly feature flags.

In this example, we’ll use a simple React application that displays a greeting based on the button clicked. For the purposes of this article, to simplify testing by skipping login and switching users, we implemented a scenario in which the value from the button is passed as a context attribute to LaunchDarkly. After the feature flag is evaluated, the appropriate text variation is served based on the flag’s configuration.

First, let’s take a look at the feature flag setup.

In LaunchDarkly , a feature flag, named Display Text on Click, has been created with three string variations:

1. Hello World
2. Welcome to gravity9
3. Nice to meet you!

The default value when the flag is “ON” is “Welcome to gravity9”, and when the flag is “OFF”, it defaults to “Nice to meet you!”.

The configuration also includes a Context with a buttonDetails context kind. This context features a buttonNumber attribute, which is passed from the application to specify which text variation the feature flag should serve.

Three segments are defined, each corresponding to a specific button number.

These segments target the appropriate button based on the buttonNumber value provided in the context.

Given the setup, we can define the following test cases:

1. When flag in ON, the default greeting should be “Welcome to gravity9”
2. When flag is ON and the “Greeting 1” button is clicked, the greeting should be “Hello World”
3. When flag is ON and the “Greeting 2” button is clicked, the greeting should be “Welcome to gravity9”
4. When flag is ON and the “Greeting 3” button is clicked, the greeting should be “Nice to meet you!”
5. When flag is OFF, the greeting should be “Nice to meet you!” whether any button is clicked or not
6. When LaunchDarkly is not reachable, the default greeting should be “Flag unreachable”. When button is clicked, the value should be “Fallback value for Greeting” {number of a button clicked}. These are fallbacks defined in the application’s code, and cannot be accessed via LaunchDarkly.

Now that we’ve defined our test cases, let’s explore how to automate them with Cypress.io.

Step 1: Enable/Disable Feature Flag with API Calls

LaunchDarkly offers an API to enable and disable feature flag targeting. You can trigger this by calling a unique URL for each action (turning the flag ON or OFF).

To do this, in the LaunchDarkly Flags section, click the three-dot menu and select “Configuration in environment.” From there, you can add a Generic Trigger to enable/disable flag targeting.

In Cypress, you can use the request command to send HTTP requests to these URLs. Below is a helper method that calls the API to control the flag state:

Step 2: Writing Test Scripts

Test Cases 1–4: Flag ON Behavior

In the first test, we will validate the default greeting, then simulate button clicks and verify the displayed greeting:

Test Case 5: Flag OFF Behavior

Now let’s ensure that the greeting defaults to “Nice to meet you!” when the flag is OFF:

Test Case 6: Flag Unreachable (Fallback Scenario)

To simulate the scenario where LaunchDarkly is unreachable, we can either modify the client-side ID or disable an internal API endpoint temporarily. Once that is done, the greeting should fallback to “Flag unreachable”:

Given it’s already been done, the test script will be:

Testing applications with feature flags requires careful planning to ensure that tests are reliable and consistent. By testing flags in isolation, creating targeted test data, organizing tests, and checking for fallback values, you can maintain robust automation tests even in the face of dynamic flag configurations. With Cypress.io, you can efficiently automate these tests and ensure your application’s behavior is consistent across various flag states.

By following the strategies outlined in this article, you can ensure that your feature flags are well-tested and don’t introduce unreliable or flaky tests in your continuous integration pipeline.