Experiments enable you to deliver feature configurations to audiences, allowing you to test the effectiveness and success of a particular new feature, design, configuration, etc. on a given audience.
These differ from feature flags in that experiments are only for a fixed duration, are scoped to a particular audience, and are measurement-oriented - testing your assumptions about what change a new feature will drive in a given metric.
Use experiments in designing your game to work iteratively and make data-driven decisions. Successful experiments can then be converted to permanent feature flags for your players.
Creating experiments #
Click the + New Experiment button to open the Create Experiment modal, then enter the following details:
- Name: Enter a unique name for this experiment
- Description: Provide a description of the experiment
- Split Key: Property to use for splitting the audience into variants. Defaults to Identity ID (random split) if not specified.
- Audiences: The audiences to participate in the experiment
- Hold Outs: The audiences, if any, to exclude from the experiment
- Goal Metrics: The desired metric to measure based on the experiment
- Monitor Metrics: Additional metrics to monitor during this experiment
Experiment details #
Select an experiment from the list to view its details page. This page contains three tabs: Details, Variants, and Phases.
View and edit the experiment’s details, including its name, description, audiences, and metrics.
View, edit, and create new variants for this experiment.
Create new experiment phases and view the experiment’s current and/or previous phases.
Phases are used to control the variants that are active during a given time period for a given audience split. At the conclusion of the phase, you can view the changes in the goal and monitor metrics for the experiment.