Experiments

Experiments let you split one channel target's traffic between two or more agent variants. Use one variant as the control and compare how much traffic each variant handles under the same target.

The Experiments API reports session counts for each variant. It does not select a winner, calculate statistical significance, or measure a business outcome. Combine the session counts with the outcome data and evaluation method appropriate for your use case.

Before you begin

You need:

• a Syllable API key for your organization;
• one channel target that receives live traffic; and
• at least two agents, one for each variant.

Built-in agent-test targets are not supported because they do not represent a stable population of live traffic. Use another channel target from your organization.

Create an API token in the Syllable Console and keep it in an environment variable:

export SYLLABLE_API_KEY="your-api-key"

The examples below use numeric target and agent IDs. Replace them with IDs from your organization. The role associated with your API key needs the permission for each operation:

Creating and updating an experiment always requires the complete variants set with at least two variants. SDK v1.0.65 incorrectly renders this field as optional; omitting it is rejected by the API.

Experiment concepts

• Target: The channel target whose traffic is split. You choose it when creating the experiment and cannot change it later.
• Variant: A named agent configuration that receives a percentage of the target's traffic. Each variant must use a different agent.
• Control: The reference variant against which you compare the others. Exactly one variant must be the control.
• Weight: A positive whole-number percentage of traffic assigned to a variant. Every weight must be greater than zero, and all variant weights must total 100.

For a two-variant experiment, two weights of 50 create an even split. A 90 and 10 split is also valid when you want to limit traffic to a new variant.

Only one experiment can run on a target at a time.

Lifecycle

Starting an experiment validates its target and variants. Review the warnings returned by the start operation. For example, an agent that does not pin a prompt version may generate a warning because its prompt could change while the experiment is running. A warning does not necessarily prevent the experiment from starting, but it can weaken how confidently you interpret the comparison.

Run an experiment

1. Create the draft

This example creates an even split between a control agent and a second agent:

curl --request POST \
  --url "https://api.syllable.cloud/api/v1/experiments/" \
  --header "Syllable-API-Key: ${SYLLABLE_API_KEY}" \
  --header "content-type: application/json" \
  --data '{
    "name": "Shorter greeting",
    "description": "Test whether a shorter greeting improves completion",
    "target_id": 1,
    "variants": [
      {
        "name": "Control",
        "weight": 50,
        "is_control": true,
        "agent_id": 1
      },
      {
        "name": "Short greeting",
        "weight": 50,
        "is_control": false,
        "agent_id": 2
      }
    ]
  }'

Save the id from the response:

export EXPERIMENT_ID="the-returned-experiment-id"

You can update the draft before starting it. An update replaces the complete variant set, and it cannot change the target.

2. Start the experiment

curl --request POST \
  --url "https://api.syllable.cloud/api/v1/experiments/${EXPERIMENT_ID}/start" \
  --header "Syllable-API-Key: ${SYLLABLE_API_KEY}"

Check the response for start-time warnings before sending production traffic through the experiment.

3. Read the session counts

curl --request GET \
  --url "https://api.syllable.cloud/api/v1/experiments/${EXPERIMENT_ID}/results" \
  --header "Syllable-API-Key: ${SYLLABLE_API_KEY}"

Each result contains a variant ID and the number of sessions handled by that variant. Different counts can be expected when weights are unequal, traffic is still accumulating, or routing conditions affect which sessions reach the target.

4. Stop the experiment

curl --request POST \
  --url "https://api.syllable.cloud/api/v1/experiments/${EXPERIMENT_ID}/stop" \
  --header "Syllable-API-Key: ${SYLLABLE_API_KEY}"

Stop a running experiment before attempting to delete it.

API operations

• List experiments
• Create an experiment
• Get an experiment
• Update a draft experiment
• Delete a non-running experiment
• Get experiment results
• Start an experiment
• Stop an experiment

For request and response schemas, TypeScript SDK examples, error responses, and authentication details, open the corresponding API-reference page.