experiments

A/B testing and experimentation platform with variant allocation, statistical analysis, goal tracking, and automated winner selection for data-driven decisions.

Overview

The experiments primitive provides comprehensive experimentation capabilities including A/B/n testing, multivariate experiments, statistical significance calculation, goal conversion tracking, and automatic experiment termination.

Parent Primitive: analytics - Analytics and metrics platform

SDK Object Mapping

This primitive maps to the decide SDK object - one of the 8 core platform objects:

import { decide, send, experiments } from 'sdk.do'

// Create experiment
const experiment = await experiments.create({
  name: 'Checkout Button Color',
  variants: [
    { key: 'control', name: 'Blue Button', allocation: 50 },
    { key: 'variant-a', name: 'Green Button', allocation: 50 },
  ],
  goals: [{ key: 'purchase', name: 'Completed Purchase', type: 'conversion' }],
})

// Get variant assignment via decide
const variant = await decide.get('checkout-button-test', {
  userId: 'user-123',
})

// Track conversion
await decide.track({
  experiment: 'checkout-button-test',
  userId: 'user-123',
  metric: 'purchase',
  value: 1,
})

// Send experiment event
await send('experiment.conversion', {
  experimentId: experiment.id,
  userId: 'user-123',
  variant: variant.key,
  goal: 'purchase',
})

Quick Example

import { experiments } from 'sdk.do'

// Create A/B test
const experiment = await experiments.create({
  name: 'Checkout Button Color',
  description: 'Test blue vs green checkout button',
  variants: [
    { key: 'control', name: 'Blue Button', allocation: 50 },
    { key: 'variant-a', name: 'Green Button', allocation: 50 },
  ],
  goals: [
    { key: 'purchase', name: 'Completed Purchase', type: 'conversion' },
    { key: 'revenue', name: 'Revenue', type: 'numeric' },
  ],
  audience: {
    percentage: 100, // Test 100% of users
    filters: {
      plan: { in: ['free', 'pro'] },
    },
  },
})

// Assign user to variant
const assignment = await experiments.assign(experiment.id, {
  userId: 'user-123',
})

console.log(`User assigned to: ${assignment.variant}`)

// Track goal conversion
await experiments.trackGoal(experiment.id, {
  userId: 'user-123',
  goal: 'purchase',
  value: 9999, // Revenue in cents
})

// Get experiment results
const results = await experiments.getResults(experiment.id)
console.log(`Control conversion: ${results.variants.control.conversionRate}%`)
console.log(`Variant A conversion: ${results.variants['variant-a'].conversionRate}%`)
console.log(`Statistical significance: ${results.significance}`)
console.log(`Winner: ${results.winner}`)

// End experiment
await experiments.end(experiment.id, {
  winner: 'variant-a',
  rolloutToAll: true,
})

Core Capabilities

A/B/n Testing - Test multiple variants
Statistical Analysis - Automatic significance calculation
Goal Tracking - Multiple conversion goals
Auto Winner - Automatic winner declaration
Audience Targeting - Test specific user segments

Access Methods

SDK

TypeScript/JavaScript library for experiments

await experiments.assign(experimentId, { userId: 'user-123' })

→ SDK Documentation

CLI

Command-line tool for experiment operations

do experiment create "Button Color Test" --variants variants.json

→ CLI Documentation

API

REST/RPC endpoints for experimentation

curl -X POST https://api.do/v1/experiments -d '{"name":"Test","variants":[...]}'

→ API Documentation

MCP

Model Context Protocol for AI-driven experiments

Create an A/B test for checkout button color with blue and green variants

→ MCP Documentation

Parent Primitive

analytics - Analytics and metrics platform

Sibling Primitives

kpis - KPI tracking
performance - Performance monitoring
trace - Distributed tracing

feature-flags - Feature flags
decide - Decision-making (SDK object mapping)
send - Event tracking

experiments

On this page