Skip to Content
SOP04 Experiment Workflow

SOP 04: Experiment Workflow

When a Node Becomes an Experiment

A node becomes an experiment when:

  • it has been selected through the priority engine
  • the hypothesis is specific enough to test
  • the owner is clear
  • the metric and guardrails are defined
  • the execution tool is chosen

What Tools Are Used

Use:

  • Optimizely when the program requires a more established experimentation layer
  • Statsig when feature rollout speed and product-side iteration matter more

UXY1 does not replace these tools.

It decides what moves into them.

How the Experiment File Is Created

Create a new file in experiments/active/ using the experiment template.

The filename should be:

exp-###-short-description.md

The file should point back to the matrix node and define the execution logic clearly enough that the team can launch and review it without extra interpretation.

Mandatory Fields

  • title
  • related node
  • hypothesis
  • area
  • primary metric
  • guardrails
  • expected impact
  • owner
  • tool
  • status
  • rollout plan

What Happens While the Test Is Running

  • the experiment owner updates status
  • any risk or implementation issue gets logged
  • no major scope changes happen mid-test unless required
  • the team watches primary metric and guardrails, not random noise
  • the linked matrix node stays in testing

Minimum Experiment Spec

  • Hypothesis
  • Metric
  • Guardrails
  • Expected impact
  • Owner
  • Status

Full Example Experiment Spec

Title

exp-014-cart-shipping-clarity

node_018

Area

CART

Hypothesis

If we show shipping threshold progress, estimated delivery window, and a simple shipping-cost explanation directly in cart, more users will continue to checkout and fewer will abandon after seeing late-stage cost uncertainty.

Primary metric

Cart-to-checkout start rate

Guardrails

  • conversion rate must not drop
  • AOV must not decline materially
  • support complaints about delivery promise must not increase

Expected impact

Medium-high uplift in checkout starts, strongest for mobile users and baskets just below free-shipping threshold.

Owner

Growth Lead

Tool

Optimizely

Rollout plan

50/50 split for eligible cart sessions for 14 days, with mobile and desktop reviewed separately.

Status

testing

Notes While Running

Watch lower basket-size segments closely. If the effect only appears below the shipping threshold, that still counts as useful learning and may justify segmented rollout later.

Last updated on