SOP 04: Experiment Workflow
When a Node Becomes an Experiment
A node becomes an experiment when:
- it has been selected through the priority engine
- the hypothesis is specific enough to test
- the owner is clear
- the metric and guardrails are defined
- the execution tool is chosen
What Tools Are Used
Use:
- Optimizely when the program requires a more established experimentation layer
- Statsig when feature rollout speed and product-side iteration matter more
UXY1 does not replace these tools.
It decides what moves into them.
How the Experiment File Is Created
Create a new file in experiments/active/ using the experiment template.
The filename should be:
exp-###-short-description.md
The file should point back to the matrix node and define the execution logic clearly enough that the team can launch and review it without extra interpretation.
Mandatory Fields
- title
- related node
- hypothesis
- area
- primary metric
- guardrails
- expected impact
- owner
- tool
- status
- rollout plan
What Happens While the Test Is Running
- the experiment owner updates status
- any risk or implementation issue gets logged
- no major scope changes happen mid-test unless required
- the team watches primary metric and guardrails, not random noise
- the linked matrix node stays in
testing
Minimum Experiment Spec
- Hypothesis
- Metric
- Guardrails
- Expected impact
- Owner
- Status
Full Example Experiment Spec
Title
exp-014-cart-shipping-clarity
Related node
node_018
Area
CART
Hypothesis
If we show shipping threshold progress, estimated delivery window, and a simple shipping-cost explanation directly in cart, more users will continue to checkout and fewer will abandon after seeing late-stage cost uncertainty.
Primary metric
Cart-to-checkout start rate
Guardrails
- conversion rate must not drop
- AOV must not decline materially
- support complaints about delivery promise must not increase
Expected impact
Medium-high uplift in checkout starts, strongest for mobile users and baskets just below free-shipping threshold.
Owner
Growth Lead
Tool
Optimizely
Rollout plan
50/50 split for eligible cart sessions for 14 days, with mobile and desktop reviewed separately.
Status
testing
Notes While Running
Watch lower basket-size segments closely. If the effect only appears below the shipping threshold, that still counts as useful learning and may justify segmented rollout later.