SOP 06: Learning System

How Every Completed Experiment Creates Learning

Every completed experiment must create a learning entry, even if the result is weak or disappointing.

No completed test ends at the verdict alone.

The team must extract:

• what happened
• what it means
• what should change next

Difference Between Wins, Losses, and Patterns

Wins

A win is a result worth keeping because the treatment improved outcomes with acceptable guardrails.

Losses

A loss is a result that failed, hurt performance, or disproved the hypothesis enough to stop the move.

Patterns

A pattern is a repeated lesson that shows up across more than one experiment, store, category, or period.

Patterns matter most because they travel.

How to Write a Useful Learning Entry

Keep it short and operational.

A useful entry should include:

• what was tested
• what changed
• what the result means
• when it should influence future decisions

Avoid vague lines like “users liked it more” or “performance improved”.

State the commercial meaning clearly.

Why Learning Is the Moat

Wins are helpful.

A structured body of learning is harder to copy.

The learning system becomes the moat because it improves:

• future diagnosis
• future confidence scoring
• future experiment design
• future speed of decision-making

Example Win

Showing delivery estimate and return reassurance near the add-to-cart button increased add-to-cart rate on mobile PDPs for high-intent traffic.

Example Loss

Adding a large comparison table above the fold reduced product focus and did not improve add-to-cart rate despite strong internal enthusiasm.

Example Pattern

When value clarity is weak, concise above-the-fold benefit framing tends to outperform longer educational content blocks on mobile-first traffic.