Key metrics LONG ENG 11-25
11. High return categories
In the experiment targeting product categories with consistently high return rates, the objective is to reduce structural fit problems that drive excessive operational cost.
The North Star Metrics include Category Return Rate and Net Revenue per Order. Category Return Rate reflects long-term improvements in fit quality at the category level, while Net Revenue per Order captures the financial benefit of reducing avoidable returns.
The Primary Success Metrics are Category Return Rate in the experiment and Margin per Order, which directly measure whether the intervention improves fit outcomes while preserving profitability.
The Guardrail Metrics, Conversion Rate and Stock-out Rate, ensure that category-level changes do not reduce purchase completion or create availability issues.
The Secondary Metrics, such as Order Value and Category Mix, provide insight into changes in purchasing behavior across categories.
12. New brand onboarding
In the experiment onboarding a new brand with unknown sizing characteristics, the objective is to quickly stabilize fit performance and reduce uncertainty for customers.
The North Star Metrics include Return Rate by Brand and Net Revenue per Order. Return Rate by Brand reflects long-term fit quality for the new brand, while Net Revenue per Order captures the financial impact of early sizing accuracy.
The Primary Success Metrics are Return Rate by Brand in the experiment and Correct Size Selection Rate, which directly measure whether the onboarding logic improves initial size decisions.
The Guardrail Metrics, Complaint Rate and Conversion Rate, ensure that early sizing adjustments do not harm customer trust or reduce purchase completion.
The Secondary Metrics, such as Fit Feedback Score and Support Ticket Volume, help diagnose early quality issues and user confusion.
13. Cross-border sizing
In the experiment addressing cross-border sizing differences between markets, the objective is to reduce systematic misalignment between regional sizing systems.
The North Star Metrics include Return Rate by Country and Net Revenue per Order. Return Rate by Country reflects long-term fit quality across regions, while Net Revenue per Order captures the financial impact of reducing cross-border returns.
The Primary Success Metrics are Size Exchange Rate and Cross-border Return Rate, which directly measure whether the new mapping improves size consistency across countries.
The Guardrail Metrics, Conversion Rate and Delivery Failure Rate, ensure that changes in sizing logic do not harm international purchase completion or logistics performance.
The Secondary Metrics, such as Delivery Time and Customs Delay Rate, provide insight into operational side effects of cross-border adjustments.
14. Seasonal new fit patterns
In the experiment launching seasonal collections with new fit patterns, the objective is to validate fit assumptions before scaling the collection broadly.
The North Star Metrics include Return Rate by Collection and Repeat Purchase Rate. Return Rate by Collection reflects long-term fit quality for the new season, while Repeat Purchase Rate captures customer satisfaction driven by good early experiences.
The Primary Success Metrics are Return Rate by Collection in the experiment and Correct Size Selection Rate, which directly measure whether the new patterns match customer expectations.
The Guardrail Metrics, Conversion Rate and Complaint Rate, ensure that experimental designs do not reduce purchase performance or increase dissatisfaction.
The Secondary Metrics, such as Fit Feedback Score and Style-level Return Rate, help identify specific styles that drive fit problems.
15. Premium vs fast-fashion sizing
In the experiment comparing sizing performance between premium and fast-fashion segments, the objective is to understand structural differences in fit quality across price segments.
The North Star Metrics include Net Revenue per Order and Customer Lifetime Value. Net Revenue per Order reflects immediate profitability, while Customer Lifetime Value captures long-term business impact driven by consistent fit quality.
The Primary Success Metrics are Return Rate by Segment and Margin per Order, which directly measure fit performance and economic efficiency across segments.
The Guardrail Metrics, Conversion Rate and Discount Rate, ensure that segment-specific changes do not reduce purchase completion or trigger excessive price incentives.
The Secondary Metrics, such as Basket Size and Repeat Purchase Rate, provide insight into behavioral differences between premium and fast-fashion customers.
16. Fit confidence score display
In the experiment displaying a fit confidence score to support size decisions, the objective is to make model uncertainty explicit and guide users toward safer size choices.
The North Star Metrics include Size-related Return Rate and Conversion Rate. Size-related Return Rate reflects long-term improvements in fit accuracy, while Conversion Rate ensures that exposing uncertainty does not reduce overall purchase performance.
The Primary Success Metrics are Recommendation Click Rate and Correct Size Selection Rate, which directly measure whether users rely on the confidence signal and whether it improves the quality of size choices.
The Guardrail Metrics, Conversion Rate and Time on Product Page, ensure that the additional information does not create hesitation or slow down the decision process.
The Secondary Metrics, such as Confidence Score Distribution and Hesitation Time, provide insight into how users interpret and react to different confidence levels.
17. Social proof “80% kept this size”
In the experiment adding social proof about the proportion of customers who kept a given size, the objective is to influence size choice through collective behavior signals.
The North Star Metrics include Conversion Rate and Size-related Return Rate. Conversion Rate captures immediate impact on purchase completion, while Size-related Return Rate reflects long-term fit quality driven by better-informed choices.
The Primary Success Metrics are Conversion Rate on the Product Page and Add-to-Cart Conversion, which directly measure whether social proof improves funnel progression.
The Guardrail Metrics, Return Rate and Complaint Rate, ensure that social proof does not create misleading expectations or increase dissatisfaction.
The Secondary Metrics, such as Trust Interaction Rate and Badge Visibility Rate, help explain how users perceive and engage with the social signal.
18. Virtual fitting room
In the experiment introducing a virtual fitting room experience, the objective is to provide a richer fit simulation to reduce uncertainty before purchase.
The North Star Metrics include Size-related Return Rate and Net Revenue per Order. Size-related Return Rate reflects long-term reduction of fit-related returns, while Net Revenue per Order captures the financial benefit of improved decision quality.
The Primary Success Metrics are Feature Adoption Rate and Correct Size Selection Rate, which directly measure whether users adopt the feature and whether it improves size accuracy.
The Guardrail Metrics, Conversion Rate and Page Load Time, ensure that the heavy feature does not degrade performance or reduce purchase completion.
The Secondary Metrics, such as Time in Feature and Feature Abandonment Rate, provide insight into engagement and usability of the virtual fitting experience.
19. 3D avatar try-on
In the experiment launching a 3D avatar try-on feature, the objective is to visualize fit more realistically and reduce systematic misjudgment of size.
The North Star Metrics include Size-related Return Rate and Conversion Rate. Size-related Return Rate reflects long-term fit improvement, while Conversion Rate ensures that the visualization does not harm purchase performance.
The Primary Success Metrics are Feature Adoption Rate and Correct Size Selection Rate, which directly measure whether users use the avatar and whether it improves size choices.
The Guardrail Metrics, Time on Product Page and Rendering Error Rate, ensure that the 3D experience does not introduce latency or technical failures that disrupt the journey.
The Secondary Metrics, such as Session Length and Interaction Depth, provide insight into how deeply users engage with the try-on feature.
20. Fit feedback collection
In the experiment collecting fit feedback after delivery to improve future recommendations, the objective is to close the learning loop between outcomes and model training.
The North Star Metrics include Size-related Return Rate and Repeat Purchase Rate. Size-related Return Rate reflects long-term improvement driven by better training data, while Repeat Purchase Rate captures loyalty effects of improved fit quality.
The Primary Success Metrics are Feedback Submission Rate and Coverage of Fit Feedback, which directly measure whether sufficient high-quality labels are collected.
The Guardrail Metrics, Conversion Rate and Survey Drop-off Rate, ensure that feedback collection does not harm the purchase funnel or overburden users.
The Secondary Metrics, such as Fit Feedback Score and Future Return Rate, help quantify how feedback quality translates into future fit performance.
21. Aggressive size override
In the experiment aggressively overriding user size choices based on the algorithm, the objective is to test whether forced correction improves outcomes despite reduced user control.
The North Star Metrics include Conversion Rate and Net Revenue per Order. Conversion Rate reflects immediate business impact, while Net Revenue per Order captures the financial effect of fewer incorrect purchases.
The Primary Success Metrics are Override Acceptance Rate and Correct Size Selection Rate, which directly measure whether users accept the override and whether it improves size accuracy.
The Guardrail Metrics, Complaint Rate and Refund Rate, ensure that forced intervention does not create excessive dissatisfaction or operational cost.
The Secondary Metrics, such as Manual Size Change Rate and Customer Support Contacts, provide insight into resistance and downstream friction.
22. Forcing size before add-to-cart
In the experiment forcing users to select a size before allowing add-to-cart, the objective is to prevent ambiguous purchases and improve data quality for size decisions.
The North Star Metrics include Conversion Rate and Net Revenue per Order. Conversion Rate captures immediate funnel impact, while Net Revenue per Order reflects the economic benefit of fewer wrong-size orders.
The Primary Success Metrics are Add-to-Cart Conversion and Checkout Start Rate, which directly measure whether the constraint improves structured progression through the funnel.
The Guardrail Metrics, Drop-off Rate and Time to Purchase, ensure that the constraint does not introduce excessive friction or abandonment.
The Secondary Metrics, such as Back Navigation Rate and Form Error Rate, help diagnose frustration and usability issues.
23. Removing size guide
In the experiment removing the size guide from the product page, the objective is to test whether simplified UI improves speed at the cost of fit quality.
The North Star Metrics include Conversion Rate and Net Revenue per Order. Conversion Rate captures immediate purchase performance, while Net Revenue per Order reflects the net effect after accounting for potential return increases.
The Primary Success Metrics are Conversion Rate on the Product Page and Add-to-Cart Conversion, which directly measure funnel performance without the guide.
The Guardrail Metrics, Return Rate and Complaint Rate, ensure that UI simplification does not significantly harm fit outcomes or customer satisfaction.
The Secondary Metrics, such as Customer Support Contacts and Help Page Views, provide insight into hidden information needs.
24. Auto-switching size
In the experiment automatically switching the selected size based on the algorithm, the objective is to test whether automated correction outperforms user choice.
The North Star Metrics include Size-related Return Rate and Net Revenue per Order. Size-related Return Rate reflects long-term fit improvement, while Net Revenue per Order captures the financial benefit of fewer wrong-size orders.
The Primary Success Metrics are Auto-switch Acceptance Rate and Correct Size Selection Rate, which directly measure whether users accept the switch and whether it improves accuracy.
The Guardrail Metrics, Complaint Rate and Manual Revert Rate, ensure that automation does not undermine trust or increase resistance.
The Secondary Metrics, such as Override Frequency and Refund Rate, provide insight into conflict between algorithm and user intent.
25. “Likely wrong size” warning
In the experiment adding a warning for likely wrong size selections, the objective is to nudge users away from high-risk choices without blocking autonomy.
The North Star Metrics include Size-related Return Rate and Net Revenue per Order. Size-related Return Rate reflects long-term reduction of wrong-size returns, while Net Revenue per Order captures the net business benefit of safer choices.
The Primary Success Metrics are Size Change Rate after Warning and Correct Size Selection Rate, which directly measure whether the warning changes behavior and improves outcomes.
The Guardrail Metrics, Conversion Rate and Warning Dismiss Rate, ensure that the warning does not excessively disrupt the purchase flow.
The Secondary Metrics, such as Warning Click Rate and Hesitation Time, provide insight into how users process and react to the warning.
Comments
Post a Comment