<aside> 🦖 Art + Science

</aside>

Science part: measure the relationship between short term metric and long term goal

Based on observational data:

• Predictability - F1 Score

• Measurability - Sample size needed to see a reasonable change in experiments

Based on experimentation data - luxury for companies that can run hundreds and thousands of experiments in the same product / have longer term holdouts

Use meta-analysis or longer term holdout to understand what metric delta movement is most highly correlated with the delta movement in the longer term