Here's a useful metric that isn't commonly seen in many reports, even though it can be very insightful.
Let's illustrate it with an example. Consider this report showing transaction conversion rates for two months:
Month | Transaction conversion rate |
June 2023 | 14,9% |
April 2023 | 14,8% |
At first glance, these rates appear nearly identical, don't they?
Let's delve a bit deeper:
[June 2023] Conversion rate: 14,9%
[April 2023] Conversion rate: 14,8%
Spot any differences? The distinction lies in the deviation rates for each month. The metric that highlights such patterns and is part of descriptive statistics is called:
Standard deviation (StDev)
Let's re-examine the report, this time with the added standard deviation:
[June 2023]
Conversion rate: 14,9%
Standard deviation: 11,4
[April 2023]
Conversion rate: 14,8%
Standard deviation: 3,38
We use standard deviation as a health indicator when analyze large sets of data. A high standard deviation suggests that we might not want to place full trust in a metric because its components vary significantly. On the other hand, a low standard deviation indicates that the metric is consistent and can be deemed reliable.
How to calculate standard deviation:
- The simplest way is to use Excel formula and let the Excel do the job.
- Here is another way:
- Calculate the Mean: First, you need to find the average of the numbers.
- Find the Deviations: Subtract the mean from each number to find the deviation for each number.
- Square the Deviations: After you find the deviation for each number, square that deviation.
- Calculate the Mean of the Squared Deviations: Sum up all the squared deviations and then divide by (N - 1) where N is the total number of values. This is called the variance.
- Take the Square Root: The standard deviation is the square root of the variance.
Formula for Standard Deviation for a Sample:
What is a good standard deviation?
There's no one-size-fits-all answer. It varies based on the specific metric in question. Through practice and observation over the years, we’ve identified our own benchmarks. However, our approach is tailored to metrics specific to efood and might not apply universally.
Where to use/avoid standard deviation?
Examples to use | Examples to avoid |
Ratings | Single occurrence events |
Conversion rates | Categorical data (demographics, city etc.) |
Loading times | Binary data (boolean etc.) |
Basket size | Nominal or ordinal data |
Relevant posts:
![[Guide] Anomaly detection algorithm](https://images.unsplash.com/photo-1594904351111-a072f80b1a71?ixlib=rb-4.0.3&q=90&fm=jpg&crop=entropy&cs=srgb&w=1920)
![[Guide]](https://images.unsplash.com/photo-1506555191898-a76bacf004ca?ixlib=rb-4.0.3&q=90&fm=jpg&crop=entropy&cs=srgb&w=1920)
![[Essay] A strange A/B test](https://images.unsplash.com/photo-1545830384-3a2061eb44ed?ixlib=rb-4.0.3&q=90&fm=jpg&crop=entropy&cs=srgb&w=1920)