An online store’s numbers change every day, even when nothing in the business has changed. Reacting to every swing wastes time hunting for causes that do not exist and often breaks a campaign that was working. Waiting too long means missing a real shift and paying for it. One day cannot settle the question. What helps is comparing like with like, knowing the normal range of your own numbers and deciding in advance what further movement would justify action.
Numbers fluctuate even when nothing has changed
Conversion rate is a proportion: how many visits ended in an order. Even if every visitor had exactly the same chance of buying every day, the number of orders would still fluctuate by pure chance – much like the number of heads in a hundred coin tosses. A simple formula for the uncertainty of a proportion describes how large that fluctuation is. The point to take from it is this: the fewer the orders, the larger the relative swings.
An illustrative example with invented numbers: a store has 1,000 visits a day and a long-run conversion rate of 2%, so 20 orders on average.
| Period | Visits | Average orders | Normal range from chance alone (± 2 deviations) | Conversion rate within that range |
|---|---|---|---|---|
| Day | 1,000 | 20 | roughly 11 to 29 | roughly 1.1% to 2.9% |
| Week | 7,000 | 140 | roughly 117 to 163 | roughly 1.7% to 2.3% |
A day with 14 orders (a 1.4% conversion rate) looks like a 30% drop, yet it sits within the range that chance alone produces. A week with 112 orders (1.6%) falls outside it and is worth investigating.
That range is the floor of real-world fluctuation. In practice, chance is joined by weekday patterns, weather, paydays, email sends and swings in advertising. It is therefore better to build the range from your own history than from the formula. The formula mainly serves as a warning: for a small store or a narrow segment (one category, one campaign, mobile visits from one source) the daily figure is almost always mostly noise. Statistics textbooks also note that with very few successes – as a rule of thumb, fewer than five orders – even the calculation itself stops being reliable.
Compare like with like
Most false alarms come not from chance but from a poor comparison. Seasonality means regularly repeating fluctuations, with retail sales before Christmas as the classic example. In many stores, weekdays differ just as regularly. A Monday compared with the previous Sunday can therefore show “growth” that is simply the rhythm of the week.
Even official statistics compare only after adjustment. The Czech Statistical Office reports year-on-year retail sales adjusted for calendar effects, and month-on-month changes adjusted for seasonality as well. For an online store that means three practical rules:
- Compare a day with the same weekday over several previous weeks, not with yesterday.
- Compare a month or season year on year, and watch the number of weekends, public holidays and shifting events (Black Friday, Christmas, Easter).
- Keep market context at hand. According to the Czech Statistical Office, online and mail-order retailers grew sales by 11.8% year on year in April
- That is one month and a whole segment, not a benchmark for your category. But if the market grows while you stand still, that is a different message from everyone standing still.
Volume must be comparable too. After a strong campaign aimed at a new type of visitor, conversion rate falls even though nothing on the site broke; the mix of visits changed, not the willingness to buy.
A range built from your own history
Manufacturing has been answering the same question for decades. A control chart plots values over time and draws limits around the average, usually three standard deviations away. A point outside the limits is a reason to look for a cause. A process is “in control” when the points lie within the limits and form a random pattern.
For daily figures the simplest version fits best: the individuals chart. Its limits are calculated from how much the value changes on average from one day to the next. In practice, take the same weekdays over the last ten to twelve weeks, calculate the average and the average difference between consecutive values. The limits sit roughly 2.66 times that average difference above and below the average.
On top of the chart, two rules from the Western Electric method are enough:
- one point outside the limits – a clear deviation, investigate now;
- eight points in a row on one side of the average – a quiet shift that nobody would notice day by day.
Each additional rule increases sensitivity, but also false alarms. With the three-deviation rule alone, a false alarm occurs on average once every 371 points; with the full set of Western Electric rules, roughly once every 92. For daily data that is the difference between about one false alarm a year and one a quarter – and only if the data meet the method’s assumptions.
Everyday tools use the same logic. Google Analytics flags a value as an anomaly when it falls outside the interval predicted by a model trained on history; for daily data the model learns from the last 90 days. Here too, yesterday is not the benchmark.
Your store’s numbers in context
Korzaro connects store, analytics and advertising data so today’s numbers can be read alongside their history.
Beware of the return to the average
An unusually bad day is usually followed by a better one, even if nobody does anything. Statisticians call this regression to the mean: unusually high or low values tend to be followed by values closer to the average, and natural fluctuation then looks like real change.
For decisions, this has an awkward consequence. Whoever changes the budget, price or campaign right after a slump will usually see an improvement the next day – and credit it to the intervention. Likewise, the best day of the month tends to be followed by a weaker one, and “something went wrong”. An intervention should therefore be judged against a comparable period and the normal range, not against the extreme day that triggered it.
When statistics cannot help
Control limits assume that days are comparable and fluctuate around a stable average. In an online store that does not always hold:
- Too little history. Two or three weeks of data are not enough for a reliable range, especially if they include a sale or a holiday.
- A change in measurement. A new checkout, an edit to the tracking code or a change in cookie consent shifts the numbers without any change in the business. First check whether the way of counting has changed.
- A planned change. After launching a new campaign or a clearance sale, different behaviour is expected; old limits do not apply.
- A small sample. The conversion rate of a category with five orders a week cannot be judged statistically.
In these situations, the word “significant” or a precise-looking percentage does not help. It is more honest to state what the data show, what they do not and how many more days or orders are needed before a decision can be made.
Three situations and what to do about them
| What you see | What it most likely means | Sensible next step |
|---|---|---|
| Value inside the normal range, no long run | Ordinary fluctuation | Change nothing, keep watching |
| One day outside the range | Possibly a real cause, possibly an exception | Check tracking, site availability, payments and ads; leave the budget alone for now |
| A run of days on one side of the average, or a week outside the range | Probable shift | Look for the cause and decide on action |
What to do now
- Pick two or three numbers you actually decide by – typically orders, revenue excluding VAT and conversion rate.
- Calculate the normal range for each weekday from the last ten to twelve weeks, leaving out major sales and holidays.
- Write down the threshold for action in advance. For example: one day outside the range = check tracking and the site; eight days below the average or a week outside the range = look for the cause and decide on action.
- Track small segments weekly, not daily. Where there are fewer than five orders per period, do not calculate a range.
- Judge an intervention against a comparable period, not against the day that triggered it.