‘Tis the season of the mileage run. Some frequent fliers find themselves just short of qualifying for a particular elite status tier in an airline loyalty program for next year. In order to secure status, some will book cost-effective flights to get the right number of miles to cross the qualifying thresholds.
In recent years the airlines have offered, in exchange for cash, the opportunity to maintain status, or even “boost” to the next level. The cash needed to be tendered generally depends on the current status level and the size of the gap between the current number of miles and the targeted threshold.
But it turns out that the math behind the dollar offer isn’t that simple. Take a look at the following graph (click on it for an enlarged view). It presents some 2016 self-reported data from flyers enrolled in the American Advantage program. The data was culled from this thread on Flyertalk.com and it shows the current mileage versus the amount of the offer. The color of the bubbles represent the current elite level: gold, platinum, or executive platinum. The size is the number of flyers reporting their mileage and offer at that point. The line represents the best linear fit of the data points, within each of the three current levels.
There is clearly some variability in the offers even when considering the current level/gap profiles. In other words, two flyers with similar status and miles often get different offers. If a very simplistic way of thinking of the offers were in place, we would expect all of the dots to fall on the fitted line or have some non-varying pattern along that line. But we see a variety of dollar offers even for the similar mileage levels within an elite level.
What’s going on here? Well, it is difficult to reverse engineer a loyalty marketing program, but there are at least two explanations. First, it is likely that there are other criteria, such as recent dollar spend, or expected future customer value, that may be part of the equation. In fact, we would hope that whatever mechanism is used to arrive at the offer is driven by some kind of statistical model to maximize the additional revenue gained by response to these offers.
But we also expect that there is some kind of experimentation going on. Experimental design is a scientific way to generate the data you need to help understand things like customer response under a wider variety of conditions than what you might uncover by just looking at historical data, or worse, by making hunches.
Variation can be your friend in measuring consumer and market response. Whether by examining historical, naturally occurring variation, or better yet, by using experimental design to generate systematic variation to more richly measure response, an understanding of the power of variation as a tool is an important capability for every company.