My Blog

A pair of B-school professors impugn our favorite metric. And they may have a point.

All of us at BrainPosse have taken a big swig of the ROI Kool-Aid©. We believe that return on investment is the most important marketing communications metric by far. So imagine our surprise when a recent article by Professors Tim Calkins and Derek Rucker, of Northwestern University's Kellogg School of Management, warned against overemphasizing what, to us, is the mother of all benchmarks.


Their concern was succinctly stated: "...there is a fundamental problem with overemphasizing ROI as the single measure of success: It is often impossible to accurately quantify the impact." They say the difficulty (or impossibility) of quantifying ROI is due to the complex nature of the various "returns" marketing delivers. And they have a valid point.

There are three basic marketing ROI models:

Transactional ROI is the incremental change in sales over or under trend line, minus the incremental cost of increased sales and the marketing communications cost, with the resulting number divided by the marketing cost to get ROI.

Relational ROI can be measured with similar formulae. Catalog retailers have done it for years by determining customer value (total profit the company derives from a typical customer over the lifetime of the business relationship), subtracting the customer acquisition cost then dividing by the customer acquisition cost to find ROI.

Brand-equity ROI begins with determining the value of a brand, defined as gross profit the company derives from selling the branded product above what they might expect to get selling an identical, but unbranded, product. The marketing cost is subtracted from that value and the resultant number divided by the marketing cost to determine ROI.

A textbook example of ROI calculation

A simple, straightforward, ROI calculation was developed from a recent effort for one of our clients. In this case, it was transactional ROI. The client had a product in one SMA. The clients' unit sales and those of the two competitors in the market were tracked and published by a government entity.

There were no external variables. Neither competitor changed marketing strategy, communications materials or media weight. No new competitor entered the market. The total unit sales reported by the government did not change.

The only factor that changed was our client's marketing and communications program, which increased revenue by a factor of three, and took them from a distant third to first in the market.
The client CFO did a meticulous accounting of incremental sales cost and we supplied total marketing cost. These were subtracted from the incremental increase in revenue, and then divided by the marketing cost to get 132% ROI.

Except: ROI was only calculated for the duration of the marketing communications campaign. When volume reached the client's capacity, the marketing communications was stopped. After marketing support was cut off, volume didn't immediately return to pre-campaign levels, it tapered off very gradually. The incremental value of that additional volume above baseline was not part of the ROI calculation.

Then periodic "refresher" bursts of marketing support brought volume back up to levels initially achieved by the campaign, but with a small fraction of the original budget. These bursts, and the resultant bottom-line increase, weren't part of the ROI calculation either.

Eventually, when both competitors began to market more aggressively in response to our client's success, additional short bursts of the original campaign were all it took to keep our client at capacity levels. But preventing the loss of revenue which might normally be expected when competitor’s ramp up their marketing doesn't figure in the ROI calculations, either.

Why's it so complex?

1. It's all related. Retail promotions are usually cited as the type of marketing activity that best lends itself to a clear ROI analysis. Offering 0% financing this week only lets retailers define the time period, see the cost of the offer, track the cost of marketing and compare profit to similar periods in pervious years (adjusted for trend lines) to get a dead-on ROI.
Except: What is the future value of new customers brought into the retailer's store or web site? And what is the impact (positive or negative) on the retailer's brand? If the retailer previously had an up-market image and clientele, a screamer price promotion's price might have to include loss of valuable brand equity.

On the other side of the marketing communications spectrum, a strong branding campaign might not have an immediate impact on sales, but could build a groundswell of product preference, customer loyalty, future sales and price elasticity to generate massive profits over time. Works for Coca-Cola.

2. The world changes. The same week a retailer runs a 0% financing offer, all the competition might be offering 25% off. Or 0% financing and 25% off. And so the promotion would, in fact, be a lot more successful than the raw numbers make it appear. Because the real revenue number should be: what would revenue have been without the promotion?

We did a campaign for a client about to get a strong new competitor. The client projected a 25% revenue loss. We developed a strategy and campaign which maintained revenues at the previous level. The numbers might look bad at first glance. There was additional marketing communications cost and the revenue didn't budge. But not budging was actually avoiding a 25% decline. The result was a major improvement in the bottom line, but one that's hard to quantify with conventional ROI calculations. Maybe those Kellogg profs could take a shot at that conundrum.

Relational ROI analysis is especially susceptible to the vagaries of change, because they're all based on past experience. New customers will remain customers for a historically constant period of time, purchase at predictable increasing, then declining, rates over that time and generate foreseeable revenues and profits. Fine, unless a tectonic change hits the market. The way gas prices are impacting car and truck sales. Or the way the internet crushed travel agents.

Relational ROI is a great way to measure past performance. Not so great as a planning tool.

3. There are always assumptions. Among the many really smart things Einstein said was: "Not everything that matters can be counted." Goes double for ROI analysis.

The simple – and admittedly somewhat simplistic – formula for transactional ROI ignores outside variables in the marketplace and attributes the entire incremental change in sales (whether an increase or decrease) to marketing communications. It is generally accurate in relatively static markets, or when sales changes can be unquestionably linked to marketing communications. The formula is less reliable when markets are in flux, or when multiple factors impact sales simultaneously. The impact of changes has to be built in as a guesstimate. But if your CFO howls, remind her or him how many assumptions are built into valuation calculations. That's what a whole lot of those footnotes in the annual report are all about.

The "lifetime customer value" calculation of relational ROI is based on the assumption that most important factors in the market will be constant. It can be thrown off by significant shifts – like online shopping.

Brand valuation ROI is the least concrete of all. Because determining that value requires a sequence of assumptions, the final figure is an approximation rather than an absolute number.
How much sweet, fizzy, black liquid would Coca-Cola sell if it was store brand? Probably none, because Pepsi and maybe RC would sell all the product in this brand-driven market segment. How much if there wasn't Pepsi or RC? Probably not much. Because cola category sales are driven by brand attributes. So how to value the brand? Maybe by attributing all Coke sales to the brand identity. How 'bout the cost of building that brand? Maybe all marketing and communications run during the lifetime of the prime consuming group should be tallied.

As you can see, this one is seriously tough.

Although we're reasonably sure that the annual Interbrand list of brand equity ranks the brands more or less correctly in relation to each other, it would be virtually impossible to confirm that the specific dollar values assigned to each are correct.

The formulas used to calculate brand equity often run to several pages, and they're full of impressive symbols like ∑ and ∆ and ≠ . But because they all contain implicit or explicit assumptions about the relationships of several basic factors, these formulas for brand valuation are more like directional guideposts rather than precise measurements. The answers they provide are more on the order of "Up," rather than "3.2 meters higher."

Since the brand valuation itself is somewhat imprecise, the return on the investment driving that value cannot be exact. That certainly doesn't mean that brand-equity ROI isn't valuable. Only that it's probably not meaningful to carry it to the fifth decimal point.

Assuming assumptions are acceptable, ROI is still the big Kahuna of metrics.

At times it can be pretty risky to present a ROI analysis as a precise figure (although we did just that with our 132% noted above). But with the understanding that assumptions are an inevitable part of the calculation, ROI is still the most meaningful metric of marketing and communications success. And by the way, it's the one CEOs and CFOs can relate to.