Web analytics are fine for companies that can measure campaign effectiveness, all the way to sales completion. But what about companies that sell via indirect channels such as through retail stores, partnerships, and Joint Ventures? What is the best practice for top brands who need to measure campaign effectiveness under these constraints?
For CPG and other brand management companies, measuring the health of the brand with its consumers who actually purchase the products through a multitude of channels such as retail stores, restaurants, franchisees as well as company owned stores has always been challenging.
For on-line retailers there are web analytics and for brick and mortar business there are POS solutions, but how does the company that sells through partners and channels measure the ultimate satisfaction and purchase intent of their final consumer?
Working with many of our large CPG clients we have developed a Consumer brand health index to measure consumers' awareness, favorability, purchase intent and brand health metrics. This index can be combined with the net promoter score index to fully understand the level of satisfaction your consumers have with your brand and to understand if that level is increasing or decreasing.
We at SensorPro work with some of the world’s best brands and many household names. We developed an approach that allows these brands to quickly evaluate the effectiveness of both offline and online marketing campaigns. The solution uses a combination of control/exposed group testing using our enterprise grade survey platform in combination with our email marketing platform and other techniques.
So how does it work?
Like every good market research project, the brand team need to define the objective of the evaluation: was it raising awareness? the announcement of a new promotion or partnership? or just the good old fashioned increasing sales?
Once the objective is defined you can move onto defining a project approach and time table. For the purposes of this article we are going to assume that the objective is to “increase sales” and that the deployment is online using an email marketing platform.
Using your email marketing platform, identify subscribers who were exposed to the campaign. The control group will be subscribers who were not provided a copy of that newsletter .
It can be more difficult to do the same thing with traditional print media, but clearly it can be done using creative techniques like mobile phone tag readers or simple urls for people who were exposed.
The other approach is to use a blind panel for the control group where you can filter out people who have had exposure via some simple survey techniques. Then, refine the groups using Cohorts -within each group we will identify Cohorts which are sub groupings based on age, gender or a demographic of your choice.
Our typical engagement will call for an initial survey to the exposed and control groups being deployed fairly soon after the deployment of the campaign. This is important as you do not want other factors influencing the decisions and answers that the respondents will provide. With the explosion of social networks like Facebook, LinkedIN and Twitter, this can raise additional hurdles but should clearly be incorporated into your approach.
For the initial survey we recommend that the survey be kept short and focused, you can always use triggers and recurring survey campaigns to send out requests for more information. The questions should clearly all be tied back to the objectives of the evaluation so if the campaign objective was to raise awareness of a new promotion, then you would measure awareness and purchase intent, whereas if it was for a new product announcement you probably would be asking about awareness and favorability.
Our objective for this sample research project was to measure the effectiveness on increasing sales. Then, depending on the time period of the promotion or campaign, you will send out a follow up survey asking about actual purchases. We typically see this follow up type survey sent out seven to fourteen days after the commencement of the campaign.
Assuming that you have properly defined your objectives, selected good groups and had a reasonable level of responses, you will want to perform a statistical significance test on your survey results. The following is an example of the type of report that helps decide if the objective was met:
The critical metric is the Lift Factor which measures the net result of the marketing campaign between the control and exposed groups.
Just because the numbers are higher or lower do not by themselves speak the result, as we used a sample of the population. So we must employ a little science to verify the results. Thankfully SensorPro has built in Statistical Significance Testing to test our hypothesis of "is the difference in responses statistically significant?" When Statisticians say a result is "statistically significant" they don't mean it's important, just that it's "probably true".
Ultimately we want to know if the difference in response happened by chance or not. If they did not happen by chance, then we are reasonably sure it was our campaign efforts that influenced the outcome. QED.
We repeat the analysis for each Cohort eg was there a significance between those born in the 80's vs 90's and make more informed decisions on these fine-tuned results versus aggregated data as a whole.
You have now opened a conversation with these prospects and customers. They have told you something about your campaigns, your products, your brands and your company. So use this data by integrating with your email subscribers and segment and target these people based on their responses.
For example if someone said that a certain new product is not favorable to them, then exclude them from future campaigns that highlight that new product. Conversely if they tell you that they like something, use that information to expand your sales by cross selling and up selling, all traditional and proven techniques.