Marketing Mix Models: Maximizing organizational impact with expertise-led stakeholder buy-in

Consider for instance that you build an MMM model that is statistically rigorous but no-one believes; alternatively, imagine everyone is bought into your MMM models, but the model has fundamental gaps and inaccurately reflects the ROI of your marketing investments. In both scenarios you end up with a less than ideal situation.

In the first, there is a lack of buy-in for your models, which means consumers of the model (ex. marketing channel owners) may decide to completely ignore the key takeaways from your model, and continue with their original way of budgeting and planning, which may or may not be reliable.

In the second case where your model has fundamental gaps, the stakeholders are now relying on your statistically unsound models, to make key decisions on optimizing potentially large marketing budgets.

Either of these scenarios creates loss in potential revenue, and the impact can be significant.

The purpose of this document is to provide guidelines to gain stakeholder buy-in, and build a model that incorporates the technical components needed to build reliable models that establish credibility.

Marketing Mix Models (MMM) is a top-down aggregate marketing measurement technique, which allows us to quantify the impact of each marketing variable (ceteris paribus i.e. holding all else constant) at driving a response, by controlling for exogenous events. It help us understand the return on investment of each marketing channel, enabling us to make decisions around budget optimization, planning and forecasting.

I. Gaining Buy-In

Marketing Mix models are only as useful as stakeholder’s buy-in and adoption of the models to make decisions from it. Simply put, if no-one is using your models to make budgeting decisions, the technical sophistication of the model itself becomes much less important.

Here are some guidelines on how to navigate the change management process from previous ways of doing things (legacy models, gut instinct, alternate measurement methodologies), to maximize implementation of the recommendations from your models.

1. Stakeholder mapping: Start by identifying who your direct and indirect stakeholders (channel partners, growth marketing, brand marketing teams etc.) are, and what are their needs, motivations and objectives?
2. Stakeholder interviews: to understand how decisions were being made previously (legacy models, benchmark studies, prior beliefs, gut instinct etc.) and where are the gaps and pain points today (data challenges, under/over attribution, navigating internal politics etc.)
3. Initial exploratory data analysis (EDA): To validate (or not) the originally expressed gaps in the data, and conduct initial data analysis to identify areas of focus.
4. Stakeholder meetings: To play back your understanding of what you’ve heard from the stakeholder interviews and the results from your original EDA exercise, as well as share your phased plan for modeling i.e. what is in and out of scope at each phase, and tentative timelines. This step will allow you to incorporate stakeholder feedback, and get buy-in for the plan and make any changes needed to your modeling plan
5. Stakeholder check-ins: Periodic check-ins with stakeholders along the way, to share preliminary results from EDA and modeling, to bring them into the modeling process, and share rationale behind the results you are seeing, gather feedback, and continue to iterate. Ex. “we expect our TV FIFA campaign to do really well, as it has done well in copy testing, and brand lift studies, and we’ve put a significant investment against it”. Incorporate these “beliefs” into your models (if it makes sense), adjust and iterate as needed.
6. Preliminary results share out: Meet with individual stakeholder groups, to share preliminary directional results from the models, incorporate any additional feedback needed. This step is important, to get buy-in from individual stakeholder groups, prior to sharing your results more broadly
7. Final results share out: If you are able to gain buy-in along all stages of your marketing effectiveness journey, incorporating feedback from stakeholders, this step becomes much easier — to gain complete alignment on the final results of your model.

To sum up: People tend to be hesitant to see a 180 degree change in results overnight; change management is an iterative process which takes time, and needs due navigation. My guidance would be to not underestimate the value of these steps to get the necessary alignment on your models.

II. The model matters

All models are not created the same. While MMM models have an element of incorporating art (prior beliefs) together with science, with increasing data sophistication and granularity, it is important to be aware of the key technical components to an MMM model, which are needed to establish reliable model results and have credibility and defensibility of your modeling choices.

A. The process: Exploratory Data Analysis (EDA)

In any marketing mix model build, you first begin with exploratory data analysis — a series of data analysis to understand the data, identify gaps and trends, and implement proper treatment of your data (transformations, imputations) to be used for modeling. Here is a quick guideline to some of the steps in an EDA process, which will allow for more informed modeling decisions along the way.

1. Understanding data gaps (% missings, unavailability of data etc.): to allow for proper imputing of variables (if missing, and not true zeros)
2. Outliers + Major Events analysis: to create 0/1 binary dummy variables to be incorporated in the model to estimate the impact of major events
3. Holiday analysis: To incorporate seasonal dummy variables for estimation of impacts like Christmas, Labor day weekend etc.
4. Data plotting and trend analysis: To understand (visually) how each of your independent variables relates to the main outcome variable, and to each other
5. Correlation analysis: to understand the linear relationship between two variables; this step allows us to see which variables are correlated to the outcome variable. It also allows us to see which variables are highly correlated to each other (and may potentially need to be combined or dropped).
6. Share of spend analysis (by channel, by campaign, direct response vs. brand etc.): an initial understanding of the spend by order of magnitude, to guide the variables you expect to see in the initial model
7. Spend vs. impressions analysis: this step allows you to see instances where the relationship between impressions and spend has shifted (which might have an impact on ROI calculations)
8. Variable importance/ Inclusion probability analysis: is a method to assign a score for each variable or feature based on how useful the variable is at predicting the target response (ex. sales)

By taking the time to do a comprehensive EDA, you will have a better understanding upfront of the variables you expect to see in the models, their relationship to the outcome, and potential model adjustments needed to get reliable results.

B. Modeling components

All models, regardless of methodology should incorporate four components of the modeling equation:

• Outcome: the value being measured by the model (applications, sales, signups, paid subscriptions etc.)
• Base: the outcome value assuming no marketing activity. It includes unobservable events that measure the change brand perception overtime.
• Marketing Drivers: controllable elements that can influence the outcome (Search Engine Marketing, Connected TV, Brand National TV, Radio etc.)
• Controls: non-marketing external elements that go into the base. (ex. Economic impacts)

While there are different approaches to building marketing mix models, there are a few key components that will allow for technical rigor, reliability and credibility of your coefficients:

1. Bayesian Estimation:

• Allows for granular and consistent modeling across cross sections, by incorporating results from higher levels of data hierarchy
• Allows for modeling under low data availability, and modeling new touchpoints.
• Can incorporate results from other studies (benchmarks, incrementality tests, multi touch attribution results etc.) to allow for unified results

2. Nested Equations: Consumers take several pathways to purchase. This allows us to account for direct and indirect pathways to purchase (ex. Google query volume, website visits etc.)

3. Ad-stocks and lags: To capture the carry over impact of marketing as well as the lagged impact of a consumer response

4. Log-Log Multiplicative model form: This step allows us to linearize the equation, incorporate diminishing returns, and incorporate how consumers are exposed to multiple marketing touchpoints that drive synergistically to drive response. Log-log also has a nifty interpretation of a coefficient: a 1% increase in the metric drives an x% increase in the outcome variable

5. Audience factor: If you have a log-log functional form, this step divides your original spend or impressions variables by an audience factor, to avoid unreasonably large returns at low levels of spend

6. Multi-collinearity: Simplistically, when one predictor variable in a regression can be linearly predicted from other variables, we need to incorporate ridge regression to serve as a regularization or shrinking mechanism, to reduce prediction error

7. Fixed and random effects: Fixed effects allows us to handle the time-invariant unobserved heterogeneity across cross sections (ex. baseline level of sales in Idaho is different than New York); Random effects allows us to incorporate the difference in slope (i.e. difference in responsiveness to marketing in Idaho vs. New York).

8. Data resampling: This step ensures that we re-estimate the models with different samples of data to allow for stability and reliability in coefficient estimates

9. Floating base: Unobserved components of a time series, to capture the slow moving drivers of outcome (auto-correlation, underlying trend, unobserved change in brand equity etc.)

10. Model diagnostics: including a holdout sample, to validate and test the predictive power of the model, and avoid over-fitting. Note: good model fit is a necessary but not a sufficient condition for a good MMM model — the coefficients matter!

C. Interpreting the model

While building your model in a reliable way, understanding the interpretation on what decisions should and shouldn’t be made from marketing mix models is an important step.

Let’s start with what Marketing Mix Models DO do:

• They help us measure Average Return On Investment (ROI) of each marketing channel and Marginal ROI (i.e. what does the next dollar bring me)
• They help bring online and offline media on a comparable field, to help optimize budgets across different channels
• They help us to control for exogenous events outside of our control (ex. Economic variables, seasonality, trend, covid impacts) to estimate directional lift from each marketing activity
• They help with ‘what-if’ scenario planning ex. What if I added USD100,000 to my TV budget, what would that get me?

What do Marketing Mix Models NOT do?

• Marketing Mix models are NOT the same as incrementality. While it is an estimation of incrementality (i.e. ceteris paribus, or holding everything else constant, what is the lift from each marketing channel) — true incrementality is done via experiments — i.e. a treatment and control group that is randomly assigned, has equal ‘opportunity to experience’ exogenous and other marketing events, to observe the true incremental lift of a marketing campaign
• Marketing Mix models are unstable under low data availability; While Bayesian techniques can be used to handle low data availability (via higher levels of data hierarchy or aggregation, or prior information from other measurement or beliefs) to ensure stability of results — in general, MMM models are not ideal to answer the question “what was the impact of this 3 week FIFA campaign at driving incremental lift”; as it takes some time to train the model.
• Marketing Mix models are not ideal to make daily granular cohort level decisions on digital media — this is best handled via attribution models in pairing with incrementality tests.

Some key takeaways:

MMM models incorporate deep technical rigor (the methodology matters!) along with art (prior beliefs, expectations, feature engineering) to build models with credible and reliable results

A reliable model fit (R-square, MAPE) is a necessary but not sufficient condition to building a great marketing mix model. The coefficients matter!

A marketing mix model when used in conjunction with other methods (experimentation, causal inference, MTA) is a great way to estimate the average and marginal return on your marketing investments, and make decisions around marketing budget optimization, what-if scenario analysis and forecasting.

Putting it all together

It’s easy to build a model. It’s building a model with in-depth understanding of the nuances in technical approach, modeling choices along the way, and feature engineering and coefficients with business rationale that make the difference between an unusable model and a usable one. Sophisticated models are a necessary but insufficient step in driving revenue impact; stakeholder buy-in absolutely critical to gain cross team alignment, and adoption of your models to drive revenue impact.

*****************************************************************

*****************************************************************

Fractal is one of the most prominent players in the Artificial Intelligence space. Fractal’s mission is to power every human decision in the enterprise and brings AI, engineering, and design to help the world’s most admired Fortune 500 companies. Fractal has consistently been rated as India’s best companies to work for, by The Great Place to Work® Institute, featured as a leader in the Specialized Insights Service Providers Wave™ 2020, Customer Analytics Service Providers Wave™ 2019 by Forrester Research, and recognized as an “Honorable Vendor” in 2020 magic quadrant for data & analytics by Gartner. For more information visit fractal.ai.

*****************************************************************