Open source made MMM cheaper, not easier

Entering the realm of Marketing Mix Modeling (MMM) is now more within reach, yet embarking on this journey still poses its challenges.

Through numerous discussions on MMM adoption, a recurring concern emerged: “We embrace the concept of MMM, but we’re unsure of how to kickstart the process.”

The solution lies in the fact that accessible open-source platforms have significantly reduced the entry barrier. However, they have not diminished the expertise necessary to generate reliable and actionable outcomes.

Open-source MMM has revolutionized the starting point

The adoption of MMM is on the rise. Nearly half (46.9%) of U.S. marketers are planning to increase their investment in MMM over the next year, ranking it as the most dependable measurement methodology (27.6%).

The open-source evolution in MMM is tangible. Three robust libraries now cover the entire methodological spectrum:

• Robyn (Meta, R): Streamlined hyperparameter search through Nevergrad, efficient model selection with Pareto frontier, and inclusive decomposition and response curve plots — offering a user-friendly entry point. It’s my preferred choice due to its high level of customization.
• Meridian (Google, Python/TensorFlow): Bayesian inference incorporating geo-level priors and systematic uncertainty quantification — more rigorous but with a steeper learning curve.
• PyMC-Marketing (PyMC Labs, Python): The most adaptable option, presenting a comprehensive probabilistic model closest to academic-grade Bayesian MMM — demanding a higher level of statistical proficiency.

This new wave of tools has eradicated the previous $150,000-$500,000 consulting barrier that once defined entry into MMM. Any team proficient in R or Python with reasonably clean historical data can now conduct modeling internally.

A crucial point to emphasize in discussions with those exploring MMM is this: “Free tool” does not equate to “free model.” While the software comes at no cost, the expertise needed to configure it accurately — a vital aspect of the process — is not.

A competitive vendor landscape with intriguing power dynamics

The rapid expansion of the SaaS layer built on open-source MMM is noteworthy. It’s essential to differentiate between several tiers:

Data-layer-first vendors

Platforms like Rockerbox and Northbeam initially focused on attribution and data collection before integrating MMM. Their strength lies in data pipelines and speed rather than in-depth modeling or customization.

Measurement-first vendors

Platforms such as Measured, Analytic Partners, Ekimetrics, and Nielsen Gracenote offer more rigorous modeling at a premium price point, featuring enterprise-grade capabilities.

Google Meridian and GA360

One notable aspect is Google’s launch of Meridian as an open-source initiative, a generous yet strategic move. When a closed ecosystem funds and packages the measurement methodology for assessing its own channels, maintaining a healthy skepticism about model priors and default assumptions is advisable.

The pivotal question when evaluating vendors is: who controls your data layer, and does this lead to conflicts in the modeling layer?

Challenge 1: Data access is the unseen obstacle in MMM

This often-overlooked implementation hurdle is seldom given the attention it warrants. A well-defined MMM necessitates:

• Two to three years of weekly data as a foundation — adequate to encompass at least two complete seasonal cycles and a substantial range of spending variations.
• Consistent granularity in channel-level spending — not simply “digital,” but distinctly categorizing search, social, display, and video expenditures.
• Offline channels (TV, OOH, radio, events, direct mail — typically managed by different teams) often utilize incompatible time granularities and are tracked in separate systems.
• External covariates — macroeconomic indicators, competitor actions, pricing details, and product launch schedules.
• For B2B scenarios, extended sales cycles and lower conversion volumes heighten the data demands, necessitating a lengthier historical dataset.

In reality, what frequently impedes MMM projects is the tedious six-week data retrieval process preceding model construction. Revenue falls under Finance’s jurisdiction. TV expenditure is overseen by the brand team. Digital spend is managed by the agency. The lone record of trade promotions is a spreadsheet from 2021.

The efficacy of the model hinges on the quality of data retrieval preceding its development, a facet often omitted in vendor demonstrations.

Challenge 2: Hands-on involvement remains essential

AI assistants have significantly eased the syntax barrier. They can assist in setting up a Robyn run, configuring a Meridian setup, or debugging a PyMC model. Nevertheless, they cannot yet navigate the critical judgment calls that validate a trustworthy MMM:

• Determining the optimal position on a Pareto frontier with numerous model solutions (NRMSE vs. DECOMP.RSSD trade-offs).
• Evaluating whether Nevergrad’s optimizer has effectively converged or is stuck in a local minimum.
• Adjusting adstock transformation parameters (Weibull shape/scale, geometric decay) to mirror realistic channel dynamics.
• Identifying why a model assigns an implausible contribution to a channel and deciding whether to address it with a prior, data correction, or variable exclusion.

In essence, relying solely on automated processes to develop an MMM may yield a seemingly functional yet fundamentally flawed model. Writing the scripts is not the arduous part. The expertise needed to validate the output includes conducting channel-specific incrementality experiments to calibrate your MMM.

Challenge 3: The indispensable human touch in MMM

Even as the tools advance to a stage where AI can execute a competent default MMM, the irreplaceable human element involves embedding business context — aspects that no model can glean from data alone:

• Understanding adstock and carryover nuances: Recognizing that your TV campaign has a four-week carryover, paid search has a three-day carryover, and branded awareness efforts maintain a prolonged decay. Such insights are not evident in the data but reside within channel experts.
• Interpreting saturation curve patterns: Anticipating when a channel nears diminishing returns before the model indicates, and questioning outcomes that diverge from this understanding.
• Establishing guardrails and anomaly management: Addressing factors like COVID disruptions, product launches, pricing alterations, and macro shifts either by explicit modeling or flagging as structural deviations. AI cannot discern a pricing crisis encountered by your client in Q3 2022.
• Validating interpretation: A model suggesting a 40% contribution from a brand spending $2 million on TV may raise concerns, warranting investigation based on intuition rather than computation.
• Organizational translation: A technically precise model loses its value if you cannot articulate why reallocating 15% of the search budget to CTV is advisable in terms understandable to a CMO and CFO.

Establishing the Foundation Before Model Construction

The initial step is to comprehend the data required to fuel the model and identify those essential in contextualizing and translating that data into effective marketing strategies. While neither task is simple or swift, both are imperative for deriving meaningful insights from your model, regardless of opting for an open-source or subscription-based platform.

A pragmatic starting point involves accessing Robyn’s demo script and experimenting with the sample data before applying it to your own dataset.