Marketing mix modeling for marketing leaders

Transcript

Linda Grönlund:
Welcome everyone to the MMM for marketing executives with Supermetrics and Dentsu. Let's meet the speakers. So Jarle, go ahead and introduce yourself.

Jarle Alvheim:
Hi, my name is Jarle Alvheim. I'm head of data technology in dentsu Norway, which is more or less working with data warehousing and all the Supermetrics connectors. And that's me.

Geir Vikane:
Hello, my name is Geir Vikane. I work as head of performance in dentsu Norway. So I have worked with our clients across digital channels and reporting for several years, and now with sales modeling.

Alessandro Cogliati:
And I'm Alessandro, a Product Manager here at Supermetrics, focusing on marketing mix modeling-related products.

Linda Grönlund:
And finally, my name is Linda. I'll be your host and moderator today. I'm a Paid Social Media Manager, also here at Supermetrics. But let's start with some background information. Why are we having this discussion about MMM?
Alessandro, where are we today with the privacy regulations and how it affects marketing?

Alessandro Cogliati:
Yes, sure. In the past years, we have seen institutions around the world introduce a new regulation to protect their own citizens' privacy, limiting their data's assets.

And then we have seen in the mobile business Apple is really focusing now on privacy. And when they introduced iOS 14 a couple of years ago, they really started to limit the hard tracking at the user level.

We also know that Google is following in the market, meaning they focus on privacy for their Android devices. They're now in beta with their solution. The solution is slightly different. They have a slightly different proposal than Apple but still focus on privacy. And we know that also Google is working to bring the same limitation to its web ecosystem—by deprecating third-party cookies in its Chrome browser.

As the picture shows, we are still in the transition period, but the path is clear. We know that those big two giants, Apple and Google, are working in that direction, the privacy direction. So what does it mean? It means that user-level tracking and user-level targeting will be difficult for the targeting audience in the future.

In online marketing, we have been used to tracking users through different channels, and this was important because we were able to track the full conversion path of a single user. Whatever algorithm we were able to apply to the conversion path, whatever has been for attribution, whatever will be a last-touch on the last-click or last-view or more sophisticated multi-touch attribution. All these algorithms rely on the tracking data from the different channels. So this, little by little, will fade away because we know that the big guys are going in that direction.

We also see in the market that some players are trying to keep the system as it's with some workaround with the fingerprinting and these kinds of things to identify the user. We know that Apple and Google are going in that direction, the direction of privacy. So that means that we need new tools. The tools that we have right now need to be more. We need additional tools to keep having effective marketing. And marketing mix modeling, MMM, is a good candidate for this.

So MMM isn't new in the market, it has been used for many years to measure the attribution of offline marketing, basically in offline channels like TV and radio, for example. And now that we know that online marketing is also going to face the same limitations, it's an opportunity to merge the attribution process into one process and combine both online and offline marketing.

But what is MMM? MMM is a top-down approach based on statistical analytics of historical data. And it can predict which factors, including the spend in the different channels, are affecting sales or any other conversion KPIs. And since it relies on historical data, a certain time, a certain period, with MMM, we can figure out the user acquisition saturation of each channel. So we can use MMM as a budget optimization for the future.

I'll give you one example. For example, my advertisement in TikTok isn't performing well this month. What if I move 10% of my budget to Facebook for the next month? With MMM, we're able to predict what will be the outcome of this move. So, what has been privacy, and why is MMM relevant right now? In the market, we've seen different solutions related to MMM, some of the solutions focus on some aspect of the MMM, and others focus on specific use cases. And here we are. One of these solutions is provided by dentsu, and we're here to learn more from dentsu.

Linda Grönlund:
Yeah, so Alessandro just talked about the challenges with digital marketing today. So Geir, I'd like to hear dentsu's perspective. What challenges do you see marketing leaders facing?

Geir Vikane:
That's a really good question because there are several challenges that we and our clients see regarding reporting these days. It's complicated to find a system that gives us both offline and digital channels in one report. We struggle to find objective reporting. And the lifetime of the cookie will, of course, continue to be an issue.

The most important part that we discuss more and more often with our clients is how much of the sales is affected by paid media and what part of sales we think will come anyway? Our solution here is a marketing mix model, but it has to be done in a different way than before. Because historically, you start to do the planning for this, you collect the data, you do the modeling, and maybe when the answer is ready for the client, they have started to do new things in their marketing, and we aren't able to fix the issues that we want to solve.

And Alessandro, you talked about attribution earlier. But since attribution is quite complex, we're going to have another go at it.

So who is better to explain something complex than the guy that makes football look quite easy these days? In a stadium, three systems watch a game and look at the world differently. Google Analytics, see Haaland scoring a goal and attributes the goal to him. Then we also have Facebook watching the same game, but they also see that Kevin De Bruyne makes the pass to Haaland and gives us two conversions. But other channels, here just represented with Snapchat, shows us that Cancelo makes it possible for De Bruyne to make the pass, and they give us three conversions.

So as you can understand, it's a difficult issue for us, and it's hard to know what to tell our clients. It doesn't make it any easier by looking at the rest of the customer journey. So what we've just shown you with the attribution that's just part of the customer journey. But in addition, we have TV, radio, programmatic display, and several other clients that also played a part, and we believe that all channels are attributed in some way to the conversion.

But how much per channel? That's the question that we need the answer. So it's like Pep Guardiola, the Manchester City manager. We need to find which player does the best job in every position on the pitch.

As I mentioned, we must shed some light on another issue the whole industry should be more aware of. How much does paid media affect sales? Here we have explained it with a campaign across Facebook and Snapchat, where we want to make people buy ice cream. In both cases, we have the same investment. And by looking at this, it's quite easy to say that the weather is why we sell more ice cream in the top example. But our biggest issue is that when we report these days, we don't consider other things than the amount we have spent and the contribution from those channels. So what we have to look at, we base our results on all the conversions coming in, but we have to get an understanding of how many sales will come anyway based on different factors like price, product, weather, and so on.

So that's the short-term issues that we have been looking at. But we also have to find a way to see the long-term, and it's possible to solve. And one of the parts that we can affect long-term is the brand with paid media. But we need to find out how much the brand affects the base sales and the brand's share in base sales. And if we can find which brand parameters affect the brand the most, it should also be possible to say something about which channels do the best job long-term.

Linda Grönlund:
Yeah. So Alessandro and Geir, you mentioned how the privacy and attribution challenges make MMM more relevant than ever. Are there any other reasons why marketing leaders should look into it now?

Alessandro Cogliati:
I think automation is the key point here compared to the traditional MMM, what it has been in the past. Traditionally, a lot of work in order to use MMM was done in the past related to how to retrieve the data, how to transform the data, how to merge the data, and how to normalize the data to the different channels so that they can be used as an input for training the machine learning model, the MMM model.

And this is what we're trying to do here in Supermetrics. We're trying to optimize the full process when it comes to the data. And the goal should be these three bullets we present in this slide. One is full automation so that we have a constantly updated model, not a model that is trained once or twice a year. So we have real-time data coming, and we can train the model for the next month's budget allocation or the next week.

The second is validating the model. We are able now, with Supermetrics, to deliver to the different MMM solutions real-time conversion data so that the customer can see what the model has predicted in the coming weeks or the coming month and what is the real data, the real sales, or the other KPIs that are coming in real-time so that you can see, is the model correct? Is the prediction correct? Is the trend of sales in line with the trend of the model? And the latest but really important part is lowering the price. We'll remove the manual work when we formalize and prepare the data and automate most of the process. In that case, we can also lower the price and make MMM more accessible to at least a bigger audience. So not only enterprise or big customers but also middle-sized and smaller companies can have access to MMM solutions.

Linda Grönlund:
Yeah, now we have talked about what MMM is and how it's a better approach to measurement. So Jarle, how are you guys doing MMM at dentsu?

Jarle Alvheim:
I'll talk about the infrastructural part of it. And it's very well illustrated with the iceberg. And both Alessandro and Geir have talked about the manpower and the money that goes into the process.

Historically, and I think this is known for everyone in the room, I guess, who has done any sales modeling in the past, MMM or that kind of stuff, or any of the data-driven projects. Two-thirds of a modeling project goes to the data infrastructure historically, and that's like fetching all the data and doing the normalization Alessandro mentioned.

And it easily took three to six months to do that job when you sent it to a modeling company. And we have been talking about models that are updating regularly, like weeks or monthly; we can't have that kind of process at all.

So I'm going to show you what we've done together with Supermetrics to build a quick overview of the steps we have gone through to do this. And the first one is, of course, using the connectors. We use Supermetrics on all the channels they support, mostly digital ones. In addition, we have to connect to a few other connectors, which are local in Norway, typically local tractors, TV meters, our own ERP system, which has a lot of data, and so on. Of course, it's important for us to use Supermetrics because it's the manpower it takes to build the connectors.

If we build a connector for, for instance, Facebook, we could count on about 50 to 100 hours to do the job ourselves. Plus, there are continuous changes in the API that also takes a lot of time. Using Supermetrics helps us make it so we can keep our organization lean and small and keep the pricing down for the clients.

After pulling the data from the connectors, we store the data in our data warehouse. We have chosen BigQuery for that. And there, Supermetrics has a direct connection to BigQuery. The great thing about using a data warehouse, instead of getting it out on files, is that you have a good possibility to store a lot of historical data, and it also keeps us querying the data quickly and efficiently.

That's also important when I will talk about the automated process in the next step. Because even though you pull the data from the connectors, there's still a lot to do with the data. That's because it usually comes in different formats, has different naming, and may have different granularity, day, week, month, currencies, and so on. And that's why we enrich the data and normalize it through third-party sources to make it smoother. I'll show you an example.

An example is, for instance, costs. If you go to Facebook, it's spend. In other channels, it may be costs. In some channels, we don't have our local currencies, so we have to convert that to Norwegian krone, or everything should have the naming costs when it comes out to the model.

The same thing with a campaign. Campaigns can be named in different ways, such as campaign name, campaigns, or plan name. That's typical issues we have to fix on the way.

When we have done that, our next spot is joining the data, which is also a major step. That's joining the channels we have pulled from the different channels and adding other data on the next slide, for instance, client data, sales, and store visits. Brand trackers can also be control variables that we need to use in the model, for instance, weather, as Geir has mentioned, traffic, sharing voice data from the vertical, and that kind of stuff. And when all this has been done, we finally have something we can use in the model. But not just in the model but the same data is used in different reporting dashboards. We can stream the data over to the clients. So when we've done all these steps, we have a data structure that is fairly flexible and platform agnostic and can be used in more or less any model, any system, and any dashboard solutions that different vendors provide.

Geir Vikane:
Yes, so I have the fun job of summarizing the technical part. We have collected spending in our channels for the last three years. As you can see here, the offline data goes straight BigQuery. And as Jarle talked about, the online data is fetched by Supermetrics and sent to BigQuery.

So from here, we store the data in our data hub and get both sales and brand data from the clients. So this is the magic part that makes us able to keep running the model as often as needed and to be able to use the results when we need them, and we can make the changes that we need to do with paid marketing as we go.

So with all the data ready, the next step is our marketing mix model to see the short and long-term results. Firstly, we see how much of the sales if base sales and how much is incremental. So here we see that 80% is base sales, and incremental is 20%. That means we can affect 20% of the sales short term with paid media.

And the next part is a look at the share of spend that each channel has and compare it to the share each, if we go back, to look at the share each channel has on contribution to sales. In this part of our analysis, we're quite pragmatic and focused on the channels that give us the most bang for the buck, which means a higher share of contribution than the share of spend, and we rank them based on that.

But as you can see, most clients' biggest part of the sales is base sales. We want to see how much of the 80% we can affect the brand. So for this client, it's 33% we can affect by brand, and the other 47% is affected by price, product placement, and other factors. Let's take it one step further and see how we can affect the 33% through paid media. So the last step is to split the brand into the different brand parameters on which we have run the model. In this case, we see that ad recall affects the brand the most on each of the parameters. We get information on which channels have a strong medium or low to non-effects.

So for this client, we have picked the channels that are strong on ad recall, and we see that TV, radio, and online video do the best job. So what we now have seen here to take this, we compare what we have done both short and long-term, get it together, and get a mapping on how we should or can create the most effect for the clients in the long and short term.

Linda Grönlund:
So what channels do you typically see working well in the long term?

Geir Vikane:
That's a very good question, but it's many different answers. So the important part here is that the model will look different from client to client. So on the models that we have run, we see that radio, TV, and out-of-home can be strong long-term channels, and channels like paid search, Facebook, and Snapchat are strong short-term.

But the fun thing is that we always get some surprises. By setting it up in a visualization like this, it's a nice tool for the clients to use, both long and short-term. But the most important part is to find which channels don't have any short-term or long-term effects that we can stop spending money on and rather put the money where we see the best effects.

Linda Grönlund:
Thank you, that's interesting. Are there any limitations or considerations that companies should be aware of?

Alessandro Cogliati:
Well, we're still in the transition period, but we're seeing big players in the market like Facebook, Google. It's not a secret that they're investing a lot in MMM. And MMM is starting to be more and more relevant for all the limitations in the user-level tracking that we discussed at the beginning of this webinar. So we don't see restrictions in terms of industry.

We have seen different industries successfully using MMM, like, games, ecommerce, and SaaS services. So there are no limitations to that. Of course, the best value out of MMM comes if you have a complex marketing setup. So if you're a company that you're working worldwide, you have marketing in all different countries, and you're using online both offline and offline channels, that's where your MMM is getting the best value out because of how to get all the variables in and how to optimize all those variables.

But we have also seen smaller companies working in one country. But already there, when you differentiate your spending in the different channels, you can optimize the budget with MMM. So we haven't seen big limitations. But of course, as I say, we're in a transition period, and we're ready to see the winning solution in the market. What do you think, Geir? What have you seen? Are there additional things you want to add based on your experience with direct customers?

Geir Vikane:
I think what you said there is really important when you work across several countries. We have some clients that work in more than one country, and then there are big differences in how the channels work in every country. So that has been very interesting to look at. Of course, we know a lot of the channels, but they work differently across different countries. That's a good insight we have gotten by running these models.

Alessandro Cogliati:
What about the creative of the campaigns? Can you measure the quality of a campaign with MMM, or is that a bit too granular, and that is something you're checking with different tools? What is your experience?

Geir Vikane:
That's also a very, very important part. Because what this tells us is which channels are the most effective. But an additional thing, as you correctly point out, an additional thing is that we need answers for the creatives we use. We can see a lot better results if we have really good creatives. And if the creatives are not that good, we might not see good results historically. So that's an important part that we should discuss further with the creative gang. But that isn't something we can answer by doing this.

Alessandro Cogliati:
From what we're seeing in the market and speaking with different players, everybody thinks that MMM will be a useful tool to be used in combination with our existing tool. And there will be no real difference — it will be an integrated tool in any reporting tool. So you'll have their unbiased solutions to tell the customer where to allocate the budget and what incremental attribution you can give to the different channels.

It would be a third-party, unbiased solution. I'll combine what you already use for marketing performance, campaign levels, and these. So we'll achieve that in the long-term with the MMM. Do you have anything to add, or can we go to the question?

Jarle Alvheim:
Yeah, that was quite a lot of questions here. One is asking about, let's see, I lost the question. There was a question regarding how much you must spend and how many channels you need to use to do this modeling. The important thing is that we have control of all the marketing you do. That's the important thing.

So if it's only digital channels or a lot of offline channels, we need to have control of the marketing budget and all the data we can get around that. In addition, of course, the model needs a lot of control variables that are special for the type of client you're.

And so I'd say, MMM, with a decent amount of spending, it should be possible to make a model for your type of industry as well. It depends, of course. Even though it's a small country, a large country, that's not the biggest difference. The important thing is that there can be several models, and not everything is fit for your company. But that's part of what we must do when doing that kind of process.

Alessandro Cogliati:
What about the pricing? There is a question about the pricing. What is your experience?

Jarle Alvheim:
Yeah, the pricing, of course, is different. It's difficult to answer; it also depends on how much data and how many channels we have to work with. So it's mostly the manpower we're talking about. It's not a lot of tech fees. That's usually just a baseline and recurring monthly fee to keep them all running. But our experience is fairly cheaper than doing traditional sales modeling, which many of you're used to.

Alessandro Cogliati:
Yeah, what we are seeing in the market is really like the emergence of this. As I say, automation will lower the price. Of course, if you go, there will be different sales points, different packages. But if you want a super granular and customized solution for your brand, the price will probably remain. But if you go down a bit in the spectrum and you'll want to use MMM or standardized use case where we can automate the full input from the data perspective, then we can lower the price. That's what we have seen in the market.

Jarle Alvheim:
And of course, one factor also playing in here is we need data from the client, some sales data, or other data. And how easily is that automated, which is also usually a big X factor in modeling?

Alessandro Cogliati:
Yes. Any other questions? There are questions about the algorithm. I'll not go so deep. But they're pretty standard. So MMM is flexible. You can really tune it, and you'll customize it according to your needs. The general algorithm is pretty standard—Bayesian linear regression, or some you can use a little bit more sophisticated. But it's different from the algorithm that you're using.

It's more like how you customize how you train the model according to your requirements. We have also seen different ways in the market how people are calibrating the model. There are techniques used in the model itself to calibrate, and we have seen also customers that are using uplift empirical testing to see the test results are used as a calibration, as an input for calibration of the market. So there are different setups, and it's just starting to do what is best for you.

Jarle Alvheim:
It's never a one-stop solution. There always has to be some customization for the client during the model. But there may be a one-stop solution to getting the data fetched out for the client because that's much easier today than before. Another question that comes here is about data and history.

For instance, due to the pandemic, how can we compare pandemic years compared to normal years? The question is, will there be a normal year after everyone has started using web shops much more since the pandemic? But that's, of course, an issue and also an important control variable in many cases. We have to consider the time shops and stores were closed, online sales, and general stuff happening in the market. We see shifts in what people bought in different times of the pandemic. So we have to take that into account.

In addition, to do good sales modeling, you need at least three years of data. Everything more is great. Do you have less? It's not so great. And in many cases, you'll end up in a situation where we have to. Let's say you're a fairly new client, and we have not done any data fetching before. Many channels have a limitation of two years maximum to pull data from. So if we start pulling the data today, you're not ready for any modeling before next year at the same time. So both historic data are important, and it's great if you have something before the pandemic as well. But we also take the pandemic into account when we do the model.

Linda Grönlund:
There was also one question, would MMM work with a seasonal sales cycle, or would you need to do a longer sampling of data sets, like an entire year's worth of data?

Jarle Alvheim:
That is also a control variable. So seasonal sales also have to be considered if that's important for your industry. As with the pandemic, we have to put those kinds of numbers in and look into them. For instance, Black Friday, Christmas sales, January sales, and that kind of sales have to be taken into account as part of the model.

Alessandro Cogliati:
There was a question about if you're a small ecommerce company that's allocating budgets in a few marketing channels, how can they test MMM? Yes, you can start. You can contact us. There was another question about how can we start. Well, you can contact Supermetrics or dentsu at the end of this webinar, and we can give all the instructions on how to proceed.

Linda Grönlund:
Okay, we're coming up on time, so no worries if we didn't answer your question. We'll contact you by email. And thank you, guys, for coming to our webinar, and hope you learned something about MMM. Hope to see you next time as well.