Multi-touch attribution (MTA) vs marketing mix modeling (MMM)

Transcript

Riku:

Hello everyone and welcome to another supermetrics webinar. My name is KO McKennan and I'm the marketing STE here at Supermetrics. And today I joined by Mark Stu, the CEO of proof analytics, and Evan Kading, our senior sales engineer and product evangelist here from Supermetrics. And the topic today is Multitouch attribution versus marketing mix modeling. And without any further ado, let's Evan take it

Evan Kaeding:

Away. All right, excellent. Thanks, Rico. Hey everyone. Like I mentioned, my name's Evan. I'm going to talk just briefly about some of the high level concepts associated with an MMM associated with MTA based on the poll responses. It sounds like we have a pretty rough amount of familiarity with these technologies. We're not a crowd of experts on either one necessarily, but there's always things that we can learn and take away from these two technologies. So I'll go ahead and dive in format of the presentation. It's going to be fairly casual. We want to keep this pretty open and keep it pretty casual with Mark and I here. So I'll go through a few slides just kind of talking at a high level what these technologies are about. We'll then have a really just kind of a conversation, mark and I about some of our experiences and some of the experiences that we've seen with clients in coming forward with some of these technologies and how to use them to address what's happening in the landscape.

So to start off and to recap as to what the definition of an MTA is. So MTA stands for multi-touch attribution, and it's really the process by which you ascribe the value associated with individual touchpoint toward the overall conversions associated with those customer journey touchpoints. So here on the right you can see we have a few different attribution models where we can ascribe certain weights to the individual touch points the customers have had. It's important to note that with MTA, it relies on mainly two things. It relies on the ability to capture and track many different points within the customer journey. It also relies on the ability to accurately ascribe or basically determine what kind of model you want to use for associating the value of those conversions or associating the value of those touchpoints with those conversions. So in this case, we're gathering data, we're using it to try to measure what is happening and then using some inference that we might think or understand about the customer journey to try to ascribe certain value to certain touchpoints in the overall customer journey here.

Now, in contrast, marketing mix modeling is really statistical analysis, right? In this case, we're really trying to use statistics to infer what the effect of our marketing is on the different channels through which we're advertising, right? And traditionally it can be taking things into account like volume of impressions, volume spend, those kinds of things as our independent variables with the dependent variable being revenue. We put dollars in, we get revenue out, we try to understand which dollars are associated with revenue. And so in this case, we're trying to use statistical analysis to observe and to understand the relationship between all of these different channels. So it's different than an MTA in the sense that we're really more concerned about measuring inputs than measuring inputs in response to the outputs rather than trying to capture and ascribe certain value to individual customer touchpoints. So now that we've got the overview of those two technologies underway, what I want to just talk about are some of the conditions in the market that we're seeing.

So what we've seen and what our CEO Mac Thunberg has mentioned on previous recordings and previous broadcasts is that the landscape is getting more challenging for digital marketers, and this can be ascribed to a number of different factors. So we've had the inability to pixel first party media for a while now, and many of the social platforms, many of the wall gardens. This is limiting our ability to track customer touchpoints across the customer journey in a variety of different ways. We also have, of course, the recent updates with iOS 14 browser limitations associated with other kinds of cookies. Those things are going to continue to happen in addition to a variety of other legal and legislative conditions that are changing and causing significant headwinds toward this trackability. Another piece that is important to mention is that we're seeing a high amount of bot traffic in the market.

Studies suggest that up to 70% of traffic, two websites, two different applications can be comprised of bots in a variety of different ways. This can throw off different kinds of technologies that are really relying on that user level or that tracking level data in order to understand those pieces of the customer journey. So it really takes a really robust mechanism to try to filter out that bot traffic. Then the last piece that we see and we are seeing here on the market on the super metric side is that we're seeing an increased emphasis on brand oriented marketing. And so we see a variety of different companies going to market with different value propositions, and we're seeing that those that are investing in brand oriented messaging are making the difference for their customers, right? They're delighting them in uninspected ways and they're really going above and beyond to make sure that their brand is memorable and recognized among their parties.

And that seems to be a winning strategy. So among these different conditions in the market, how best should we respond to them? How can we mitigate some of these headwinds that are making marketing challenging? And so we have three things that we're going to discuss here. First of all, what we recommend is evaluating the viability of your current marketing measurement strategy when all of these different headwinds are taken into account. So this really just means understanding what's happening and taking a hard look at your measurement practices as they are impacted by these different changes. So we've seen over the last five years, over the last decade, things are definitely not the same. If your marketing measurement strategy is not changing with the times, it's certainly time to evaluate its viability in light of these significant edwins. Another piece of the puzzle is familiarizing yourself with these different phenomena, right?

So understanding how modeled conversions work. Now we're seeing Facebook launching modeled conversions for a variety of different platforms. We're seeing this in Google, even in Google Analytics. They're starting to introduce model conversions and other responses to the signal loss phenomenon, right? Advertising networks are certainly doing the best they can to mitigate these headwinds associated with both the legislative and the technological pieces. And so understanding the different technologies and different ways to really respect user privacy, but also to understand what's happening with the signal loss and why some of those traditional methods are not going to be as effective as they used to be is going to be important to succeeding in the marketing landscape today. And then finally, understanding some of the main benefits and limitations, of course, of the MMM and MTA solutions that exist in the market today. So with that in mind, I'll go ahead and stop sharing my screen here. And so Mark and I here have, I'd say a fairly loose agenda in terms of what we're looking to discuss. Like I mentioned, we're going to keep the conversation pretty high level. Before we dive into that, Riku has another set of questions that he'd like to pose to the audience here. So Rico, I'll let you take that away.

Riku:

Yeah. So currently we'd like to ask you the question that are you currently using MTA or MMA in your organization? So really interesting to see that. And we're getting good amount of results. So around 39%, 38% are only using MTA and 37 are currently using neither, and then a little over 10% or either using MMM or both. So thank you for that. And Evan and Mark the Floris once again, yours.

Evan Kaeding:

Alright, excellent. Thanks Rico. Interesting results there. So it sounds like have a fairly mixed group, but kind of leaning toward MTA is kind of what I understand. Mark. One of the questions that we get a lot here at Supermetrics, and when customers start to ingest their marketing data, they start to clean it up, they start to use it to evaluate insights. Maybe they have MM aspirations, maybe they're just looking to understand what is kind of all the hype about with MMMA question that often we get asked is what size of company does it start to make sense for with MMM? Is it in relation to revenue, in relation to advertising spend? How do you and your clients go about evaluating the size of a company relative to how MMM is going to work for them in their business?

Mark Stouse:

So I think there's a number of ways to kind of think about this, right? I mean, you can certainly define it in terms of size and say that, hey, this is more typically some MMM would be seen in the upper part of the mid range of businesses all the way up into the top, top part of the enterprise. But also I would say that that's a little bit of a distortion right there, just because within that whole span of businesses, there are smaller businesses that are actually extremely aware and mature around data and the collection of data, they really care about it, they have it organized, it's part of their culture, and then you have some really giant Fortune 25 type companies that really struggle with it and that aren't anywhere near as mature. And so I think you have to look at that, and then I think you have to look at risk. What is the risk levels and what kind of risk is being presented in these different companies for making a bad decision? And so if the consequences, either in terms of cashflow or in terms of earnings per share or whatever it is, if the consequences of a bad answer and a bad action, meaning an unproductive action is high, then they're going to be more inclined to go this way.

Evan Kaeding:

Understood. So essentially what you're saying is weighting the investment of the MMM versus the potential risk of the outcome, whether it's a flat outcome or potentially a negative outcome. Is that what you're saying?

Mark Stouse:

Yeah, and I would say it's not just about weighing MMM, it's your whole data strategy, your whole analytic strategy. That's really the key. It's all one piece, right? You can't really, so for example, if you think about MTA or MMM, you're talking about two approaches that hinge on data quality and availability. It's maybe not the best analogy, but if these two analytical approaches are like a gun and you don't have the ammunition for the gun, meaning the data, you've just got a expensive paperweight, that's really what you got. So it really is of a piece, right? You've got to think of it in those terms.

Evan Kaeding:

Yeah, certainly. And we certainly see customers across the range, SMB enterprise, mid-market, like you mentioned, the data maturity is kind of all over the place. Have you seen customers, and I'd love to hear your experience on this when you're starting to bring customers into an MMM solution or an MTA solution for that matter, how do you see that the best customers have built a culture around data? I mean, what is it that has enabled some customers to be successful with data and others not to be? Because that emphasis in quality and clean data is obviously very important for the success of either method in this case.

Mark Stouse:

Yeah, I would frame the answer to that question around a quote that I read recently by a guy named Bill SCH Marzo, who is Dell's chief evangelist around data and analytics, and he's kind of known as the dean of big data. And his whole point is that the economies of learning dwarf the economies of scale. And so where that is relevant here is that if your culture prizes learning, which means that it also understands that in order to learn you have to sometimes get it wrong. That is a key thing because what we see is in cultures where people are measuring in order to defend themselves and to always be seen as, right, that's a very stale culture when it comes to this kind of thing. You have to really want to know the truth. And the truth is not a static thing. The truth moves around a lot, particularly given a lot of the externalities. And your lead in is a perfect example of this. So five years ago, people did not feel that there was sufficient problems with MTA to not use it. Today, all kinds of things have changed in that space that have compromised the average MTA implementation considerably. That is a headwind, that's something new. So life changes, everything moves around, and you got to be able to move with it. So that would be my answer.

Evan Kaeding:

Yeah, certainly. No, it makes a lot of sense. We find that the organizational maturity around data is super important. I'm glad you emphasized that. It really has to be part of the culture in terms of seeking truth potentially, rather than seeking to defend one's particular viewpoint or defend the use of one particular technology as well. So that's certainly an important viewpoint. Obviously, mark, you're the CEO of a business that runs an MMM solution. I'd love to understand how you help your customers and clients think about their marketing spend in the context of MMM versus MTA. Because I think as a marketer, when you go into these two solutions, you have this idea of being somewhat deterministic, right? Whereas an MM MMTA can come in, it tracks every customer move, it knows all of the customer touch points, and we're going in and ascribing particular value to those. It's somewhat of a mindset shift, right? When we move into the world of MMM, can you explain, and maybe that's an assumption that I'm making, but curious to hear your response to that and how you felt customers overcome that?

Mark Stouse:

Yeah, that's actually very, very accurate, but it is really the shift between measurement and analytics. It's about, so measurement is the collection of data, and by definition that means it's always about the past. It's never about the future. Analytics is what takes all kinds of data and finds the relationships that exist or don't exist between these different things. And you don't have to guess the weighting, you don't get to put the weighting on it as you do with MTA. The math tells you what the weighting is. And so you're basically also looking at a projection, a prediction into the future. And that is a whole nother thing. That's something that MTA nor any other kind of measurement does for you. And this is particularly relevant. And so this is where, again, we're coming back to rapid and volatile change. The events of the past, say two years have really thrown gasoline on this whole question because 2020, because of covid, because of other things people saw a lot of disruption and past is no longer prologue and it probably will not be stable or a steady state anytime soon.

And so being able to identify these changes in enough time to react correctly before those changes overtake you, and essentially you're moving from a map, which would be MTA to A GPS, which is MMM, and particularly when MMM became automated, it really did operate exactly like it does operate exactly like A GPS because you're not only seeing the historical causality and the prediction going forward, but you're able to check the accuracy of that prediction against what's happening right now and say, okay, you know what? We're either on track or we're not on track, and this is what has changed and it's shoving us off course, and so we need to make the following adjustments to get back on course. Now you're talking about something that's very, very dynamic and very responsive, whereas MTA for all of its great qualities is not.

Evan Kaeding:

I think that's an important point to point out is that the technology around MMM has evolved considerably since its origins. I mean, I know for example, in my agency background, I can tell horror stories about the annual semi-annual big MMM data dumps where our client asks for, okay, we're running the MMM, export all of your data from all of your ad platforms, categorize it, tag it, ship it to us at the end of the month, and we had people quitting left and right during that exercise. Can you tell me a little bit more about how automation has changed the game for MMM and why it may not be such a periodic and specific technology these days?

Mark Stouse:

Yeah, absolutely. I mean, because it's huge. I mean, it's like automation for a lot of things. It really, really changes the game, particularly in terms of what is called latency, which is the speed with which things can refresh. So as an example, right now we are talking to a very large technology company who has, actually, it's in the B2B space. So this is not a consumer packaged goods company, and they have been an mm M user. They are an MMM user right now. It is costing them about two and a half million dollars a year to do it. So this is indicative of the fact that historically this has been a very expensive proposition that only gets them one model, one giant model that is only able to be refreshed every six months. And the latency on the delivery of the results is such that right now, I dunno, two weeks ago they took delivery of the results for the back half of 2020.

So we're talking, we're in the back half of 2021 right now. So this is 100 for them, it is 100% historical. They're not able to control their future with this at all. It's not about bad math or a bad model. The company they're working with is an outstanding firm. It is about the ability to speed it up dramatically and drop the cost dramatically so that you don't just have one model, you have many models and you can scale it to any market that you need to look at as opposed to, in this case, they're only applying this to giant model to two countries and they're in 70 80 markets. So this is an exemplar of the change that has happened with automation. And so with proof, for example, the recap, if you're measuring, if you're introducing data into these models on a daily basis or a weekly basis or a monthly basis, it's going to automatically recalculate those models at that frequency.

So if you're measuring daily, and actually most data is collected daily, even if you're not aware of that in your company, it is most likely collected daily. So you're getting a daily update on where you stand. And this is where actually, this is the GPS comparison. As you think about using A GPS on your phone, it's keeping track of your progress around across this route to wherever you're going. And it's keeping track of key external factors that might either slow down or speed you up on that journey, and then it's making a recommendation to reroute if that's necessary. That is MMM in a nutshell right there. That's exactly what it does in an automated format.

Evan Kaeding:

And a moderate MMM implementation has a variety of automated pipelines essentially. So there can be a substantial amount of data cleaning that is involved, but the tools in today's ecosystem exist to automate a pretty significant amount of that.

Mark Stouse:

And that's the amazing part about Supermetrics and about the partnership that we have with you guys. So Supermetrics is not just a giant aggregator of your data and funneling it into a tool like proof. There's auto cleanse and auto harmonize capability in Supermetrics. That is an absolute game changer, again, in terms of speed and latency. So any data scientists will tell you that the process of cleansing and harmonizing data is like 90% of their job. And it also really, pardon my French, it sucks. If you talk to a data scientist, they're going to tell you that's the least favorite part of their job. And so the fact that Supermetrics has automated this is a huge part of the fact that we're able to operate MMM at really low latencies.

Evan Kaeding:

Of course. Now you mentioned when you were describing the latencies, now that we have the ability to automate a lot of the data ingestion, a lot of the data preparation, the cleaning, it kind of unlocks some new possibilities where not only can we have more than just a single model, but we can have multiple models that are potentially chained together that have aggregate effects, but we can also have models that have maybe different inputs as well. Maybe we're talking about inputs like customer satisfaction and how that influences, I dunno, something like brand awareness. So curious to hear your perspective on alternative model types that you've seen where maybe we're not just measuring advertising spend or we're not just looking at advertising spend at how it impacts revenue. Maybe we're looking at some other independent variables and other dependent variables in that.

Mark Stouse:

The other part of it, it's just so exciting if you are a marketer, and for that matter, if you're a business leader, so based on the analytics, I can tell you that the last part of a company that should have any question about its ROI is marketing. So the reason for that is that marketing spend is a business multiplier, business performance multiplier, and not just in sales. The same dollars that are being spent to help salespeople sell more product to more customers faster and more profitably than sales could do by itself is also helping recruiting and retention.

It spreads out all over the company. And some companies obviously have unique use cases, but there are some really broad statements that apply to all of them. And so this is where it gets really cool. So we have a customer right now that is talking through a major project idea that they have. They've been doing this for a while. I mean sometimes it takes big companies a while to move, but it's really cool. So they have instrumented CX customer experience probably 10 or 20 different ways, and then they have instrumented all of their customer facing functions. So this is not just marketing and sales be services, support, contracting, accounting, product, all this kind of stuff. And they've instrumented that across multiple KPIs and they want to use proof to begin to see what are the relative contributions that all these different pieces supply to overall cx and how does that shift over time, particularly given the fact that each one of these pieces not only intersects with the customer experience at different points, but also there's a time lag effect that's part of this whole thing.

And that I'd be remiss in saying is also a huge element of MM. So time lag is the simple concept that says, I spent the money today and it's not going to have an impact tomorrow. It's going to take a while. And everything that I spend in every channel that I spend it in, it's all going to have a different time lag out and a different, what I mean by that is different arc to value. And so if you don't know what that arc is and you don't know when that culminates and delivers the maximum value back to you, and I'm talking about in the calendar itself, you will never find it. You'll never know what your ROI is completely without understanding time lag. So what happens with a lot of marketing teams right now is that in an effort to cope with this, they're shortening their time horizons and they're measuring ROI on maybe one quarter or two quarters because that's what they can see. But the problem is, is that they are actually denying or walking away from 90% of their total ROI. So when they do that, so I mean it's just a major loss when you don't account for time lag. Actually, you could say that the number one enemy of marketing ROI is time lag, and particularly on the brand side, all brand investments take a lot longer than anyone thinks to materialize, but they also hang around a lot longer.

If you want to think of it in those terms, is a lot longer than anything. On the demand side.

Edward Ford:

This show is brought to you by supermetrics, the marketing data hub that helps you connect, transform, and export your data to any destination over 750,000 marketers worldwide. Use supermetrics to turn raw marketing data into business growth. Visit supermetrics.com to find out how.

Evan Kaeding:

Sure. And what you're saying is that one of the only ways we can understand that is using statistical inference, right? Well, all of these things are going to be well outside of any kind of first party or even third party attribution window essentially that the week.

Mark Stouse:

Yeah, absolutely. I mean, and guys, it is probably really super important to make this statement, right? MMM is the application of multi-variable regression analytics to marketing and the business equation. It is the same math that's being used to explore climate change, to look at epidemiology and the pandemics that we've recently experienced experience, most data scientists would say that multi-variable linear regression is the chosen approach to understanding 80, 85% of the world's questions. It undergirds the scientific method of inquiry. Totally, right? It's not that there's not more to it than that, but that's a huge chunk. That's probably 90% of the scientific method. So this is not speculative at all that the math here is massively proven. And as far as proof is concerned, we don't introduce any new math. We haven't innovated in the math at all. We automate the best regression math that's available.

Evan Kaeding:

Certainly, certainly. Well, and essentially what you're saying is we can take a lot of this data, we can take this scientific method, and essentially with MMM, if we can have different inputs and different outputs, essentially we're just building a big experimentation machine for your business.

Mark Stouse:

Yeah, absolutely. Totally. So circling back from that to the intersection of that culture, one way that we qualify prospects is we ask them a simple question, do you believe in the science behind climate change? If they don't, they're very, very, very unlikely to believe the output of MMM, right? It's the same thing. So what we're really talking about is a person or a culture open to having their beliefs challenged by empirical evidence. And so we use climate change as a proxy of sorts for understanding where that culture is. And if they say, yeah, absolutely 110%, I think the science is totally solid, then that's a pretty strong indicator. But if they say, ah, I don't know, I don't really believe in that stuff too much or whatever, it's probably not a hot prospect for MML.

Evan Kaeding:

Sure, no, I can understand that. And reluctance to look at evidence-based measurement or estimation can certainly be a red flag in that regard. One question that I want to throw your way, and this is oftentimes a critique of MMM would be that we are using historical data to try to predict things about the future. And as we talked about, especially in recent history, it's been very tough to use historical data as we've seen sales plummet and then go back up with the opening, the closing, the reopening of economies and all kinds of things. And one might argue that MMM may be a better suited tool for that task, given that it is a measurement tool and it's responding to things closer to real time, where at least it's not drawing perhaps on as much historical reliability as an MMM might. What would you say in light of that critique?

Mark Stouse:

Well, I think that, again, it comes back to the issue of what is past and what is future. So all measurement, not just MTA is about the past. It could be the recent past, but it's still the past and you can't make, particularly in a highly volatile situation, you can't make any projections in your head about what is going to happen based on the data alone, right? It's just not there. You want to talk about a high risk decision making process do that. One of the historical limitations, again of MMM done the old way, the human powered way, was this high latency between recals. So you calculate the model and then you have to wait six months to recalculate the model. And all kinds of stuff can change during that period of time. And so the validity of that initial calculation ages out. It becomes less and less and less reliable and relevant the closer you get to the new recalculation and you just don't know. So this has actually been a, where automation has made just a giant change in the MMM space, and that is the fact that you can catch the differences, catch the changes very, very quickly in a dynamic, highly relational way as opposed to just a bunch of data points somewhere that is key. I, so I grew up in a sailing family sailed a lot. We're talking about ocean racing and stuff like that. I was trained from the time I was about 10 to be the navigator.

One of the things that, and I'm old enough to where I saw a lot of the technology changes around navigation and boats really come online and change things. So I grew up having to, in my earliest days as navigators, shoot the sun with the sexton, if you've ever seen pictures where guys looking through thing like this and he's making a calculation based on the position of the sun and says, okay, that and my chronometer, my watch tells me where I am. The problem was is that if you had bad weather or just clouds or whatever, sometimes you couldn't see the sun for days. And so this is an example of very, very delayed or high latency recal on the navigation. And so what you would find is you'd have a plumb line on your chart that says, Hey, this is where I am, this is where I want to go.

And then because of this multi-day and sometimes longer time lag between recals, the actual position of your boat would get pushed around a lot by the wind and the currents and all this kind of stuff. And so it was this giant zigzag back and forth across the plumb line, and that's the way it had to work. And then all of a sudden we got autopilot. And so that allowed us to steer more precisely. And then we got sat now, and that was the ability to know instantly where we were. Think about it as a G Ps proto GPS, and then the two got linked together where the information from the satin nav with the GPS was actually governing the autopilot, and all of a sudden the zigzag shrunk dramatically, and the error factor on either side of the plumb line on the chart got to be very, very low.

Now what does that mean? Why is that important? Well, if you're racing right, you've just eliminated a ton and wasted time, so you're getting to your destination a lot faster. You're also always on a boat anyway, out in the ocean. You're worried about food, you're worried about fuel and water and all this kind of stuff. And the longer it takes, the more waste there is in your time schedule, the more things get a little bit dicey sometimes. And so it became an exponentially more safe thing when you had this kind of technology. Today, it's the same thing with MML. If you're able to understand the changes, not just what you're doing, but what is swirling around you and causing the efficacy and impact of what you're doing to change. If you can do that, then the world is your oyster, right? I mean, it's really about being able to understand what's coming at you fast enough to make a change that works. And that's the doubt, that's the

Evan Kaeding:

Building essentially building an information feedback loop that can help inform decision making and the tactics you used in your business across a variety of different disciplines as you've talked about. Yeah, very, very good analogy. I like the sailing analogy. I think I'll stick with that one going forward. Last question before we move on to audience questions. You've obviously brought a lot of customers on board and up to speed with MMM technologies, perhaps with little experience, perhaps with a lot of experience. I'd love to understand what you see for customers as the largest barrier associated with bringing on an MMM technology. Is it difficulty in understanding how the models work, difficulty in accepting evidence-based measurement or evidence-based inference from statistics? What do you see as the biggest challenges for organizations that have brought MMM on or have tried to bring MMM on?

Mark Stouse:

Yeah, I mean, I think it's really important to say that not everybody is successful and there's some key reasons why they're not successful, and it's as true for the automated side as the legacy non-automated side. So this is not about the math and it's not about people's ability to understand the math because they're typically, marketers are not doing that anyway. This is really about two things. Do I really understand the questions that I need answers to? Sometimes the best and shortest way to get those questions is to go talk to your business leaders and say, what have you always wanted to know about marketing's impact that you have never really got you feel like you've never gotten an answer to? You're going to find that you'll probably have a list of 20 to 25 questions within about 30 minutes. It is pretty top of mind for a lot of those guys.

And then the other part is, is there the organizational will to provision the data, to bring the data together, to measure stuff, to share data, all this kind of stuff is really important. And there is a relationship between these two things. So there's very much a numerator denominator relationship between the data that you have that would be the numerator and the questions that you want to answer with that data. That would be the denominator. And so the temptation that people have is to start at the bottom. This is what we're doing and this is what we're measuring, and we're going to try and climb this ladder all the way up to value and ROI and all these better decisions. Actually, it's the reverse and the scientific method would totally bear this out as well. You have to start with a question. You have a hypothesis about what the answer to that question is. That hypothesis would be, Hey, we're doing what we're doing. We're investing in the marketing channels that we're investing in because we believe it's having an effect on X, right? So that's your hypothesis to that question. The hypothesis generates a model, and a lot of that is automated as well within proof. That model then says, okay, I need the following data types in order to arm myself arm the model.

And once that happens and you hit calculate, it starts happening in perpetuity after that, or in the case of the old way of doing MMM, there's this high latency that happens. So that is really the issue. If you can't get the data, which today is honestly, no one can't get the data, the data is all over the place and you don't actually need big data to run M-M-M-M-M is actually a lean data or a small data type application. So it's very, very approachable from that standpoint. We also live in the golden age right now of being able to buy the data that you need, even historically with a credit card. So even if you haven't been collecting a certain type of data, you can usually find somebody who has collected it for you and who is more than willing to sell it to you as a subscription.

And then you can put it through Supermetrics and get it auto cleansed, auto harmonize, put it straight into proof, and you're in business. So it's really data availability and knowing where you want to go, meaning, again, using the navigation analogy of where you want go means, Hey, these are the questions. This is what I want to know. This is what I'm trying to achieve. If you add those two points, you will be successful. With MMM, it's a lock, right? If you don't have one of those two, you will not be successful with MMM or anything like it.

Evan Kaeding:

Surely. Yeah. Thank you for the response there. I think very insightful to make sure that the structures are there for collecting the data, but also just a rigorous application of the scientific method is ultimately what it takes, and that can have some organizational barriers for sure. Super. Well, that's most of what I wanted to talk about, but I'm sure we have a lot of audience members who have some questions as well. Rico, do we have any fun questions that we can tackle? Well,

Riku:

I'm happy to say that there are plenty of questions today. We might not be able to answer all of them today, but we will follow up on those questions later. So first of all, we have a question about how much historical data is good enough for building a working marketing mix model. This asker has basically one year of data. Is that enough to build a model?

Mark Stouse:

So this is all about, it's not really about whether it's a year or two year, six months. It's about if you are measuring something daily and you have six to 12 months worth of daily data, then you've got enough In most applications. If you are only collecting data monthly, which means in a year you only have 12 data sets, you've got a problem, you're going to need more time, you're going to probably need three years worth of data, maybe even four years of data. The good news here for most situations is that almost everything is measured daily. So if you think about this for a second, you and your company might be used to getting a monthly report out on a particular KPI, but that was not just measured one time on the 31st of the month, and then they give it to you on the second day of the next month. That's not the way that works. At least 99% of the time, that's not the way it works. They're collecting data far more frequently, and that report out is a consolidation. So that means that the daily data or the more frequent data exists, and if you can get a hold of that, you just won't have an issue there.

Riku:

Great, thank you. Mark, we have another question. How would you go about selling the idea of moving from MTA to MMM to the leadership team?

Mark Stouse:

Okay, so this is a great question. And so there's no delicate way to go about Asbury this question. So I think that you're going to find that most c-suites are not terribly impressed with MTA. They are aware of the statistical problems with MTA, they're aware of the fact that you cannot optimize spend based only on data. They're aware of all the holes, and they read the newspapers and the websites just as much as any marketers. So they're aware of iOS 14 and the Google restrictions and the privacy restrictions, and the fact that the trend line on all this stuff is not positive for marketers. So what you're already seeing is that people are saying, wow, you know what? There's still some really good customer journey data here that we can still collect. We're obviously having to move more and more to first party type data and all that kind of stuff, but it's still potentially good data, particularly as long as we can exclude all the bot traffic.

That's a huge issue right now and has been a huge issue with MTA, so they're kind of casting about right now for an alternative. And I think that if MMM had not been automated, MMM would not be up for consideration. The old style MMM just doesn't scale. It's too expensive, it's too slow. It's just a problem mathematically, not in terms of the quality of the answers, but your ability to operationalize it has just always been kind of eh, right? But automation totally changed that game. And so we have found that actually the c-suites are far more open to a conversation about m and data strategy in general than some marketing teams. And I think that that's not the way it should be. And so I think you're going to find to sum up that you are pushing on an open door with most C-suites.

Evan Kaeding:

And Mark, to follow up on that, I know here at Supermetrics, we often recommend for our customers that are maybe taking their first foray into a marketing data warehouse to start with something small and manageable with very measurable results, right?

Mark Stouse:

Oh, absolutely. I mean, yeah. So m, the temptation, particularly for people who really become converts to MMM and everyone, we all know people who come very zealous about something that they really believe in. They want to boil the ocean, they want to do it all like now. And that is just not the way to approach this at all, right? So again, kind of a classic strategy that works really, really well is to say, okay, I have sales data and I have something like brand data. So you kind of start at the end of the funnel, not the top of the funnel, and you say, okay, I'm going to model the interactions between brand data and maybe some other things against sales, right? Sales being my dependent variable. And we're going to kind of explore that and we're going to say, okay, man, there's some really strong relationships here in different ways.

And so now what we want to understand is, okay, so what are all the different parts of our brand investment? How are they driving our brand strength, which is then also driving in turn sales in some way. And so there's that. You kind of are deconstructing the whole thing as you move backwards, that is actually the right way to do it, and it creates a very manageable thing. And you're also getting to the heart of what most board members and c-suites care about the most, which is not how this social channel performed against this social channel, against this social channel relative to everything. It's like, okay, I just want the short version of the story. We have brand and demand, and we have sales, and we have three legs of sales productivity, essentially more deals, bigger deals, and faster deals. How is brand and demand contributing to more deals, bigger deals, and faster deals? And that is something you can typically calculate very quickly, and your board and your C-suite will be going, wow, that is so cool. That is exactly the kind of stuff that we're looking for. Here's some more questions. Can you put these on the list for us? And you'll just never really truly run out of questions. But the important part is you don't have to do it all at once.

Riku:

Yeah, that's very true. Rico, we probably have time for one more question. Anything else you want to pick out? Yes, this is a good one. How do you think the open source community have or will impact the MTA and an MMM landscape?

Mark Stouse:

And can you give me more color on that? Can you explain that question a little more? I mean, I think I know who they're going want to make sure,

Riku:

Basically, because as you know, there are a lot of companies selling MTA and MMA products, but how would the open source community affect that? How can they improve upon the approach, maybe even innovate?

Mark Stouse:

Yeah, I think that, so proof automates R code packages the very, very best, most vetted, most popular R code packages in the world for regression analytics. That is the basis of the math in proof, and we are very transparent about that and we publish it and all that kind of stuff. So we're already leveraging the open source community. So the reality here is that there's kind of two aspects to this question. One is about the math, and to the extent that really, really smart data scientists further improve the math, then that would be pretty much automatically represented in proof. And for that matter, data science teams would probably also be picking it up pretty quickly. Quickly. The rest of the innovation, which is more challenging for open source, but is probably very likely to happen, and actually I welcome it happening, is around innovation, around the ux, the way that people consume it, the way that it's automated, the data flows into it, all this kind of stuff. We're already seeing for probably the last five years, we were almost alone in the space as automated MMM, it's now exploding. We're starting to really see a lot of competitors, which I think is great. It's great for customers, and it's actually may seem a little counterintuitive, but it's great for us too, right? It's great for the space. So I think you're going to see more and more of it, and it's going to keep us all sharp. It's going to keep us all innovative.

It's going to be great. I mean, this is actually where it's all headed, guys, and I know you've heard that before, but the reality here is that this is the only approach that science leaves it. The rest of it is just data. And I don't mean that data is not important. It is. It's really super important, and cleanse and harmonized data is even more important. But in terms of better decision making, MMM econometric analysis or multi-variable regression, it's all the same, is the foundation of decision making in all kinds of stuff. It's like you want to manage a financial portfolio, you're going to end up using regression to do it,

Evan Kaeding:

But I imagine a lot of those same open source tools can help give companies something small and manageable that they can start with. And we here at Supermetrics see it all the time. Companies come to us, they say, Hey, we want to start working with MMM. It's a small team they're using in some open source models. They can take that as far as it leads them. Maybe that's great. They build an entire team around it, and then if it becomes too large or they get enough executive, then they can bring it into a commercial vendor like proof as well. So we see that as a very valid path for starting exploring, but also for scaling up an MMM technologies as well.

Mark Stouse:

No, totally agree with you. I mean, you can get started in automated MMM for about 50 grand a year. That gets you five models, so that's very, very consumable for most teams. Average contract for us is around $150,000 a year for the first year, and then it kind of tends to scale up from there. But we're still talking about a dramatic discounts and a dramatic increase in scalability versus the old way of doing it. I do think that open source, in some ways, I consider us to be open source in the sense that we have the same disruptive capability on price and scalability, certainly. Awesome. Well, I think that's all the questions we have time for. Yes, and I would like to thank you, Evan, and especially thank you, mark, for hosting this webinar. Very insightful, very great. We're wrapping up here things. If you have any other questions, feel free to ping me.

Ping Evan both on LinkedIn, very active on LinkedIn, and I know we'll get right back to you. We're not going to bug you either. This is not about feeding a funnel for us. This is about helping educate people and help you, right? So if you want an answer to a question, feel free to ask. Yes, definitely. So be sure to reach out to these guys. But from my side, from Evan's side, from Mark's side, we'd really like to thank you for attending today, and we hope to see you soon. Have a great day or a great morning or a great evening, wherever you are. Thank you and good luck.