Layering multiple measurement methods: How top consumer brands measure marketing

Transcript

Evan Kaeding:
Hello everyone, and welcome to the Supermetrics Marketing Intelligence Show. My name is Evan Kading. I'll be your host today. I'm the Head of Solutions Engineering here at Supermetrics. My guest today is Michael Kaminsky, CEO and founder of Recast, which helps companies focus on their marketing measurement techniques and everything to do with their media optimization. Michael, welcome to the podcast. Would you like to introduce yourself to our audience?

Michael Kaminsky:
Yeah, absolutely. Thanks so much for having me, Evan. So, I'm Michael, technically co-founder and co-CEO of Recast. My backgrounds in econometrics, statistics, and causal inference have applied that to medicine and environmental economics and now to marketing. And I'm excited to be here to talk about, you know, this topic that I have become so passionate about, which is marketing measurement and how to help marketers, you know, make better use of their dollars. So, thanks for having me.

Evan Kaeding:
Of course, yeah. I'm happy to have you here, and it is important, of course, to acknowledge the co-founder and co-CEO piece as well. That's an important distinction, so thanks for making that. Michael, one of the reasons we wanted to have you on the podcast is you recently wrote an article that I saw in Lenny's newsletter, a big popular newsletter in the product management and growth marketing space around different ways that large companies measure the impact of their media spend.

And some of the takeaways that we saw at Supermetrics were profound for companies evaluating their media footprint in the space today. Could you give us a summary of what the takeaways of some of the, of give us, could you give us a summary of what some of the takeaways from that article are and maybe the different methods that you're seeing companies out there in the wild use today to measure their media footprint?

Michael Kaminsky:
Yeah, absolutely. So I think the biggest takeaway, the most important thing that most marketers should take away, is that big mature brands use triangulation to measure the effectiveness of their marketing spend. And so when I say triangulation, I really mean bringing together evidence from different types of measurement methods. So, at least today, right now, a lot of marketers who came up in the last 10 years they're very familiar with digital tracking methods like multi-touch attribution, MTA, last-ditch attribution, first-ditch attribution, et cetera. That's a great way of measuring digital marketing and its impact on digital distribution channels. But when you're a bigger, more complex brand, you must start bringing in these other measurement methods and evidence. So, the three methods that we talked about in that article are digital tracking, multi-touch attribution, and two experiments or lift studies.

And then three, media mix modeling or this type of statistical modeling for measuring marketing. And those three types of methods are widely used, especially the bigger and more sophisticated marketing teams. And so marketing teams today need to be thinking about our roadmap for measurement and how we'll eventually get to this stage where we're using all three of those to holistically measure the effectiveness of our marketing.

Evan Kaeding:
Awesome. So, if I'm going to recap that for our audience, it will be a combination of three different methods. What you talked about is triangulation. So, we're looking at MTA, MMM, and conversion lift studies. For those who may not be familiar, please give us a quick overview of what's an MTA, what MMM is, and what are conversion lift studies.

Michael Kaminsky:
Yeah, absolutely. So, MTA stands for multi-touch attribution. And again, I like to sort of call all of the different types of methods similar to digital tracking. The idea is we're gonna track people across the internet, see what ads they engaged with or clicked on prior to conversion, and then assign every conversion, some combination, one, or some combination of those different ad touch points. So we're gonna track people across the internet. And so...

That's the idea behind that methodology. A lot of great brands were built on top of multi-touch attribution, and whether that's just via Google Analytics or via some of them, or an in-house built tool or some other vendor that they might be working with, it's a really powerful methodology if you're primarily distributing or selling online, right? So you have a Shopify store, and you want to see if they clicked on the Facebook or Google ads before purchasing? It's a great methodology for that.

The downside of that methodology is that it doesn't really measure incrementality, which is this idea of the causal relationship. We can't say if they made that purchase because they clicked on that ad or if they just happened to engage with that ad prior to purchase. The real downside of digital tracking type tools is that they're not really set up to measure causality, which is the thing that we really need to know as marketers in digital channels, but it works less well when you have a more complex marketing mix. So if you're also advertising on radio or podcast or different types of TV channels where there's less reason to believe that the click and the engagement are going to be able to be tightly connected to the conversion event, then you have to start making a lot more really strong assumptions in order to really be able to action off of those digital tracking type methods.

Additionally, right with the changes to privacy regulations and Apple's iOS 14.5 app tracking transparency rules. It's just a lot harder to do that sort of tracking in a really sophisticated way. And so now marketers are starting to realize, okay, there's all of these gaps in our tracking, and it's not measuring incrementality. And so that methodology is starting to get a lot weaker. So, digital tracking MTA is one pillar. Second pillar.

Uh, experimentation, lift studies, geo holdouts. They all sort of fall into this idea of marketing experimentation. The idea behind this is that we can actually run a deliberate experiment to get a true read on incrementality. This is a similar way to what we would do if we were testing whether a new medication works, right? Or whether the vaccines work. The idea is you want to take two groups of people with whom you would show ads. Show ads to one group, not the other. And then look at how many more purchases you got in the group that you actually showed those ads to. And so you can do that at the individual level, which you might do via Facebook's lift study tool or Google's lift study tool. Or you might run it on your own at a geographic level. So you're going to split the country into different states or regions or whatever the relevant political breakdown is in your country, and you'll show ads to some of those regions and not to others, and then look at the relative lift in number of conversions generated. It's a very powerful tool for measuring incrementality, but it's complex to run, right? They can be expensive to run, complex to set up, different marketing partners, right, or platforms like Facebook and Google and TikTok, and whoever have different ways of doing this, and there are snapshots in time.

Michael Kaminsky:
So if you run the test between April 1st and April 15th, those test results apply to April 1st to April 15th, but not necessarily to July and September and November. And so, as we move away from the time when that test was run, there's a lot of good reason to believe that we need to be discounting the read that we got during that test period. It doesn't last forever. And then the last pillar is marketing mixed modeling or media mixed modeling. And this is an econometric method of trying to estimate incrementality. The idea behind it is you look at historical observational data. So how much money did we spend on TV every day going into the past? And how much money did we spend on Facebook every day going into the past, et cetera, for all of the different marketing channels? And you use econometric or statistical methods in order to find the relationships that are in that observational data. So we can say things like when we spend an extra thousand dollars on television, holding everything else constant or controlling for everything else, we get an in number of dollars of revenue. Maybe it's $2,500. And then when we spend an extra $1,000 on radio, we get an extra $2,200. And that then can then feed into budgeting and planning decisions at a slightly higher level.

Evan Kaeding:
Got it. Okay, so that's a good overview of the three different methodologies, I think, all of which rely fundamentally on very different technologies and, to some extent, different statistical techniques. For marketers who are approaching these concepts, and this is a question we get a lot at Supermetrics, which of these methods should I use across my digital media footprint? There's also an element of size that our clients and customers really need to consider.

when they're evaluating these different methods for tracking the effectiveness of their marketing spend, what are some of the things that you look at in terms of the size of the footprint of the media spend that might necessitate some choice of a particular method for marketing measurement?

Michael Kaminsky:
Yeah, really good question. Excuse me, we'll cut that. Yeah, really good question. So the way that I think about it, and this is like really rough rules of thumb. So, for the listening people, this might not apply to you. But in general, if you're spending only less than a couple million dollars a year, call it $3 million a year, I'd say really just focus on your digital tracking methods and even in platform reporting.

Evan Kaeding
sure thing.

Michael Kaminsky:
At that level of spend, you really want to be focused on, like, product-market fit, creative testing, those sorts of things to get to that next level of scale before you start to bring in some of these other more advanced and potentially distracting methods. Once you're above that couple million dollars of paid media spend, in general, I recommend that brands start building what I call their testing muscle. And so this is this idea of,

Start thinking about how you would run a test. Try to run a test. Learn how you will explain the results of that test to the marketing team and your executive team. Start educating the rest of the business on incrementality and how that's different from what you might be getting out of Google Analytics or your other tracking methodologies. And so start doing that, start playing with it, start thinking about it, and start educating the team, build that muscle internally, and learn how to interpret those results. You don't necessarily need a vendor to help you do this. You can set up these tests on your own. I think that's a really good next step. And then from there, once you're in $5 million a year of paid media spend, $10 million plus a year of paid media spend, that's when you wanna start getting really good at testing and start thinking about using MMM, media mix modeling, in order to bring everything together and make those high-level budgeting and planning decisions.

So again, these are very rough cuts. This will be different for different brands, but that's roughly the stages I think about in terms of paid media spending for when they should start investing in these different technologies.

Evan Kaeding:
got it. So, if I'm to articulate that succinctly, if you're under a few million in ad spend across a given year, you should probably really be focusing on your channels, on creative testing, and probably just focusing on the signals that you can get out of a well-implemented digital tracking solution, maybe a Google Analytics, some other lightweight, an MTA. And then, as you grow and as you scale that ad spend, as more resources become available to your business, or you see continued amounts of growth, then that's when you can start to layer in elements of conversion lift studies or MMM into your business for further resource planning. Did I get that right?

Michael Kaminsky:
Yeah, I think that's exactly right. And really, you know, if you're only spending a couple of million dollars a year, you really should probably only be in one or two channels. And I think you're, you know, I see brands doing this a lot where they're like, Oh, you know, we want to be in podcast and Facebook and Google and YouTube, but they're spending very little amounts of money. And the problem is that stretching across all of those channels is distracting for the team, right? It takes a lot of work to get good at Facebook, and Facebook can scale into a really big channel. Same thing for Google search. It will depend on your individual business, like which of these channels will be your best channel. But in general, I think at that smaller, like subscale level, it's better to pick one channel, focus on it, and get good at it, as opposed to trying to do these little small experiments in a ton of different channels that you're never going to get good at because you're not going to invest enough in and actually to learn how to do that channel well. And so I'd say for those smaller brands, focus on

Evan Kaeding
Mm-hmm.

Michael Kaminsky:
One or two channels get really good at those, use those to scale, and then you don't have to. The measurement problems aren't as bad when you're only in one or two channels. The measurement problems get worse as you add on additional channels. And so for the smaller subscale brands, focus, I think, is really the name of the game.

Evan Kaeding:
You mentioned the word sub-scale here in particular, and the way that I think about scale is that once you've essentially thoroughly saturated your target audience within that channel, that's really one of the reasons that you would start to add additional channels into your mix. Okay, I've taken Facebook as far as it can go in my business. I'm no longer able to track whether or not this is making incremental returns for my business. That's where I need to start branching out into other channels. Is that right?

Michael Kaminsky:
I think that's right. That's right. And so, again, you can build, and I've seen lots of companies build really large businesses only advertising on Facebook. You know, 50-plus million dollar a year revenue businesses only on Facebook. And then they add on Google, and that gets them to a hundred million dollars a year. And so if you're doing like $8 million a year of revenue and you're trying to think about what's our podcast plan, I'd say, like you actually, something other than that might be the right fit for you. You should start on these easier channels.

Of course, for the listeners, there are going to be exceptions for that. You listener might be the company where, at $8 million of revenue, podcast makes a lot of sense for you, but I'd say broad strokes, right? Especially for like DTC companies that are growing. It's going to make a lot of sense to focus on the on really great digital platforms that exist that have massive reach, are fairly straightforward to operate, and are well-known rather than trying some of the harder-to-manage channels to start.

Evan Kaeding:
Certainly, certainly. And what would be some examples of some of these harder-to-manage channels? Are we talking about DSPs? Are we talking about out-of-home? Are we talking about, you know, trying to measure something like the effectiveness of the podcast, or what are some of those other hard-to-measure channels?

Michael Kaminsky:
Yeah, all of those. So out-of-home, podcast, you know, internally managed display is really tricky. Anything with, like, you know, video-oriented. So, CTV linear TV is obviously really difficult. Um, YouTube can even be, I think, quite challenging because you have to believe that people are watching the video and then maybe browsing on their phones. And so the measurement and they're more sort of awareness-based channels. Whereas Google Search and Shopping and Facebook are bottom-of-funnel channels. They're like, again, not easy. It takes a lot of expertise to get good at those, but they can reach massive scale, and they're a lot more straightforward to think about how those channels work and how you can measure them. So for, again, customers that are just getting started or listeners that are, you know, earlier on in their journey, focus on those channels before you add on a bunch of complexity into your business.

Evan Kaeding
Yeah, certainly, certainly. And one of the things I think that's important to double-click on here is you made mention of incremental conversions. Now, I think when you're operating subscale, you put money in, you get conversions, you're happy at the end of the day, right? You're selling products, your services are selling, that's great. But when you get larger and when you have a larger media footprint, that incrementality ends up mattering quite a lot more. Could you maybe start to dissect what do you mean when you talk about incrementality? And how should our audience and our marketers be thinking about incrementality as they approach their channel mix?

Michael Kaminsky
Yeah. So incrementality, again, is this idea of a true causal relationship. Would the person have purchased it without that marketing activity or that advertisement? Um, people are familiar with thinking about incrementality in the context of branded search. So let's talk about that. If someone goes to Google and searches for your brand name, and then they click that first ad, which is your company, and they make a purchase, the question that we as marketers should ask ourselves is, did we have to pay for that click or not? There's a case to be made that they were already searching for your brand. They were good. They wanted to purchase from you. And then you just gave Google the extra, you know, five cents or $5 or whatever that click costs you, even though they were already going to purchase. And so that spend or that ad wasn't actually incremental for you. So.

There are cases where that's true. There are cases where the branded search is very incremental. So, I don't want to say branded search is always not incremental. It's definitely not true. But it's a really easy example for people to start to think about, like, well, maybe it isn't actually incremental, and maybe we shouldn't be spending that money on branded search. 10 or 15 years ago, eBay ran a study doing this where they were spending huge amounts of money on branded search. Because people would go to Google, and they would say eBay socks. And then, they would click on the first link.

eBay would pay Google money for that click. And then those people would browse on eBay and buy socks, but those people were looking to get to eBay. Like they didn't need that extra ad, there was not doing anything really for eBay, and they turned off that channel, and their revenue stayed perfectly flat. They had been spending tens of millions of dollars a year on these ads that were doing nothing for them in terms of driving actual incremental conversions. So that's a good example that a lot of people are familiar with.

And then you can start to think about, like, well, maybe that's true for our other marketing channels as well. Right. Just because Facebook is really, really good at targeting people who are in the market for certain products. And so the question is, is like, well, every single ad that Facebook is serving, is that actually incremental, or were those people going to purchase anyway? And Facebook's just really good at predicting who is going to purchase or not and then serving those people ads. And so, maybe it's the case that Facebook is taking too much credit for ads that were going to happen anyway because those people saw our ad on YouTube or on TV or whatever. And that's true across every different marketing channel that you're operating in. And so smart marketers really spend a lot of time thinking about incrementality. And then they start to think about, okay, how are our different measurement methods measuring incrementality? And how far away from incrementality might they be? Last touch attribution, right? Where you give the whole credit to the conversion to the last thing that someone clicked on or engaged with tends to over-credit more bottom-of-funnel channels. It's easy to think about why that might be true. And so good marketers might still use last-touch attribution, but they also wanna be thinking in the back of their heads, well, we're probably over-crediting the more bottom-of-funnel channels. And so we might wanna think about, you know, maybe we don't take this at face value. Maybe we're going to adjust it. And then maybe we're going to run an experiment, you know, every six months on some of our top-of-funnel channels to see how far away from incrementality that touch-based attribution number actually is.

Evan Kaeding
Yeah, it makes sense. And that's probably horrifying but simultaneously exciting for a lot of marketers out there to think that there are opportunities that are $10 million in size out there where, hey, maybe some of those conversions aren't incremental. I think the more experimental marketers get excited by this. Maybe some who are used to the traditional way of running things are a bit more scared about this. But that's why it's important to test. When you run those tests, you identify those opportunities to invest in channels that truly do move the needle for your brand. Exciting stuff, Michael. Let's pivot to MMM here because I know that's an area of expertise, certainly in your in your wheelhouse as well. MMM, of course, has been around for quite a while, as has the field of econometrics. So, starting out in the 60s, companies like Nielsen had solutions to detect the statistical influence of marketing spend on the conversions associated with a given brand.

That study has now matured, and we're now about 60 years past the 60s now. What has changed between MMM then and now?

Michael Kaminsky
Yeah, really good question. So, um, just so our, so our readers can sort of, or listeners, sorry, readers, uh, our listeners can conceptualize what MMM is doing. Let's, let's try it. Let's, you know, the example that I like to give is to put yourself in the shoes of the CMO of Pepsi in 1985, and you need to make your budget decisions for 1986. How are you going to allocate your marketing dollars across the different channels you operate in? And that might be TV, radio, and print are going to be your main paid channels.

And then you'll have some options around, like discounting and in-store promotions. And ecommerce is not a thing. Maybe some people are buying a few things online, but it's not a thing. There's no digital tracking. There's no last touch on what ads people engage with before they buy a Pepsi because all of the Pepsis are being sold in stores. So, you know, grocery stores, some of the big box retailers exist by this time but really like that grocery stores, corner stores, that's what we're talking about. And so if you're trying to make that decision, it's a really, it's tough to think about how are you going to go about, how are you going to go about doing that? And so what practically you would've done is you would have hired some consultants who are econometricians or statisticians. They would collect your data historically. Okay, how much did we spend on TV and radio and et cetera by week going back, you know, the last couple of years? And then how much did we sell by the week going back, going back the last couple of years? And we're going to do some regression modeling to try to understand, okay, when we spend an extra thousand dollars on TV, it generates a number of additional dollars for the sales. And then you're going to take that information and then build it into your plan. And then you're going to go to upfronts where you're going to buy a whole bunch of, you know, impressions or whatever for TV for the next six or 12 months. And then you're going to repeat that process once every six months or once every 12 months, depending on your internal planning cycle. So that's the idea, right? That's where marketing mixed modeling comes from.

And the idea right at the high level, we're going to take aggregate data, and we're going to use statistical models to find those causal relationships in that historical data, very much the same. The difference is that in the last 10 or 15 years, we've developed a whole ton of new, both statistical models and machine learning models that allow us to estimate much more complex statistical models than what was possible 10 or 15 years ago and definitely than what was possible 50 years ago.

Michael Kaminsky:
So, 50 years ago, you had very small amounts of data, and you had to estimate a very simple model on top of that. Now you can estimate much more complex models using supercomputers in the cloud in a way that has made it so that in the past, whereas you might've needed a consulting statistician to spend hundreds of hours handcrafting this special model for your business, now you can apply the power of machine learning and sort of simulation-based statistics in the cloud to be able to get to a much more powerful model, much faster and more affordably. And so that's on the computation side. Additionally, on the theoretical side, there's been a lot of evolution in the realm of statistics and econometrics in terms of people thinking about causality. How do we measure causality? How do we do that in an observational context? How do we do that in a really smart way? You bring those ideas together, and it's brought us to a point today where we can build models much faster that are much more actionable and useful for driving actual day-to-day, week-to-week decisions for marketers.

Evan Kaeding
And if I'm going through the exercises of preparing my data for this model, building up the cloud infrastructure, potentially the data scientist teams, or hiring a company like Recast to help me with my MMM journey, what are the decisions that I should expect to make once I actually get the results of that MMM? Is it going to tell me which channels to invest in? Is it going to tell me which campaigns I should invest in? Is it going to tell me which kind of creative I should be putting out in front of my audience?

Michael Kaminsky
Yeah, really good question. This is an area of active research. So a fundamental problem with these sorts of observational models is that you have to sort of estimate the model at a fairly aggregated level because there are limited amounts of statistical signal because we're operating in the aggregate. We're not operating at the individual user level. And so, in general, MMMs tend to work at an aggregate level. So we're talking about, you know, you might be able to compare Facebook prospecting with Facebook retargeting, with Facebook awareness, with Facebook Ads. But at the individual campaign or creative level, there's often a much more limited amount of spend and therefore, a more limited amount of variation for the statistical model to be able to exploit. The way these statistical models work is they require variation in the inputs to be able to get an estimate. So if you spend the same amount of money every day, there's no variation there. The model won't be able to give you an estimate. If you only spend a couple of dollars, right? Hey, we spent $10 this week and $5 the next week, you know, and we're a big business. The model isn't able to differentiate between those changes and just the noise and the random variation in the overall business. So, in general, right, MMMs are going to estimate at that sort of more aggregate level, how is Facebook prospecting performing as a whole, and how is Google shopping performing as a whole?

There has been some active research on how we can break that down even further, but it requires a lot of computing power and often requires making some fairly strong assumptions about how those internal campaigns or pieces of creative relate to each other. And so I'd say if you're embarking on this journey, you should probably assume that you're going to operate at this more aggregate level and be making budget decisions. Should we spend more on Facebook retargeting or not, as opposed to expecting that you're going to be able to get down to the creative or campaign level like you might be able to get to with in-platform reporting or some types of digital attribution methods?

Evan Kaeding
Yeah, and I think that's where what you said earlier about triangulation really becomes important because are you gonna rely on MMM for 100% of your marketing decisions? I don't think you're, and I don't think anybody's making that promise. I think it's a question of whether you can use MMM to guide you on a macro level. And then, at the micro level, you probably want some other tools to help you make those more tactical decisions in the channel.

Michael Kaminsky
That's exactly right. And that's what we recommend. You know, we'd never, I'd never recommend that someone throw away their digital tracking tooling to just use MMM because you need to make those, you know, more fine-grain decisions. Like should we use the blue background or the red background in this ad? And MMM is really not gonna be able to reliably tell you that. And so you're gonna need to combine these different measurement methods. But again, the thing that I emphasize to marketers is you need to be really thoughtful about which method you are using for which type of decision. And what are the limitations of those different methodologies in this, like the plane of decision you're actually making?

Evan Kaeding
Yeah, it's certainly an important thing. There are a couple of things as well that we certainly find at Supermetrics that for companies who are looking to start doing MMM, there are a lot of challenges, both data challenges and organizational challenges. And Michael, I'm sure you've seen some of the same. Could you give us some examples of customers you've worked with who say, hey, we want to get started with MMM? We're ready to go. But all of a sudden, it turns out that they've got some roadblocks in the way. What are some of the big roadblocks you've seen in your practice for companies who are ready and want to get started with MMM?

Michael Kaminsky
Yeah, really good question. So there are two types of roadblocks. So, one is on the data side, and then one is on the organization side. And so we should probably talk about both of those. So on the data side, at least the way that we think about the right way to do MMM is you want to do it in an iterative, um, you know, in a really iterative way. So you want to run the model, make changes, run the model again, and see what changes the model continuously adapts to those changes that you're making. That's how you can maximize signals from the immune. It's how you can build trust within the organization. And so we think that's a really powerful way of approaching this process, as opposed to once every six months, getting a PowerPoint deck, thinking about it, and then doing nothing. So that's part of our core philosophy, but that implies that you have good, clean data that you can shove into this model on a weekly cadence. And to be able to do that, you really need to not have a human in the loop doing manual data cleaning if you want to be able to achieve that. And so that means you need an automated data pipeline set up that are pumping all of the data that you need for your MMM into a centralized data warehouse that the MMM can then read from. And so tools like Supermetrics are amazing. They can really help companies get set up with that sort of data, especially from, you know, the digital platforms, get that all into a data warehouse. So it's always there and ready to go.

I think companies often struggle on the non-digital side. So, TV, right? It's often you're working with an agency, and you have to rely on the agency to get you that data in a consistent format. And who knows what that agency may or may not be doing in the process of getting you that data. Sometimes, you're running local radio. If you're running local radio, there's gonna be some Steve in Alabama who has to assemble your reporting sheet and email it over.

And how are we going to, like, what's the pipeline to get the data from Steve into our data warehouse, other channels, you know, I think companies should be thinking about this today, but if you're going to do an influencer program, how are you going to track that a lot of influencer programs start out in Instagram DMS and they never make it to the data warehouse at all. And so I think if you're a business exploring those channels, you want to be thinking about what's our process for collecting that data, cleaning it, and getting it into the data warehouse with the rest of our data so that in the future if we ever want to do some amount of statistical modeling, we'll have that data available. And so again, different companies, you know, a lot of companies that have been founded in the last 10 years, they're very smart about data, they have this idea from the very beginning that they want to build this infrastructure. Those companies are generally on a really good path. Older legacy businesses often have a lot of issues in terms of going back to get this historical data for some of these programs that are run by an external agency or have been managed in-house sort of haphazardly.

And so that can be a real challenge. And so I'd encourage every brand to start thinking today about how are we going to get our data warehouse into this good clean state where it's updating regularly so that we can report on it, you know, even without talking about MMM at all, but then also will eventually enable some of these more powerful and sophisticated tools that we might want to do eventually. So.

Evan Kaeding
Yeah, I was going to say that's a really good breakdown. And we obviously at Supermetrics can certainly help on the data side. But I think what you highlighted is just as important, if not more important, which is priming your organization to make sure that they're ready to actually consume the outputs of this model in a meaningful way, right? And making sure that the expectations are set correctly. And there's organizational buy-in behind, hey, this is something that we're going to do. It's going to take some work and we're going to get some results, and those results are going to be actionable, but they're not going to answer everything. We need to continue to have a strategy around media measurement that's informed at a cadence that's also regular and well decided upon.

Michael Kaminsky
Yeah, and I mean, I want to talk a little bit about the organizational complexity because it's really real. And Eric Sufert, who does mobile dev memo, I think just this week published an article on, you know, the problem of trying to explain a probabilistic attribution methodology to the CFO and the executive team. And it's, it's a really great article. I'd encourage everyone to go read it. This is a real issue. So, over the last 10 years, you've trained your executive team and the board to only look at last-touch attribution, which a lot of organizations have done. When you want to start bringing in these other sort of more probabilistic or incrementality type measurements of experiments and MMM, it can be a real issue for them because they're so used to looking at what was the last touch CPA for Facebook and what was the last touch CPA for Google? And having to re-educate that organization that looks last-touch CPA is valuable, but it's not incrementality. It has these flaws and limitations, and in the long run, it's going to cripple our business because if we only measure everything on last-touch attribution, we're never going to be able to start working in these more top-of-funnel channels that are going to be under-credited via that type of measurement methodology. And so I think for organizations that are looking to get to that next level, you really need to start today in terms of educating the executive team, the board, and the CFO on the right way to think about measuring marketing effectiveness. And incrementality tests are part of that.

MMM might be part of that, but you need to start to wean them off of this idea that we're going to make every business decision based on last-touch attribution and start to educate them about these other ways of thinking about measuring marketing effectiveness. Otherwise, you're going to get to this point. You've run an MMM. You have the results. They're not going to line up with your last touch attribution because if they did perfectly line up with your last touch attribution, why are you doing it?

And then no one's going to make it. No one will want to act off of them because they're going to say, well, we trust last-touch attribution, and we're not ready to make any changes. And that's going to be a real problem for you as a business. And so, for marketers, I want to encourage many marketers today to start thinking about how we will start talking about this as an organization. How can we educate the rest of the team that digital tracking isn't the end all be all of marketing, and we actually need to be thinking in this more holistic way?

Evan Kaeding
Yeah, definitely. And I think it's impossible to understate the organizational challenges just around folks who may not understand what marketing's contribution to the business is relative to what it's been historically. I think the fundamental repositioning of marketing and how it delivers customers at the end of the day needs to be reconsidered. Hopefully, some of these new methods, or I guess methods that have been around for a while but some of these methods are now enhanced by new and recent technological developments, can really help along with that as well.

Michael Kaminsky
Absolutely.

Evan Kaeding
So, Michael, that's most of my questions today. But if you were to advise those interested in maybe getting started with MMM, whether that's today or six months from now, what would be the things that you would recommend folks focus on today?

Michael Kaminsky
Oh, man. So, the number one thing is to start educating your executive team on how to think about incrementality. I think that's the number one thing that every marketer should be doing. Make sure the whole company understands incrementality, that idea, and why it's really critical to get right as a business beyond that, I'd say, um, start running experiments. So start talking about, uh, incrementality and then start running experiments and start to again, think about.

Okay, why are these results different from what we see in the platform or via our last touch attribution? That's the best groundwork that you can lay for doing MMM and more sophisticated marketing measurement approaches in the future.

Evan Kaeding
Awesome. So, educating stakeholders on incrementality, making sure to prime individuals who may not necessarily be immediately connected to marketing on the role of what marketing is and how it should evolve in the organization.

Michael (37:37.304)
Yep, exactly.

Evan Kaeding (37:38.866)
Awesome. Makes total sense. Michael, it's been fabulous having you on the podcast today. Certainly look forward to having you back, hopefully at some point, but I've learned a lot, and I hope our listeners have as well.

Michael Kaminsky
Evan, thank you so much for having me. This was a great conversation. Um, if any of the listeners want to reach out, you can find me on LinkedIn. You can check us out at getrecast.com. Thanks again for having me. Really great, and good luck to all the marketers out there.

Evan Kaeding
Thanks a lot, Michael. Much appreciated.