Nearly every business around the world has experienced rapid, dramatic changes in consumer and business behavior over the last few years. A decade or more of relative stability was abruptly interrupted by a pandemic, followed more recently by inflation and ongoing concerns about the possibility of a prolonged economic downturn. Historical data on sales, marketing, product usage, or other operational metrics becomes an imperfect guide to the future in times of significant turbulence, and this may be even more pronounced when the immediate past includes a large (hopefully) non-recurring event. Even in the best of times, attributing business results to the actions of your teams is challenging. So, how do you make sense of data as you work to navigate changing conditions?

1. Beware of Outliers

Recent years have been anything but predictable. Often, to evaluate the success of a change, companies rely on historical business data and compare results – sales, bookings, churn – before and after a product update, a new customer success initiative, a marketing campaign or another operational adjustment. However, history can be an unreliable comparison, particularly in the wake of a pandemic or in the face of volatile, inflationary, or recessionary conditions. Data in a volatile economic climate is especially challenging as it can only accurately represent a single moment in time. Even under normal circumstances, external “outlier” events such as unexpected press coverage, changing customer demographics, or seasonality can temporarily skew your analysis. Straightforward before-and-after comparisons will inevitably lead to incorrect conclusions if qualitative factors are not considered. Most businesses need a more sophisticated analytical toolkit to account for the unpredictable and unexpected.

2. Experimentation Leads to Enlightenment

Experimenting or trying, failing, and then trying again, is not a new concept. Yet, testing an unknown result can still feel risky to businesses. In the early 17th century Age of Enlightenment, French philosopher, and creator of the first encyclopedia, Denis Diderot outlined the three principles of knowledge: observation to collect the facts, reflection to combine them and most importantly, experimentation to test and verify them.  Any data-driven culture should be grounded in experimentation: testing changes and letting the data guide you. We see this approach regularly applied to marketing. You may have heard about – or even tried! – A/B testing on website button colors or email headlines. But the power of experimentation also matters in product development, customer support, operations, and in countless areas across your business. With experimentation, some customers receive a new experience while others receive the existing “control” experience. Results between the two groups can be compared because they occur under the exact same external conditions – allowing you to see the true effect. The ability to understand what is currently working and why will help your team find signal through the noise so they can invest their time and resources more efficiently.

3. Validate, Invest, Succeed

Change is inevitable in any economic environment. The value of experimentation is in understanding that change and steering it toward growth. Resources, whether time or budget, are scarce even in the best of times. As businesses continue to navigate changing market conditions, they need to be highly conscious of how resources are allocated. Here too, experimentation is key.

While teams often consider experimentation in the context of optimizing success – picking the option customers like best – A/B testing can also be used to avoid failure; quantifying the impact of product, process and other changes that can challenge results. When was the last time someone in your product organization told you, “We changed something, we think customers like it, but we don’t know how much sales have changed?” Or, “it was only a back-end change, customers won’t even see it!” Experimentation can eliminate risk and be an early identifier of the difference in conversion, sales or any other metric used to measure the new experience against the control group or status quo. Informed by the data, your team can invest talent and budget in ideas and experiences that work and drop the ones that don’t.

As we witness dramatic changes in how people work, learn, shop, and interact, adaptable, innovative companies will not only survive, but thrive.

Understanding the value of experimentation and how crucial it is to be building a durable business will help companies quickly adjust, offer new products, services, and delivery methods to meet customers’ changing needs during this uncertain time. Importantly, experimentation done right not only fuels innovation and growth, it also serves to reduce risk by measuring outcomes to inform decisions. A culture of experimentation can solicit new ideas and solutions and unleash the creativity of employees in every part of your business. Even the ideas that don’t work out can inspire your team to think differently about customers and what they need.

ABOUT THE AUTHOR

Cathy Tanimura is Vice President of Analytics & Data Science at Summit Partners. She works closely with the Summit team and our portfolio companies to help create and execute strategies that effectively use data and analytics to drive better decisions and build better products. She is the author of SQL for Data Analysis: Advanced Techniques for Transforming Data into Insights. Prior to Summit, Cathy worked at Strava, where she built an analytics and data science team focused on product, marketing and business development. Previously, she led analytics teams at Okta and Zynga.

Related Content

Additional frameworks and perspective from Summit Partners

Subscribe to The Ascent

The Ascent Newsletter

 

The Ascent is our quarterly newsletter offering practical perspectives to help leaders navigate the challenges and opportunities of building a growth company. Sign-up to receive the latest issue and stay up to date on the latest perspectives, stories and activity from across the Summit Partners Network.