Ready to Experiment? A/B Testing Optimizes the Considered Purchase Journey

Press enter to search
Close search
Open Menu
email marketing
A/B testing provided insight into customer behavior while increasing conversion rates by optimizing individual features of Wyndham’s digital experience, one at a time.

Ready to Experiment? A/B Testing Optimizes the Considered Purchase Journey

By Jessica Porges, Head of Digital Experience Optimization, Smart Panda Labs - 11/04/2019
Jessica Porges
Jessica Porges, Head of Digital Experience Optimization, Smart Panda Labs

Think about the considered purchase – the vacation club, hotel, and timeshare investment – that requires consumers to seriously weigh their options as their decision comes with a high degree of financial or emotional risk and reward. Today, it’s incumbent on businesses to optimize the considered purchase journey to drive sales – the hospitality industry is no exception.

Not too long ago, Wyndham Vacation Rentals was looking to convert more of its site visitors to purchasers and seeking ways to stave off their digital-savvy competitors such as VRBO and Airbnb. Wyndham faced these challenges and more – their website was slow, code was poorly maintained, and site analytics were not optimized to provide consumer insights. Wyndham needed a digital strategy to help drive revenue growth and, therefore, greater value for the business – a critical strategic goal for the organization. While they were effective at driving prospects to their site, they weren’t converting those prospects into new customers. Ideally, they wanted the site to drive more revenue without incurring incremental costs (i.e., marketing and advertising). They partnered with Smart Panda Labs, a digital strategy firm that helps the hospitality industry optimize their customers’ and prospects’ online journeys to drive increased conversion rates for considered purchases.

Experimenting with the Customer Journey

To understand why Wyndham was not converting site visitors into customers required a comprehensive audit of the customer journey and an assessment of the tools Wyndham used to gather and analyze customer data. Smart Panda Labs blended a program of strategic consulting, digital experience optimization, analytics and intelligence, and marketing technology support in developing an action plan focused on experimentation.

Areas of improvement were identified based on analyses of the entire customer experience – keyword search behavior, email copy, form design, checkout funnel – focusing on pain points and supporting promotional initiatives. For Wyndham, the team experimented with and focused on digital assets for webpages including home, product, and landing; and shopping carts and forms.

Experimenting with the personalization of these assets was key to achieving Wyndham’s business goals. Smart Panda Labs created an A/B testing roadmap using the Optimizely experimentation platform, which allowed the team to experiment across product and marketing teams. A/B testing pits two (or more) variations of a digital campaign or website against one another – A against B, or ‘control’ against ‘variable.’ Wyndham’s audience was split between these simultaneous versions of an experience and the performance of these variants was measured using the metrics that mattered most – conversion rates, revenue, revenue per visitor, and engagement.

A/B testing provided insight into customer behavior while increasing conversion rates by optimizing individual features of Wyndham’s digital experience, one at a time. These features, such as calls to action or images, were tested, optimized, refined and tested again, establishing an iterative, continuous improvement process. Even the slightest change to a small detail can have a profound impact on conversion, revenue, and revenue per visitor. A/B testing placed data at the heart of Wyndham’s digital decision making and was an invaluable strategy that trumped assumptions with real information in real-time.

In addition to A/B testing, analytics were leveraged to reveal precisely how prospects interacted with Wyndham’s site. To understand the patterns that drove prospects to the booking engine, key metrics monitored included:

  • Site visitors
  • What visitors did on the site (behaviors)
  • Which visitors converted
  • Which conversions drove revenue

Data Analysis – The Reveal

CTA test results revealed a lot that Wyndham did not know about their customers and helped to guide more strategic digital decision making. For example, customers who accounted for 20% of annualized incremental revenue use Microsoft Edge web browsers. Why? Applying some Smart Panda Labs creative intelligence, the team hypothesized that Edge browser-users trend older and are less tech-savvy.  They are the ones who use the default browser on their PC versus downloading browsers such as Chrome or Firefox.  Analytics data was used to validate this hypothesis and: While Edge visitors only accounted for 3.9% of website traffic, they accounted for 20% of incremental revenue with many in the 45-54 age range and most in the 55-65 range.  Wyndham had speculated they had an older demographic but never had the data to back up their hunch. With fact-based insights, they were armed with the information they needed to cater to and target this audience.

Conversions are So Uplifting

While it should be noted that some tests for Wyndham called for multiple variations to be tested as part of a single experiment, for the most part, Smart Pandas Labs focused on two keys areas for A/B testing and analytics: 1) Call to Action (CTA) on the Unit Page; and 2) the User Experience (UX) of the Payment Information Page.  Smart Panda Labs chartered a digital path for Wyndham that effectively elevated company bookings. 

Results from CTA Experiment

  • Unit-to-payment information conversion lift of 9.5% with a projected $4.2 million  incremental annual yield
  • For Microsoft Edge users, unit-to-booking conversion lift of 35% (see Sidebar: Data Analysis – The Reveal)

Results from UX Payment Page Experiment

  • Increased revenue per visitor (a metric that blends average order value and booking conversion rate) by +5.63% or about +$33 per visitor to the payment information page.
  • While the average revenue per session on mobile devices was less than half of that on the desktop - in line with the historical trend - the experiment seemed to have the most impact on mobile devices, with an 11% increase in revenue per session (compared to desktop with a 5% increase). The majority of this was driven by increased conversions, not increased average order value (AOV).

Another lesson learned from the Wyndham experimentations revealed that optimizing the mobile experience is becoming more important. The mobile experience accounted for 41% of sessions during the experimentation period, compared to 33% in the previous year.

Consider this: in a short four months, the digital strategies and tactics deployed by Wyndham resulted in an enviable annualized revenue increase of $8M. The evidence is clear –  if you want to generate higher conversion rates to increase revenue without incurring incremental marketing cost – get ready to experiment and optimize the considered purchase journey.

About the Author

Jessica Porges is head of digital experience optimization at Smart Panda Labs, a digital consultancy that uses data and creative intelligence to drive customer lifetime value.