CASE STUDY: ANALYTICS - How data uncovers landmines that kill your business

Written by: Bruce E. Segal
LinkedIn Profile: http://www.linkedin.com/in/besegal

After Neilsen revealed Twitters low 40% reuse rate, one company realized that to survive it had to do the same analysis

Summary: When using free trials to launch a new product, it is especially critical to determine if “tryers” will become “buyers.” Segment “tryers” by recency and frequency to see if you have a market or need to develop a new product. Why do this? Even with a hyper growth rate (1000%+/yr.) only 40% of all new Twitter users use it after one month, reports Neilsen. Twitter loses users faster than it gains them. And that is unsustainable.

THE CHALLENGE

The company launched a mobile application that integrates with a contact manager through a website. (Details are masked to conceal its identity.) It used a “freemium” model to launch, giving away the service for free. Many people signed up to use the product, but the company had no revenue. And, the company’s web usage data provided no analysis, insight or action steps.

So the challenge is to identify revenue streams for the company. Are there markets that will use and pay for its product or service? What is the lead funnel from sign-up to users, to repeat user, to frequent and recent user? Can we forecast if users will become buyers?

Surface results looked favorable, when viewing website reports and graphs. They showed more than 7,000 people signed-up in 10 months.
Sign-ups spiked with favorable press on TechCrunch and other sites. Yet, some reports showed a misleadingly rosy picture. The graph above mistitled “Total Users Per Day” really showed cumulative sign-ups; including people who never used the product. The graph below shows a slow and steady increase in unique users each week. While it shows information useful to see “user” growth, it ignores that only 55% of sign-ups converted to use the service even once. And that masked a big problem in conversion and adoption of “tryers” to “buyers,” and
retention of potential “buyers.”

The company needed to understand sources of growth and emerging characteristics of users, information critical to reaching real and informed decisions. It needed analysis and insight into how likely someone is to go from tryer to buyer. And it did not get that from web reports.

THE SOLUTION

So, the solution is to look beyond site reports. Do analysis. Segment users by recency (time since last use) to measure adoption, retention and defection. See if users make discernable patterns that reveal potential revenue streams, market opportunities or product benefits.

Using acquisition tactics, mostly promotions like blog mentions on TechCrunch and word of mouth referrals, the company reached 7,000+ sign-ups in 10 months adding 150 to 600 monthly. We analyzed the conversion rate of sign-ups to users and did a summary frequency analysis which revealed that the reports masked that of all sign-ups–45% never use the product, 30% use it once, and only 26% use it more than once. And this is a free product.

Then we analyzed recency rates.1 How recently a user last used a service or product is a good predictor of future usage and retention rates. Recency analysis revealed of those who sign-up, only 2.4% of them used it within the past 15 days and 4.0% used it within the past month. In the best case, 45% of all sign-ups stopped using the free product 2 months ago or longer and are “Lost.” In the worst case, those same people comprise 80%+ of Users; a huge defection rate.

To build a lead funnel we matched percents in the two charts to forecast use and defections. Of every 1,000 Sign Ups…
446 Never Convert to use service.
554 Convert to use service (24 Current, 78 Lapsed, 453 Lost)
296=1-Time (10 Current, 41 Lapsed, 245 Lost);
258=Repeat (14 Current, 37 Lapsed, 208 Lost).

For every 1,000 Sign Ups, only 24 are Current and 453 are Lost (1-Timers and Repeaters combined). An industry bench mark put this in perspective and highlighted the urgency the company faced. Neilsen analyzed recency rates for Twitter, FaceBook and MySpace users. While not a perfect benchmark, Neilsen’s data gives a directional comparison. It found 60% of Twitter users stop using it after one month. At that rate, keeping only 40% of its users after one month, there comes a time when there are not enough new users to replace those who defect; it is unsustainable. A 40% retention rate limits Twitter to a maximum reach of 10% of the web.

In comparison, at the same growth stage both FaceBook and MySpace had higher retention rates of about 70%. No matter how we slice our data, if this product does not raise its retention rate to 60% or more, then at some time there are not enough new users to acquire to replace defecting ones.

This analysis let the company ask users why they use and like the product and non-users why they stopped using it, or never used it after
sign-up. The answers gave the company the quantitative and qualitative foundation to develop the product, identify a market, and make a go/no go decision.

THE RESULT: ANALYSIS. INSIGHT. ACTION!

The result is the company advanced from data and reports to analysis, insight and action. Analysis and insight identified revenue streams and forecast if tryers would become buyers, which let it make informed decisions and take action.

Analysis: Even as a “freemium,” the product has low retention rate, only 4% of sign-ups used the product within the past month, which is significantly lower than benchmarks, Twitter at 40% or FaceBook and MySpace at 60%+.

Insight: Use analysis to identify users and non-users to get feedback for product development, or to make go/no go decision.

Actions: Ask the 45% of lapsed users why they stopped using product and if cause is fixable. Ask the 4% of recent users why they use it and if reasons apply to others. Make informed product development and go/no go decisions.