If users churn, do they ever come back?
Posted May 20, 2016
Perhaps one of the most commonly used metrics in mobile marketing is the retention rate - which tells you what percentage of users returned to your app (or, equally valid, how many still use it) after a given number of days. This rate determines two things. On the one hand, it measures how useful or entertaining an app is to users over a longer period. On the other hand, it reflects the success of marketing outreach to reach the right audience.
But behind that number rests an important assumption.
The inverse of the retention rate is frequently regarded as the churn rate. If we retain 15 % of our users on day 7, that would mean 85 % are gone!
This definition can serve us well, but as we’ll see, it’s mostly untrue.
If you look at a graph of retention rate over time, you will see that it gradually decreases over time. The line goes downwards. You can, for example, see this in our Mobile Benchmarks.
With our cohorting tool, you can run this graph for 30 days, 12 weeks, or an unlimited number of months. Eventually, some weeks out, only a small percentage of users are left. The sheer majority have churned.
What happens to that majority? What do they do?
One intuitive assumption would be that they’ve tried the app and turned it off, never to return.
But assumptions like this can lead us to make dangerous decisions about our stack. We might question if there is any point to sending push notifications to users that have been inactive for six, eight, or twelve weeks? Should we ever try to re-engage these users? When you build databases, do you need to keep their data?
The dataset we have in front of us today tells that very story.
What’s the data say?
The story starts with the tech team. In our never-ending work to enhance the performance of our systems, we wondered if the in-memory databases could be shrunk by implementing an expiry date to some long-untouched records. The in-memory databases store attribution data, and currently do so indefinitely - whether or not the record has ever been “touched”, i.e. the user has been active within an app prompting the records to be retrieved.
Are the databases allowed to forget users who have been inactive for a very long time?
On a system that processes petabytes of data every month, you need to carefully consider decisions like these. Even the smallest percentage errors stack up. So the guys pulled out the data. The question to answer was this:
How likely are users to return to an app after long periods of inactivity?
The graph below shows to what extent the system would duplicate user records if they expired after a certain amount of time.
In the period we looked at, the number of users returning after a 50-day break was equivalent to 22.16 % of all new installs in that period.
Say you have a userbase growing by 1,000 new users every month. If 50 days is long enough that you’d treat those users as gone forever, you’re losing crucial data on more than 200 users!
At 90 days, the number is still over 10 %. If the tech guys decided to expire user records after 90 days, then:
- your install numbers would be off by 10 %;
- you’d pay ad networks for up to 10 % of existing users that look like new acquisitions;
- and your analytics would significantly overestimate short term retention and underestimate long term retention.
We only start seeing sub-percentile rates at 300 days!
So what does this mean?
Well, for one, the decision was made for us: User records need to be kept for a very long time.
Presumably, the limit should be as far out as the device is still likely to be in use by the same user. (Next up - benchmark how long it takes for our company smartphones to burn out.)
Sure, that’s going to cost a lot more money. In-memory databases aren’t cheap to run. But that’s the sort of tech you need to be able to instantly look up user records, in real time, and let your algorithms make decisions about attribution, callbacks, and respond to SDK calls instantaneously. Essentially, it’s the glue that holds the accuracy of our attribution logic in place.
For that majority that makes up the churn rate, all hope certainly isn’t lost.
You can use this data to make decisions about when you cut off a user from your comms.
For how long should you keep trying? Well, depending on the effectiveness of your re-engagement, and how valuable a user is to you, there may well be a very good point to staying in touch until 150, 180, or 250 days. From what we’ve seen though, it’s quite unlikely that a user is going to come back if they’ve been dormant more than a year.
And for sure that data should still be around.