Retention Analysis in Adobe Analytics – Part 2: Custom Segments and Metrics
User Retention is crucial to any digital offering. If you optimize your offering to a point where users come back on their own, you can not only save on marketing cost but also engage your existing users more. This makes retention analysis a prime example for how digital analytics can provide tangible business value.
In the previous post, we used Cohort Tables and some builtin features of Adobe Analytics to analyze User Retention. But there is a lot more Adobe Analytics has to offer once we start using Segments and Calculated Metrics. In this post we are going to build our very own Segments to see how many of our Users we are able to retain. Based on those Segments we will then define some Calculated Metrics to make our lives even easier. I’ve also put the results on the Open Adobe Analytics Components Repository. Let’s start building!
Simple User Retention Analysis in Adobe Analytics
A first step in analyzing User Retention is to define your business goals. This is necessary, because before we start building any segments we need to define two checkpoints in the user’s journey: The inclusion criteria (for example, a user’s first visit) and the retention criteria (for example, another visit after the first one). Those should sound very familiar to what we needed for Cohort Tables. With those two example checkpoints we would be able to answer business questions like “how many of our first time visitors came back?” in a given date range to judge how efficient our marketing is.
But let’s start with a simpler example. What if we wanted to know how many of our users visited our site two or more times in a given date range? We could do this with a very simple Segment like this:
We would then use this segment with the Unique Visitors metric like so:
So what have we done here? With this Simple Retained Visitors segment, we are asking Analytics for all the activity of users who came to our site at least two times in the date range we selected. This is important to remember: A user must fulfill this requirement within the date range that we are analyzing. Any activity outside that date range is ignored.
In our example, 11,532 users had at least two visits within the week I am analyzing. We can also see that those users seem to come back quite often, since the total row shows much less users compared to the sum for all those days. Another important detail for this table is that what we see here is all of the activity of those users in this week. A user who visited our site on three days would show up on all three rows! Because of this detail, a single row in our table gives us information on how many of the users on that day had at least one more visit across the whole week.
Now let’s put this into perspective. We can quickly drag the Unique Visitors metric right next to our Retained Visitors column. By using the Matisoff Technique we can create the percentage of Retained Visitors by just dividing them by Unique Visitors and formatting it as percent:
Now we can clearly see that 54% of our Unique Visitors had two or more visits within this week! We can also see that we had slightly more of those users on Aug 6 compared to Aug 3. Another interesting detail: Our total row is quite low compared to the rows below, because our Retained Users come back on multiple days, which again confirms our previous observation.
This analysis is very helpful already. To give you an example, we could apply this type of metric to our Marketing Channels or Device Types to look for some potential for optimization. But we won’t do this here because we have quite some other things to explore!
Complex User Retention Analysis in Adobe Analytics
That metric above is very helpful as it is. But one of the problems I’ve experienced with that kind of metric is how to read the trended table: Instead of understanding it as “how many of the users from that day were retained users”, people often read it as “how many of the users who were active on that day came back later that week”, which is not the correct way of reading that table.
Luckily, we are working with Adobe Analytics, so we can just build a metric that does exactly that. But before we can do this, we need to build a segment for it. This is how it looks like:
Let’s go over this in detail. The first change is the Segment scope: We are looking for Hits instead of Visitors now, because we need quite some more conditions below that. Next, we have our trusty Simple Retained Visitors Segment to return all Hits from our retained visitors. But then we go way further: Because of the way Sequential Segmentation works in Adobe Analytics, we need to exclude all Hits that follow the first visit of our visitors in the selected date range. What? I know. Let us look at this Segment in action:
That looks quite different! With this segment, we are segmenting for only the first visit in a sequence of multiple visits for users who came to our site at least twice. To prove this, let’s rebuild the table from before, create the percentage metric and throw in the Visits / Visitor metric:
Here we first look at the second column: This ratio of 1.0 clearly shows that we are segmenting for exactly one visit for each visitor, which is the first in a sequence of multiple visits. So it really is working! The ratio on the right is precisely the information we want: 71% of users who came to our site on Aug 3 came back after that in the selected date range.
We can also see that it works because the metric is way higher at the beginning of the date range compared to the end. This is a natural thing, since our users on Aug 9 had way less time to return to our site compared to those on Aug 3. That’s trended information is pretty awesome! But what about the users that came to our site for the very first time?
New User Retention Analysis in Adobe Analytics
Luckily, we have already explored the Visit Number Dimension in the last post. Building on that dimension we can segment for returning first time visitors without any hussle:
Simple, right? All we have to request is the activity from users who had both their first and their second visit during our Date Range. Let’s see how this looks like in a table:
Huh, that looks funny. Why is the number of Visitors so much higher at the bottom of our table? If we look at our segment definition, we find the reason quite easily: We are looking for all the activity from users who had both their first and second visit in our date range. This is different from what we did before, since our segment population grows over time as more and more users had their first visit. We can look at this in detail by breaking down the first and last day by Visit Number:
Here we have it: The growth in our segment population is simply because we give our users more time to come back towards the end of the week. Now all we need to do is stack the preset First Time Visits Segment in both columns and calculate the percentage again. We end up with a table like this:
I hope this is a bit more intuitive compared to the segments above. In the left column, we first segment for all the activity of our Returning First Time Visitors and then narrow it down to only the first visit of those users. This gives us the number of users who had their first visit on a given date and returned later. If we divide this by all the first visits on that day, we get the percentage of users who came first on a given date and returned later, which is again decreasing over time. We already know this phenomenon from before, but can’t we do something about the decrease?
Time Restricted New User Retention Analysis in Adobe Analytics
That’s quite a headline! To get rid of our decreasing numbers over time, we can just modify our Segment once more:
We didn’t have to change much in our segment: All that’s new is the “Within 1 Day” restriction, since we are using a Sequential Segment now. We repeat the same steps as above to get our trusty table:
I love this! The percentage we see here gives us an awesome overview of how many of the people who came for the first time on a given date returned within just one day! This is why our metric goes down only on the last day, because those people did not have as much time to come back. Let’s have some fun with this metric: We can simply change our segment from “within 1 Day” to “within 1 week” or “within 1 month”. If we throw those three metrics in a graph and table, we get this nice result:
There are a couple of things for us to learn here. First, we see that our retention rate goes up if we give our users more time to come back. Duh! We can quite nicely compare how the retention rate differs from one time restriction to the next by comparing the columns in our table. I’ve put a box around a very interesting section on our graph to show something very cool: We can actually see how our retention metrics collapse into each other as the time restriction is reached, so the monthly retention becomes equal to the weekly retention (once there is less than a week left in the date range) and equal to the daily retention on the last day!
Wrap up
So let’s go over what we have built today. We started with a simple measure of how many of our users came more than once to our site in a given date range. Based on that we created a sophisticated Sequential Segment to see when those users had the first visit in a sequence. Then we took it once step further by analyzing how many of our first time visitors came back at all and within a given time. I find this quite remarkable, demonstrating how easy it is to build those segments and metrics.
What we didn’t cover in detail is why we would want to build those metrics in the first place. We might apply them to marketing campaigns, feature releases, or test groups, but why would we want to do that? What should come first is setting the business goal. If we first define a goal like “at least 50% of the users we acquire through a marketing campaign should return within 1 week”, we could then build a metric and report to analyze our marketing performance. It’s very important we start with that definition to keep our business aligned to the same goal!
As always, I’ve put all the metrics and segments from this post on the Open Adobe Analytics Components Repository. You can just go there and look at the definition in detail or copy the API requests for all the components I have used. If you should have any remarks or ideas, just submit them there!
German Analyst and Data Scientist working in and writing about (Web) Analytics and Online Marketing Tech.