Retention Analysis in Adobe Analytics – Part 1: Cohort Tables and Builtin Functionality

User Retention is crucial to any digital offering. If you optimize your offering to a point where users come back on their own, you can not only save on marketing cost but also engage your existing users more. This makes retention analysis a prime example for how digital analytics can provide tangible business value.

In this post, we are going to take a look at how we can analyze user retention with the most advanced digital analytics tool, Adobe Analytics. We are going to start in this post with the builtin analytics dimensions and metrics, then take a look at cohort tables, and in the next post even build our own Segments and Calculated Metrics to help us understand retention. Let’s get started!

Builtin Retention Metrics and Dimension

To start things off, we will take a quick look at what Adobe Analytics has to offer out-of-the-box to help us understand certain aspects of retention. Thanks to the stellar processing Adobe Analytics does with all collected data, we have information about the number of the visit a certain action took place. That information is captured in the Visit Number dimension in Analytics. If we quickly combine it with the Visits and Page Views metrics in a table, we get something like this:

The Visits metrics is best suited for this kind of analysis since the Visit Number is valid for the entire visit. Every event and activity belongs to a certain visit with a certain Visit Number. A simple table like this already tells us quite a bit about our users: 21.5% of our Visits in the selected date range were there for the first time, but only 14.5% of Page Views happened in those first Visits. Now we already know that our return visits are more engaged visits than the first one!

We can use this dimension in a lot of fun ways. For example, consider this table:

Here I just quickly filtered the Visits metric by the Device Type dimension. This quick analysis shows us that our retention (defined as the percentage of return visits) is slightly higher on Desktop than it is on mobile devices like phones and tablets. Another fun way to use this dimension is as metric filter to use it with other dimensions, like this:

Here I’m looking at the page report with the Visit Number as filter on top. The first item is really interesting: While only 32% of First Time Visits use this page, it’s already at 41.7% in the fifth visit! This page becomes more and more important for our loyal users. This is a prime candidate for personalization activities!

While it’s fun to look at the different visit numbers on their own, it can be quite challenging to do the mental math to estimate new and returning users. Because of this, Adobe Analytics has two builtin segments based on that dimensions:

We can see that the numbers match our first table perfectly! This is because the definition of First Time and Return Visits is quite straight forward:

That Return Visits segment is almost disappointingly simple. There is no secret sauce here, it’s plainly based on the Visit Number, returning Hits where it is greater than 1. If we want to reduce the cognitive load on our tables above or just show some quick percentages, those segments are our way to go. We can always break them down by the Visit Number dimension if we want some additional depth.

A neat application for those segments would be Marketing Campaigns. By using our two segments, we can quite quickly find out how many of our campaign respondents are new or returning users to adjust our campaign targeting. On top of that, we can then drill down to see the behavior of those users in relation to their individual customer journey, expressed by the Visit Number dimension. Quite cool!

Another cool builtin Metric is “Visits / Visitor” or “Visits per Visitor”, which gives us the number of times a user came back to our page. We can use this metric together with a Campaign dimension as well to see how often users from a certain campaign came back without using a different campaign!

Going further with Cohort Tables

Talking about neat things: Cohort Tables in Adobe Analytics are definitely among the coolest tools we have at our disposal. They allow us to monitor user behavior through time, which is really what Retention Analysis should be about. While the Visit Number gives us a snapshot in time for all users, Cohorts trace individual users along their journey on our digital experience. So let’s have a look!

In Analysis Workspace, drag in a new Cohort Table Visualization or select it from the options in an empty Freeform Panel. What we get is one of the most powerful but complex interfaces within Workspace:

There is a lot to unpack here. The most obvious (and only mandatory) settings are the Inclusion and Return criteria (1 and 2 in the screenshot). The Inclusion Criteria (1) defines, which Users we want to analyze over time. This marks the starting point for our cohort. First, we need to drag in a Metric (which can be anything except Unique Visitors or Calculated Metrics) which we can then further segment. For example, if we wanted to look at people who clicked through on a specific marketing campaign, we could use the Click Through metric and segment for that specific campaign.

Next, we need to do the same for our Return criteria (2). Here we define when we want to count a included user as a retained user. We have the same options here as for the Inclusion criteria, giving us all we need to specify the event or state we want our users to be in. For example, if we want them to visit a certain page again and again, we could use Page Views as metric and segment for that specific page.

Then we have the options below that, allowing us to trace users for different granularities (3) like days or weeks and choose if we want to look at Retention or Churn (4). That last option defines how the result is calculated in the returned table: If 4 out of 10 people come back within one day, Analytics would calculate that as “40% Retention” or “60% Churn” and display the result accordingly.

Last, we have some quite advances Settings (5) for the way we calculate the results. We will go into detail here later.

To start this complex topic off, let’s look at a very simple example. I just dragged in the Visits metric for both the Inclusion and Return criteria:

Let’s hit the Build button and see what we get for one week of data:

Now that looks fancy! Secret tip: If you ever need to get rid of an annoying manager at your desk, just use quickly create one of those tables to scare them away. What we are looking at here is the retention for the Visitors of a certain day (“Cohort” column) that match our inclusion criteria (“Included” column) over the following days (“+1…” columns). In this example, we had 5,439 Visitors on Aug 10 of which 1,747 came back one day later, which is 32.1% of our Aug 10 Visitors.

Please note: the Visitors on the rightmost columns don’t necessarily need to also have visited our site on the previous days! To confirm this, we can right-click the “+3 days” value in the Aug 10 row to create a segment from that cell (cue Matisoff) and see what Analytics is doing behind the scenes. Let’s look at the definition:

Here we can clearly see that our included Visitors do not need to have come back on any previous day between Aug 10 and Aug 13. And we can see learn something else: For almost any new and shiny visualization in Analysis Workspace, Analytics just uses good old segments behind the scenes. But let’s explore the other options we have with Cohort Tables. Click the little pencil icon to get back to edit mode.

The first obvious analysis I want to conduct is looking at only our First Time Visits and see how those developed over time. To do this, all I need to do is drag in the First Time Visits segment from before in the Segments slot of the Inclusion criteria like this:

Let’s build that table!

Very cool analysis! We now see that 13.5% of the Users who had their first Visit on our site on Aug 10 were also active on Aug 11! In the Average Retention row we also have the average of 13.7% for all the days in our table (we could also disable that row in the settings). I think of this Cohort feature as this post’s first tables on steroids, since we are now able to really track our users through time!

Coming back to the question I teased above: How can we see, which of our users were active on consecutive days instead of any day? This is what the Rolling Calculation checkbox is for:

This innocent looking checkbox completely changes how Analytics calculates our table data. We can clearly see this once we build our table:

But where did all my users go? We see that the “Included” and “+1 Days” column are still the same as before, but the numbers drop rapidly after that! Let’s get to the bottom of this by pulling another Matisoff and looking at the segment behind the “+3 Days” column in the first row:

Well, that escalated quickly! This monster has very little to do with our innocent segment from before. But it tells us exactly where our numbers are coming from: For a Visitor to be counted in that cell, they must have had their first Visit on Aug 10 and be active on Aug 11, 12, and 13! I’m not surprised there are not a lot of those users.

Once again I love how easy it is to get a complex segment like this within a few clicks in Workspace. But there are still some more use cases to explore, so let’s advance!

Advanced Cohort Tables in Adobe Analytics

If we are brave enough to enable the “Advanced” checkbox in the Cohort Table Settings, we have two new options at our hands:

With the first option, we can create a “Latency Table”. If you ever want to make a Data Workbench veteran flinch, just remind them of those. Once we build a table with that option enabled, it looks a bit different than before:

Here we notice two things: First, my selected date range (Aug 10 to Aug 16) is interpreted in an… interesting way, since we don’t have as many columns as we would expect. Second, the table layout looks different now because we now have some days prior to our selected days in here. We read this table in a similar way to before: Of the 5,439 Visitors on Aug 10, 29.9% were also active on the day before and 32.1% on the day after.

So with Latency Tables, we are not only able to look at the activity after the inclusion event but also before that! Pretty neat, right? But now we will go even further, by activating the Custom Dimension Cohort Option:

This gives us another slot for a dimension. While this could be any Analytics dimension, we will start with the simple Device Type dimension. Once built, our table now looks like this:

This is quite different from our previous tables. The first thing to note is that our rows now have turned into the top items from our selected dimension (if we wanted to only include certain elements, we could have filtered the dimension when building the table). Also, we are only looking at one cohort now (starting on Aug 10), which is the first day of my date range. With a table like this, we are able to answer some very different questions, like “how many of our desktop users on Aug 10 returned the next day?” (answer: 34.3%).

Let’s take a look at how Analytics defines the segments behind those cells. By applying our Matisoff-maneuver, we can once again take a look behind the curtain and see what is going on:

Fascinating! There is something important for us to note here: The Custom Dimension we selected is only applied to the Inclusion criteria, but not the Return criteria. While this is a non-issue for the Device Type dimension, not knowing about this can lead to confusion with other dimensions. Let’s look at one more example.

Are you thinking what I’m thinking? Yes, let’s throw our Marketing Channels in there:

Now that should get you excited! We are looking at how many Users of a certain Marketing Channel on a certain day came back the next days. How awesome is this?! And now we see why the definition of those numbers actually matters: When looking at those channels, we might think that a Users had to use the same channel on a different day to be included, but that is actually not the case!

A SEO User on Aug 10 might come back through SEA on Aug 11 but would still be included on the SEO row! This is because the Custom Dimension “Marketing Channel” is only applied to the Inclusion Criteria, but not the Return criteria. I don’t think this is a bad thing, but it’s not very transparent how those numbers are actually created, so we need to keep it in mind.

Wrap-up

Congratulations, you made it trough one of the most complex topics in Analytics and Visualizations in Workspace! I hope you understand why my Retention regarding Adobe Analytics is so high: It’s just a lot of fun to build those sophisticated tables with a few clicks and be able to look at the segments behind them to really see what Adobe does behind the scenes.

With this post, you should be able to analyze your user’s retention over time and in regards to the devices or channels they used. In the next post, I will give you some even cooler tools to work with, because we are going to build some custom Segments and Calculated Metrics!

Stay tuned!

Frequently asked question

Frederik Werner

German Analyst and Data Scientist working in and writing about (Web) Analytics and Online Marketing Tech.

2021&2022 Adobe Analytics Champion

EMEA Adobe Analytics User Group Lead

www.frederikwerner.de/