Product Analytics Frameworks: Top-Down Analysis

Intro

This post presents a sample set of typical top-down product analytics exercises for a consumer app. This approach is useful for teams that want to use analytics to help align product development priorities or come up with feature recommendations.

The proposed analysis includes three areas: (1) broad overview of the state of the app, (2) description of the app usage ecosystem, and (3) deep dive on a specific product feature. The goal of analysis in each area should be to (a) establish shared understanding of usage patterns, and (b) guide discussion towards important focus areas and actionable improvements.

By the end of the project, you should have experience building basic growth metrics, performing descriptive feature analysis, and producing actionable recommendations for a product team.

Getting started

I’ll reference Poesie, a poetry-reading app I built and launched in early 2018, during some examples. You can download it from the iOS App Store here: https://itunes.apple.com/app/id1332841582.

As you begin using the app, think critically about each step in the user lifecycle: sign-up, first experiences, basic interactions, and repeated use.

Think from the perspective of users coming in with different hopes or expectations for the product: How well does the experience match those expectation? Where does it fail?

Think from the perspective of the development team: What is the mission of this app? Does the feature set (and user experience) match that mission?

Basic analytics

The broadest analytical overview of apps looks at core growth metrics. With any consumer app that hopes for repeated use, growth can be broken down into new user acquisition and ongoing user retention. A basic growth analysis organizes and describes the user base using this framework in order to begin aligning the product team around areas of opportunity.

  1. Produce a basic plot of user growth. => How many WAU do we have? What is our growth rate? Any key changes over time or notable moments to investigate?

Example: https://medium.com/@benbregman/growth-accounting-walkthrough-cbeac63009a3

Ecosystem description

Once we have begun to understand the acquisition rates and user stickiness over time, we can shift this understanding into describing the product itself. What features are being used (or ignored) by users? Which features are contributing the most to retention? A successful ecosystem analysis should describe high-level usage patterns of the app and begin to uncover the most important (or lacking) features.

  1. Produce a description of feature usage / user segments. => How many users use each feature? Or, framed differently, what different types of users (defined by feature usage) exist in our user base?

Feature deep-dive

Based on your results from the previous analysis, you might begin to care deeply about improving a certain aspect of our app ecosystem — perhaps improving the experience for a certain type of user or improving the usage flow for a certain feature set. What do the current usage patterns for this feature look like, where are the key possibilities for improvement, and what suggestions can you propose?

  1. Produce a usage funnel for a specific feature => What different steps need to happen for a successful experience with the feature? Where are the drop-off rates at each step?

Conclusion

Tying everything together, a useful output from this entire analysis might look like:

  1. Basic analytics: We discovered that most (X%) user drop-off occurs after a few days of use. In other words, if people build a successful usage habit in their first week of use — they will probably stay with the app for a while (X months/years/forever). How can we help people build (and sustain) that successful usage habit?

As you proceed, always remember to keep in mind that the goal of every analysis/recommendation should be to help the app achieve its mission and provide a better user experience. If we have product market fit, a better user experience should manifest in improvements to user retention. Since this is an observational analysis (i.e. no A/B testing), you should also keep in mind the difference between correlation and causation: successfully retained users may display certain usage patterns, but it is not guaranteed we can encourage such usage in other users (let alone expect that usage to lead to retention).

Good luck with your analysis!

I love music and literature. This is my app: https://itunes.apple.com/app/id1332841582.