When designing products, how to select appropriate metrics that reflect
- Quality of user experience
- Goals of your product
Try the Google’s H.E.A.R.T framework. Original Post by Kerry Rodden. Source – LINK
- Happiness: measures of user attitudes, often collected via survey.
- For example: satisfaction, perceived ease of use, and net-promoter score.
- Engagement: level of user involvement, typically measured via behavioral proxies such as frequency, intensity, or depth of interaction over some time period.
- Examples might include the number of visits per user per week or the number of photos uploaded per user per day.
- Adoption: new users of a product or feature.
- For example: the number of accounts created in the last seven days or the percentage of users who use a particular feature.
- Retention: the rate at which existing users are returning.
- For example: how many of the active users from a given time period are still present in some later time period? You may be more interested in failure to retain, commonly known as “churn.”
- Task success: this includes traditional behavioral metrics of user experience, such as efficiency (e.g. time to complete a task), effectiveness (e.g. percent of tasks completed), and error rate.
- This category is most applicable to areas of your product that are very task-focused, such as search or an upload flow.
When measuring each of these keep things simple and focus on three things in each area:
HOW TO USE ABOVE PROCESS:
- Choose one or two categories in the HEART framework that are the focus of your product or project.
- IDENTIFYING GOALS
- Identify clear goals across your team
- Do not define goals in terms of existing metrics
- E.g. – “Our goal is to increase traffic to our site”.
- Everyone wants to do that, how will the user experience help ?
- A better version would be say in the goals column of the engagement row in above chart:
- For users to enjoy content and keep discovering more to engage with.
- IDENTIFYING SIGNALS
- Must discuss and narrow down which signals seem to be the best predictors of the associated goal.
- Look for data that you already collect on user metrics or data that you would need to.
- Also, must know how easy or difficult each signal is to track ? Is product instrumented to log the relevant actions, or could it be ?
- Also, chose signals that you expect to be sensitive to changes in your design.
- A good example in the goals column of the engagement row in above chart would be:
- Amount of time users spend engaging with content on your product in a specific section of the product.
- IDENTIFYING METRICS
- Refine the signals into metrics you will track over time or use in A/B testing
- Analyze data collected to decide what’s most appropriate.
- An example in the goals column of the engagement row in above chart would be:
- The average number of minutes spent engaging in content on the site per user per day
- Note this is where business meets technology in terms of making it make sense for senior stakeholders for decision making
- Avoid the temptation to add “interesting stats” to your list. Ask yourself, will you actually use these numbers to help you make a decision ? Do you really need to track them over time or is a current snapshot sufficient.
- Avoid unnecessary implementation effort and dashboard clutter. Again go for MVP in this scenario too.
Sources of images above:
Google Article source: Link