Summative Research: Problem Statement Development
Background:
At Charter Communications, live TV is an important component of the core business. In many respects it is what separates Charter from streaming media providers like Netflix. It is important therefore that the experience of consuming live content is as high quality as possible. This means it should be easy to navigate the channel guide, find something interesting to watch, and ultimately tune to and consume that content.
At the time of this research our flagship TV product had been in the market for a few years. The product organization was aware of some problems with the live experience but only through some anecdotal evidence.
The goal of this research was to investigate the live TV experience from the customer perspective to derive customer backed problem statements that the product organization could begin tackling.
We had three main areas we were interested in:
Generally, how do customers find and consume live content on our platform?
What pain points existed in the current user experience?
How impactful overall are these pain points?
The Study:
We wanted to get the perspectives of a relatively large number of customers making individual moderated interviews impractical. We also wanted rich qualitative feedback meaning unmoderated sessions wouldn’t be effective. So, together with my team of UX researchers and the market research team at Charter, we designed a study we referred to as a “Town Hall” format.
In this town hall format, we recruited 36 current Charter customers who used Charter products to consume Live TV. We then divided them into 2 groups of 18 based on which specific products they used.
During each session, a moderator hired from a 3rd party vendor lead the group through a specific set of topics we devised ahead of time. We asked them to describe and show us how they navigate the guide, find something to watch, use or more advanced features like filtering, and generally describe how they interact with our product. For each of the topics they discussed the moderator would ask participant to move to one side of the room or the other to indicate if they agreed with or disagreed with the concept. In this way we were able to better quantify the sentiment. This also allowed for us to get feedback from a large number of participants even if some were reluctant to speak during the session.
Results:
The study format was very good at illustrating high level behavioral themes as well as gaps in our customer’s knowledge of our feature set and usability flaws with those features.
Fear of Missing Out (FOMO)
A very common behavior our participants engaged in was to scroll through numerous channels in the guide (in some cases in the hundreds) when searching for content to watch. Even after they had found something they thought they’d like, participants stated that they would continue to search through the guide. The reason they gave was that they didn’t want to miss out on something that might be better.
Advanced Features are Unknown
We also found that most participants were not aware of our more advanced features such as filtering the guide by genre. These were designed to help with the first problem we found but they tended to live behind context menus that get explored very rarely.
Usability Problems
When we showed our advanced features to our participants it became clear that there were usability issues that needed to be addressed. For example in one session, it took at least 5 participants working together for 10 minutes to figure out turn off a filter they had set.
Default Channel Grouping is not Clear or Useful
Our channel lineup is grouped by type of content. For example, you’ll tend to find all the sports channels together or all the home improvement type channels grouped together. When we asked participants if they noticed any type of grouping in the guide they were mostly unaware. They did recognize that above a certain channel number the content got “good” but outside of that there was no awareness of channel grouping. Further, if a user is aware the channels are grouped in this way, there is no easy way to navigate based on these groups. A user would need to memorize the channel numbers of the groups in order to gain any advantage from them.
Based on these insights we developed 5 user backed problem statements
Users don’t have an easy way to see all the content that is relevant to them.
There is no quick method for users to navigate a large number of channels.
It is difficult for users to discover advanced features within our application.
Usability problems within our advanced features make them difficult for the users who have found them to use.
Grouping channels in the guide provides no direct benefit to our users because they are not aware of the grouping nor are there features to take advantage of the grouping.
Based on this work, the product organization has prioritized work on the filters feature in order to make it easier to discover and more useful to users. Further qualitative studies will be conducted on filters, ultimately leading to the development of an A/B test of new filters concepts against the current version.
Skills Used
Project Management
Vendor Management
User Needs Analysis
Qualitative Study Design