An integrated approach to TV Recommendations
This white paper is also available in PDF format
Introduction
Finding compelling shows to watch is the lifeblood of any TV service: when viewers can find content that appeals to them, satisfaction goes up, usage goes up, and businesses succeed.
However, with a growing number of channels and more and more programmes available on demand, viewers get confused, lost, and end up sticking to traditional TV channels, frustrated by repeats of old TV shows without knowing what is available on premium channels or video on demand services.
With recommendations technologies, viewers are automatically advised of TV shows that they might like without the need to search for specific items or browse through pages of a TV guide. This results in increased viewing of premium content, greater satisfaction for the viewer with increased loyalty and higher revenues for the business.
This white paper reviews the different underlying technologies that can be used to provide automatic recommendations for TV shows and examines the benefits of combining these approaches into a single integrated system that drives recommendations based on multiple sources of data into a single, transparent, user experience.
Market drivers for recommendations
The reduced costs of television production and transmission combined with the ever increasing back catalogues of the major broadcasters had lead to an explosion in the amount of content available to viewers.
Most TV service providers have used this is a point of differentiating, each claiming to have more content available to watch than any of their competitors, but the viewer is frequently left frustrated by a bewildering array of choice as the TV guide technology has not typically been deployed to support the quantity of content that has been made available.
Recommendations technology is one of a number of solutions that can be applied to address the issue of find-ability and is typically used to help users discover programmes that they might not otherwise have known were available.
The challenges for recommendations
The difficulty in implementing recommendations is that different users have different tastes and opinions about which television programmes they prefer.
- Quality – a substantial portion of the shows that are recommended to the user should be shows that they would like to watch, or at least might find interesting
- Transparency – it should be clear to the user why they have been recommended certain shows so that if they have been recommended a show they don’t like they can at least understand why
- User feedback – people are fanatical about television programmes and if they are being recommended a show that they don’t like they should have an immediate way to say that they don’t like it and subsequently never have it recommended again
- Driving take-up – ultimately the recommendations needs to drive the take up of the shows that they are recommending. This can only be measured by monitoring the shows that are recommended and seeing how user behaviours change
The different approaches to recommendations
Most recommendations systems use a combination of different approaches, but broadly speaking there are three different methods that can be used:
- Collaborative filtering of different users’ behaviour, preferences, and ratings
- Automatic content analysis and extraction of common patterns
- Social recommendations based on personal choices from other people
Each of these approaches can provide a level of recommendations so that most recommendations platforms take a hybrid approach, using information from each of these different sources to define what shows are recommended to the users.
Collaborative filtering
Collaborative filter methods are based on collecting and analysing a large amount of information on users’ behaviour, activity or preferences and predicting what users will like based on their similarity to other users. The most famous collaborative-filtering algorithm is Amazon’s approach which provides recommendations for future purchases based on the users’ previous activity.
Figure 1 - Passive collaborative filtering from Amazon
Such an approach is referred to as “passive filtering” because it provides recommendations based on activity without explicitly asking the users’ permission to do so.
It is also possible to implement active filtering where the user provides information that will be used as the basis for recommendations and such an approach is common amongst DVD rental services that use ratings of previous purchases for recommendations. One of the most famous active collaborative-filtering systems is Netflix’s recommendations engine which has been advertised through the Netflix prize, whereby the firm is offering a million dollar prize to any team of people who can improve on the predictions of their recommendations algorithm based on a novel approach to collaborative filtering.
The main issue with active collaborative filtering for TV shows is that viewers will only rate a show after watching it. And there has been limited success in getting users to build a sufficiently large database of information to provide solid recommendations.
Passive filtering is less problematic when collecting the data, but requires substantial processing in order to make the data attributable to a single user: viewing information is typically not usable, but information on which shows people have clicked on within the electronic programme guide (EPG) or any favourite shows that users have highlighted provide an excellent basis for passive filtering. The major disadvantage of passive filtering is that users cannot easily specify which information they want to have used for recommendations and which they don’t, so any information used for passive filtering must be carefully governed by a set of business rules to reduce the potential for inappropriate recommendations.
The final element to consider in passive filtering is the degree to which it amounts to user profiling. Many European countries have a strong culture of data privacy and every attempt to introduce any level of user profiling can result in a negative customer response
Content analysis
The second common approach to recommendations is to analyse the content and make recommendations based on similar elements in the content. For music content, there have been several highly sophisticated content-analysis algorithms that analyse audio for common features and provide recommendations based similarity of sound.
In the TV world, the only content-analysis technologies available to date rely on the metadata associated with the programmes. The recommendations are only as good as the metadata, and are typically recommendations within a certain genre or with a certain star.
The usual approach for extracting important information from content is either to use marked up metadata, with keywords such as actors’ names and directors or to use a term-frequency algorithm to automatically extract key words.
The advantage of content-analysis-based recommendations is that they provide quick recommendations across a wide range of content without relying on gathering user activity information.
The disadvantage is that TV metadata is generally not generated with automatic recommendations in mind and is frequently not suitable or provides irrelevant recommendations.
Social Recommendations
The idea that people like to watch TV shows that other people have suggested to them is the basis of social recommendations. This can involve friends, family, TV critics and the publishers of newspapers and periodicals. They differ from other recommendations in that they are not typically relevant to the user who is receiving the recommendation other than based on who has made the recommendation.
Social-networking technologies allow for a new level of sophistication whereby users can easily receive recommendations based on the shows that other people within their social network have ranked highly, providing a more personal level of recommendations than are achieved using a newspaper or web site.
A number of social networks dedicated to providing music recommendations have emerged over the last few years, the most well known of this being last.fm which encourages users to track all of their listening habits with the website and then applies a collaborative filtering algorithm to identify similar users and then ask them for recommendations.
With the emergence of sites like Facebook there are more personal services emerging with friends recommending specific programmes to each other, and this is likely to be a growth area as social networking tools become more sophisticated.
The advantage of social recommendations is that because they have a high degree of personal relevance they are typically well received, with the disadvantage being that the suggested shows tend to cluster around a few well known or cult-interest programmes.
An Integrated Approach to TV Show Recommendations
In order to integrate the different methods that can be used to derive recommendations it is necessary to derive a probabilistic map of the likelihood that a viewer might like a certain programme. The mathematical basis for this is Bayes’ theorem.
The basis of Bayes’ theorem is predicting the probability that one event will occur given another. In the case of TV show recommendations this is typically the probability that the user would like a show given that they like another.
A Bayes network is a set of such conditional probabilities mapped together. Figure 2 of a Bayesian network for the UK’s top TV shows during June 2009. In this case, the length of the line between the shows indicates the probability that a user who likes one show will equally like another.
Here we can see that some shows, like Doctor Who and The Bill act as popular nodes that have a high overall probability that users will like them because they are linked to many other shows.
Figure 2 - A bayesian network showing popular UK shows during June 2009
If we are to make recommendations based on a single show, we would highlight that show in the network and see which shows are adjacent to it, and then recommend these shows.
Figure 3 - Recommendations for Holby City
In Figure 3, the user likes the show Holby City, as shown below, then they can be recommended the show in grey, with Casualty, the Bill and Doctor Who being the most likely recommendations.
Figure 4 - Recommendations for Holby City and Eastenders
If we further know that the user likes EastEnders, then the recommendations can be refined with the common shows between these two becoming the most likely show that the user would like, as shown in figure 4.
In this case, the most likely common shows between Holby City and Eastenders are Coronation Street and The Bill, and these would be the best shows to recommend.
Thus with Bayesian network recommendations, a sophisticated level of suggestions can be provided on the basis of the TV shows that the user likes alone, with no need to profile the user against the demographics, or activity.
Indeed, using a Bayesian network, the best recommendations can be made simply from a list of favourite shows.
Building the Bayesian Network
A Bayesian network can be produced through a combination of the different approaches described in the previous section: collaborative filtering, content analysis, and social recommendations, as well as incorporating other sources of information such as demographic information, viewing figures or both.
In building the network, these sources of information will need to be combined together with different weightings according the quality of the source of the information.
TV Genius typically weighs any collaborative-filtering information based on favourite shows or active filtering as a strong driver in building the network with elements such as content analysis a weaker driver.
Once the network has been built it will normally need to be modified by business rules in order to prevent inappropriate recommendations and handle outliers. Inappropriate recommendations might typically be based around children’s- and adult-specific content and are typically handled through the metadata of the TV shows.
Outliers are difficult to handle and are typically new shows that do not have sufficient history for a basis of content analysis and have no commonality on the basis of content analysis with existing shows. These would typically be handled on the basis of social recommendations but there are occasions where new shows are seeded into Bayesian network manually to prioritise them from day one.
Key benefits
The main difference between using a Bayesian Network as the basis for recommendations and other approach is that all suggestions are made on the basis of television shows and in the context of TV service this has the following benefits:
- Transparency – there are no hidden variables behind how the recommendations are made. They are clearly associated with the shows that the user is looking at and/or has expressed an opinion about
- No barriers to entry – users do not need to enter any personal information or answer any questions before they receive recommendations. There is no need to monitor the user’s activity or store a profile centrally
- Speed to market – any collaborative-filtering approach requires a large amount of information in order to build the network before recommendations can be sensibly made. Because the Bayesian network does not contain any user information it can be shared between businesses and used to provide recommendations from day one
- Easy to maintain – with no need for complex business rules or user segmentation, recommendations stay relevant and current without a large administrative overhead.
What to look for in a solution provider
There is a proliferation of companies offering recommendations services to the TV Industry at the moment. When looking for a platform it is important to consider the following:
Do they already have a solid Bayesian network to apply to your content? Most providers are usually happy to share a snapshot of their Bayesian network under a non-disclosure agreement so you can get confidence that the service will be able to start making sensible recommendations from day one.
Is there a feedback mechanism between take-up and the network? If some shows are particularly popular on your service, can this be fed back into the network to promote these shows further?
Are there existing application programming interfaces (APIs) that you can trial to test the service? Trial the recommendations on a web or mobile service before you implement a full roll out.
Does it support promotion of individual content items? As new shows launch there are always budgets available to support promotion and any recommendations engine should take this into account
What reporting is available? You should expect if not real time, then at least daily reporting, showing the take-up of the recommendations, which are the shows that people are selecting the most and how this is driving take-up?
How much experience do they have? It takes about two to three years of real-world deployments for most suppliers to tune their recommendations engines to a stage where they can be reliably deployed anywhere. Make sure that you are not the first implementation of a new recommendations system unless you are sure it is much better than other ones.
How transparent is the recommendations algorithm to the user? Users don’t like it when technology does things that they don’t understand and are particularly sensitive to automatic profiling and targeted advertising. Make sure that the user experience makes it clear why the shows are being recommended to the user at all times.
How easy is it for the user to start using the service? The more barriers there are to start using the service, the fewer people will use it. Avoid services that require the users to answer questionnaires, however short, before they can get going.
How much overhead will there be in managing the service from your side? Do you want a recommendations engine that requires a team of back-end staff to manage the service or one that runs automatically based on user activity and feedback? Make sure you are clear on the staffing implications of the product you choose.
The TV Genius advantage
TV Genius is the premier provider of online TV discovery technology and the TV Genius content discovery platform is used by millions of people across Europe to plan their TV viewing. Designed from the ground up as a platform to build TV guides, it has unprecedentedly rapid time to market and the flexibility to deploy and modify services as viewing habits change.
The geniusRecommend product is available on the TV Genius Content Discovery Platform and has the following advantages:
- Simple implementation – a full recommendations service can be deployed with very low levels of systems integration, reducing the need for internal resources to support a deployment and reducing time to market
- Fully managed service – there is no overhead in updating the product as TV services evolve and a full 24×7 service-level agreement ensures any problems are resolved in as little as 15 minutes
- Extensive client base – with a proven client base, TV Genius can be trusted to deliver. A sample of clients includes AOL, Arena, Bauer, BSkyB, BZ Berlin, Daily Mail, Freeview, IPC Magazines and ITV
- Low cost of ownership – with little systems-integration cost and a fully managed service, the cost of deployment is much less than comparable solutions that require extensive customisation and management
- Shared data set – with a large number of customers across Europe, TV Genius has built a Bayesian network based on a wide range of existing usage patterns that provide a high quality service without a long period for training the system
TV Genius Customers
