The Duine Framework is a (collection of) software libraries that allows developers to create prediction engines for their own applications. A prediction engine is a component that predicts how interested individual users are in pieces of information. Such predictions can be used to personalise information to users, specifically in recommending to users what information is and is not of interest to them.
Duine is the Irish Gaelic word for person and is pronounced as “dinne” (dinner without an ‘r’).
Duine has been developed by the Telematica Instituut/Novay and is based on scientific research on personalisation and specifically into recommender systems. The scientific research results on which the Duine software is based are available at the Telematica Instituut/Novay, you can download the pdf here .
The Duine recommender is a software module that calculates how interesting information items are for a user. The resulting interest is quantified by a number, called prediction, ranging from -1 (absolutely not interesting) to +1 (definitely interesting). When applied in, for example, an electronic TV Guide the Duine recommender can calculate how interesting each TV program is for a particular user. These predictions can be used in various ways: e.g. to provide a user with a list of the top 10 most interesting items, to sort a list of items with the items with the highest prediction at the top, or to present an indication of the interest to the user for each item (e.g. using a number of stars).
The Duine recommender also processes and stores ratings that users give to an information item and interests of the users in aspects of the information (categories, genres, people, topics etc). All data associated with a user is stored in a user profile.
The Duine recommender has learning capabilities. When a user rates an information item, the recommender extracts data from this item (e.g. keywords or genres of a TV program description) to determine the interests of the user. By using smart learning algorithms the recommender slightly adapts the user profile after each rating, based on these interests.
The figure below gives on overview of the Duine features. The features and terminology will be explained in more detail below and in other sections of this website.
The recommender calculates predictions by executing algorithms that reason about the current user and the information item, resulting in a prediction value. These algorithms are called prediction techniques. The prediction techniques use user profiles and information items as input for their calculation. Examples of prediction techniques are:
Collaborative Filtering (also called social filtering) : People who have rated the same items the same way in the past probably have the same taste. Based on this knowledge one can predict how much a person likes an unseen item when similar users have already rated that item.
Information Filtering : Extracts information from an item (e.g. the description of a TV program), does a textual analysis and matches the result with the user profile to determine how interesting the item is.
The Duine Recommender currently offers 7 prediction techniques (including the 2 mentioned above). The other techniques are: Case-based Reasoning (based on how similar items have been rated by the user in the past), GenreLMS (reasoning on genres), TopNDeviation (popularity), AlreadyKnown and UserAverage .
The Duine Recommender is a so called hybrid recommender, which means that it combines multiple prediction techniques instead of relying on one specific technique. A hybrid recommender is better than a single-algorithm recommender:
1. It provides more accurate predictions
2. It reduces the cold-start problem
Every prediction technique has its own strengths and weaknesses. The accuracy and usability of a technique largely depends on the kind of data that is available in the user profiles, the number of users and the metadata that is available for the items. For example, collaborative filtering requires that a sufficiently large number of other users have already rated an item for which a prediction is needed. Information filtering requires the existence of user profiles that can be matched against metadata in an item. A hybrid recommender checks these conditions, and dynamically switches for each prediction that is to be made to the technique that provides the best result given the current state of data in the user profiles. The decision rules that govern the switching behavior are called prediction strategies.
The cold-start problem refers to the situation where a recommender does not have enough information about a user or an item to make good recommendations. During a cold start situation a hybrid recommender switches to a simpler prediction technique like TopNDeviation (popularity). These may not be as accurate as more complex techniques, but they provide better results than no predictions and can often learn quicker than those complex techniques. Later, when more ratings and user profiles become available prediction strategies automatically switch to more complex and accurate prediction techniques.
Prediction techniques and prediction strategies can be wired together by means of configuration. This does not require any software coding. For each application domain the configuration will have to be adapted to the specific characteristics of that domain. For example, a recommender for TV programs might need another configuration than a recommender for theatre and concerts. The final accuracy of the recommender is largely determined by the optimization of the configuration towards the application domain.
Extensibility : The Duine recommender can be extended with new prediction techniques and strategies, with new profile models and with feedback processors. Furthermore the provided spring hibernate persistency implementation can be replaced by another implementation, and you can replace the JCS cache by another one.
Validation : The Duine recommender framework comes with tools for validation. Given a dataset with ratings by users, the validation framework has tools to measure the accuracy of the predictions. The validation framework can be used to optimize the recommender configuration towards a specific application domain.
Explanation API : Duine can explain how the prediction result was determined. This explanation API can be used to create end-user friendly explanations in applications.
