Learning graphical models of preferences Theoretical results Projet BR4CP Damien Bigot, Yann Chevaleyre, Fr´ed´eric Koriche, J´erˆ ome Lang, J´erˆ ome Mengin, Bruno Zanuttini November 13, 2013

In order to present to the user, in a user-friendly way, the various options available at each stage of a configuration process, it is desirable to be able to predict the user’s behavior (i.e., her future choices) and hence to reason about her preferences on the items. Task 2 studies preference learning models and algorithms for configuration systems, by applying concepts and techniques coming from machine learning in order to build preference models of a user or group of users. The problem of eliciting preferences has received an increasing interest in the machine learning community. In configuration settings, the set of items on which the user’s preferences bear have a combinatorial structure, therefore the user’s preferences cannot be represented as a simple linear ranking function, but are characterized by a partial or total ordering on a high-dimensional feature space. We report here on some theoretical results on learning preference relations on combinatorial domains with structural dependencies. We focus on two models for representing such preferences: local utility functions, and conditional ceteris paribus preference rules. In the sequel, we consider a set of variables V; each variable X ∈ V has a finite domain that we denote by X, that we assume to be of size d. The items of interest are the elements of the cartesian product of the Q domains of the variables, we denote it by V: V = X∈V X.

Local utility functions The items can be ordered by means of a utility function, that assigns a real number to each item. Defining such a function by its extension is not realistic since its domain size grows exponentially with the number of variables. However, it is possible to define several sub-utilities, each local to a small set of variables, P and take their sum. Then an ordering over V is defined by: o  o0 if and only if u(o) = i ui (o[Vi ]) > P u(o0 ) = i ui (o0 [Vi ]), where each ui is characterized by a set of variables Vi ⊆ V, and where o[Vi ] denotes Q the projection of item o onto X∈Vi X. Such a decomposition of utility function u is called Generalized Addititive Independant. This is manageable if the definition of the ui s is compact, that is, if the size of each Vi , called the degree of ui , is small. This can be ensured by considering a bound k on the size of the Vi s: in the sequel we denote by GAIk the set of utility functions over V which admit an additive decomposition with sub-utilities of degree ≤ k.

Conditional ceteris paribus rules An ordering of the items can also be defined by means of so-called ceteris paribus rules, which state the ordering of the values of a given variable X ∈ V “all other things being equal”(ceteris paribus): given a partially defined item o ∈ V − X, users of a decision aid system will often be at ease to order the possible completions of o with values of X. The representation of the user preferences with such rules can be extremely compact if this ordering only depends on a small set of variables U ⊆ V − X. Then, a conditional ceteris paribus rules for X has the form u :>, where > is a total strict ordering of the values in X and u ∈ U : it indicates that if two items o and o0 are such that o[U ] = o0 [U ] = u, and if o and o0 are identical except that o[X] > o0 [X], then o must be preferred to o0 . A CP-net over V is a directed graph, the vertices of which are the variables in V, and where an edge from X to Y indicates that the ordering of the values in Y depends on the values of X. If Y has k incoming edges from variables X1 , . . . , Xk , then the CP-net also contains dk conditional ceteris paribus rules associated 1

with Y that describe the ordering of the values in Y for each combination of the values of the Xi s. This is manageable if each variable only has a small number of incoming edges. Note also that, for important technical reasons, it is usually assumed that the graph is acyclic. Therefore, in the sequel, we denote by CPNk the set of acyclic CP-nets over V where no variable has more than k incoming edges. Given a CP-net N , we denote by >N the transitive closure of the set of pairs of items ordered by all the conditional ceteris paribus rules of N . Note that >N is usually not a complete ordering.

Complexity of queries Before trying to learn a representation of a user’s preferences, it is interesting to know how easily this representation can be used afterwards, for recommendation purposes. Dominance A first type of queries that is often mentioned in the literature is the dominance query: given two items o and o0 , is it the case that o is preferred to o0 ? Clearly, answering such a query is easy with a GAI decomposition of a utility function u: one just has to compute and compare u(o) and u(o0 ). Dominance queries are hard to answer in general acyclic CP-nets, but can be solved in polynomial time for polytreeshaped CP-nets [4], and even in linear time for tree-shaped CP-nets [2]. Optimization A most useful task for recommendation is that of finding the best / most preferred item, or the p best items. It is hard in general when preferences are represented with GAI decomposition, but can be done efficiently if the corresponding GAI-net has small cliques; the computation can be sped up using an upper-approximation for GAI-nets with large cliques [12, 10]. Given an acyclic CP-net N , computing the optimal, that is, non dominated, outcomes, or computing the p most preferred items, is easy [4, 6]. However, the problem becomes hard when there are constraints on the set of feasible items [5, 6], unless the structure of the constraints is “compatible” with the structure of the CP-net.

Expressivity of the models Note that a CP-net does not in general yield a total ordering of the outcomes, only a partial one. And that not all total orders over V can be represented by a decomposition in GAIk or a CP-net. In machine learning, a measure of the expressivity of a set of functions for binary classification is its Vapnik-Chervonenkis dimension (VC-dim in the sequel): the higher the VC-dim of a set, the larger it is, and the more difficult it is to identify one of its elements. [1] show that the VC-dim of GAIk – when utility functions are considered as classifiers that associate to each pair of items (o, o0 ) ∈ V 2 an element of {, 6} – is in O(2k nk+1 ). An interesting constraint on the set of GAI decompositions is the number of sub-utilities used; then, the VC-dim of the set of GAI decompositions with at most l sub-utilities, each of degree no more than k, is l × 2k . [14, 8] show that the VC-dimension of CPNk is in Θ(n2k ) – again when a CP-net is considered as a binary classifier.

Passive learning Given a bound k on the degrees of the sub-utilities or on the in-degrees of the nodes of a CP-net, the learning problem is to identify which model in GAIk or CPNk can represent the preferences of a given user, or group of users. In a passive learning setting, we assume that we are given a set E of examples of user preferences, that is, a set of triples of the form (o, o0 , r), where o and o0 are two items of V and r indicates whether o is preferred to o0 , or strictly preferred, of if the converse holds, of if the user is indifferent between o and o0 . In this setting, the question is then: given a fixed bound k, can we efficiently find u ∈ GAIk such that u(o)ru(o0 ) for every (o, o0 , r) ∈ E? Or, can we efficiently find a CP-net N ∈ CPNk such that o >N o0 for every (o, o0 , ) ∈ E? GAI decompositions The answer is ”yes” for GAI decompositions: utility decompositions of degree bound by a (small) k are efficiently learnable, by translating each example into a linear equation, where the unknowns are the values of the sub-utilities to learn; see e.g. [13, 1].

2

A more difficult problem is an optimization one: given that the degree of a decomposition gives some insight into the dependance structure of the user, is it possible to find a decomposition with the smallest possible degree that explains a given set of preference examples E? [1] show that there is not always a ”simplest” decomposition, and explore some ideas to find one of them, by translating the set of examples into an integer linear programming problem. However, minimizing the size of the representation of a utility function is an NP-hard problem. CP-nets In the quite restrictive case of separable CP-net – with no dependencies between the variables – passive learning is tractable (but becomes NP complete if E contains negative examples, of the form o 6 o0 ) [15]. Passive learning of acyclic CP-nets is not tractable [9]. Additional results hold for learning CP-nets with more general structure from swap examples, that is, from example (o, o0 , r) where o and o0 are identical except for the value of one variable (in this case, each example gives one rule of the CP-net) : the class of tree-shaped binary CP-nets is PAC-learnable from swap examples, but finding a polytree-shaped CP-net that implies a set of swap examples, of finding one whose dependency graph is a chain, are NP complete problems [7, 8]. Note that if one wants to be attributeefficient, then learning tree-shaped binary CP-nets becomes untractable. When the structure of the CP net is known, finding tables that implies a set of examples for a CP-net with a given tree-structure is an NP-hard problem. However, [15, 2] show how learning a CP-net with a given tree-structure that implies a set of examples can be encoded as a SAT instances, which can be solved using efficient SAT solvers. Because CP-nets can only represent partial orders, and not all of them, one first has to decide what is meant by “learning a CP-net from a set of examples”. Finding a CP-net N that implies E (such that o >N o0 for every (o, o0 , ) ∈ E) could be impossible because not all partial orders can be represented by a CP-net, so it is often more reasonable to look for a lower approximation of the user’s preferences, that is, a CP net which is consistent with the examples [15].

Elicitation Interactive systems can be used to elicitate the current user’s preferences by asking her to pick some item from a given set, or rank a number of given items. In order to gain as much insight into the user’s preferences with as little effort from her as possible, it is crucial to compute item sets which will lead to the shortest interaction possible for learning the preference model. [14, 8] propose an elicitation algorithm for the class of acyclic (possibly incomplete) binary CP-nets, such that for any target CP-net of description size s, the algorithm makes at most s mistakes and uses at most slog2 n membership queries. [11] propose an algorithm to elicitate the values of the sub-utilities, given a known structure for the GAI decomposition.

Further questions We finish this report with a few questions that would be worth investating. • Queries: a most interesting type of query in the context of recommendation / configuration is to ask for the best partial item, given the values of some already known variables. • PAC learnability: learnability of the models has been studied with respect to comparison examples, involving two items. It would be interesting to study it with respect to the problem of ranking p items. • Unsupervised learning: since in general it is not feasible to elicitate the full preference model of a single user, recommender systems often use clustering techniques to assign a new user to a group of users with similar preferences. These methods need a notion of distance between users. There are known measures of similarity between orderings, it would be useful to find algorithms to effectively use them when the domain of items has a combinatorial structure and the preferences are represented with compact models like GAI decompositions or CP-nets.

3

Summary GAIk

CPNk

dominance queries

easy [4]

NP-hard polynomial with polytree shaped CP-nets [4] linear with tree-shaped CP-nets [2]

optimization queries

hard, unless corresponding GAI-net has bounded treewidth [10] O(2k nk+1 ) [1] O(l2k ) if l sub-utilities PAC-learnable [1]: bound on number of required examples function of 2k nk+1 tractable if known tree-structured GAI net [12]

easy if no constraint [4] NP-hard with feasibility constraints unless constraint structure “compatible” with CP-net [6]

VC-dim passive learning

elicitation

Θ(2k n) [8] hard translation to SAT if known tree structure [2] PAC-learnable from swap examples [8] |N | log n queries (where |N | = size of target CP-net)

References [1] D. Bigot, H. Fargier, J. Mengin, and B. Zanuttini. Using and learning gai-decompositions for representing ordinal rankings. In J. F¨ urnkranz and E. H¨ ullermeyer, editors, Preference Learning: Problems and Applications in AI. Proceedings of the ECAI 2012 workshop, pages 5–10, 2012. [2] D. Bigot, H. Fargier, J. Mengin, and B. Zanuttini. Probabilistic conditional preference networks. In A. Nicholson and P. Smyth, editors, Uncertainty in Artificial Intelligence: Proceedings of the 29th Conference. Association for Uncertainty in Artificial Intelligence, juillet 2013. [3] C. Boutilier, editor. Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI’09), 2009. [4] C. Boutilier, R. I. Brafman, C. Domshlak, H. H. Hoos, and D. Poole. CP-nets: a tool for representing and reasoning with conditional ceteris paribus preference statements. Journal of Artificial Intelligence Research, 21:135–191, 2004. [5] C. Boutilier, R. I. Brafman, C. Domshlak, H. H. Hoos, and D. Poole. Preference-based constrained optimization with cp-nets. Computational Intelligence, 20(2):137–157, 2004. [6] R. I. Brafman, F. Rossi, D. Salvagnin, K. B. Venable, and T. Walsh. Finding the next solution in constraint- and preference-based knowledge representation formalisms. In F. Lin, U. Sattler, and M. Truszczy´ nski, editors, Proceedings of the 12th International Conference on Principles of Knowledge Representation and Reasoning (KR’10). AAAI Press, 2010. [7] Y. Chevaleyre. A short note on passive learning of cp-nets. Rapport de recherche, Lamsade, March 2009. [8] Y. Chevaleyre, F. Koriche, J. Lang, J. Mengin, and B. Zanuttini. Learning ordinal preferences on multiattribute domains: the case of CP-nets. In J. F¨ urnkranz and H. H¨ ullermeier, editors, Preference learning, pages 273–296. Springer, 2011. [9] Y. Dimopoulos, L. Michael, and F. Athienitou. Ceteris paribus preference elicitation with predictive guarantees. In Boutilier [3]. [10] J.-P. Dubus, C. Gonzales, and P. Perny. Fast recommendations using gai models. In Boutilier [3], pages 1896–1901.

4

[11] C. Gonzales and P. Perny. Gai networks for utility elicitation. In Proceedings of the Ninth International Conference on the Principles of Knowledge Representation and Reasoning, pages 224–233. AAAI Press, 2004. [12] C. Gonzales and P. Perny. Gai networks for decision making under certainty. In Multidisciplinary IJCAI-05 Workshop on Advances in Preference Handling, pages 100–105, 2005. [13] T. Joachims. Optimizing search engines using clickthrough data. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, July 23-26, 2002, Edmonton, Alberta, Canada, pages 133–142. ACM, 2002. [14] F. Koriche and B. Zanuttini. Learning conditional preference networks with queries. In Boutilier [3]. [15] J. Lang and J. Mengin. The complexity of learning separable ceteris paribus preferences. In Boutilier [3], pages 848–853.

5