A Knowledge Representation and Logic for Beliefs Area Keywords: Knowledge representation, non-monotonic reasoning, probabilistic reasoning, uncertainty

Abstract In this paper we present Belief Augmented Frames or BAFs, a system that combines knowledge representation with non-monotonic reasoning. A BAF agent represents the objects and concepts of the world as frames. Each frame however is augmented with two belief values; one belief value indicates how much he beliefs in the existence or truth of the object or concept, and the other indicates how much he beliefs that the object does not exist or that the concept is not true. This approaches allows a BAF Agent to separately evaluate arguments for and against a proposition, and frees the agent from the traditional statistical assumption that p(¬P) = 1 – p(P). A BAF Agent likewise forms associations between frames represented as slot-value relationship pairs, again augmented with two belief values representing the belief in and against the validity or truth of the relationship. This paper also presents BAF-Logic, a system of logic for reasoning over BAFs and relationships between BAFs, and analyzes the semantics of reasoning and frame operations. It closes by citing some application examples.

1

Introduction

The reasoning over non-monotonic knowledge and beliefs in agents has been extensively researched. One of the earliest attempts at modeling beliefs and knowledge is found in [Hintikka, 1962]. The seminal AGM model is proposed in [Alchourron, Gardenfors and Makinson, 1985] to model the change in an Agent’s belief state upon learning a fact A. Others have added uncertainties to the belief states in the form of statistical measures. [Milch and Koller, 2000] proposes Probabilistic Epistemic Logic (PEL). PEL is parameterized by a set of random variables, and a system of logic is defined to perform reasoning. Algorithms are provided to convert the PEL into Bayesian Networks that represent the Agent’s beliefs in a compact form. [Bacchus, 1990] developed on Kripke style [Kripke, 1963] semantics by adding a statistical function to the accessibility relation between possible worlds. In this way he provides for graded degrees of belief, and he argues that this simplifies the semantics by

providing a simple statistical function that maps between possible worlds, instead of a complex accessibility relation. Unlike statistical functions, belief functions express beliefs as a range of probability values instead of as a singlepoint probability. By representing beliefs as a range rather than a single point, belief functions allow us to represent degrees of ignorance in the beliefs. This is particularly useful when making a subjective opinion, where a confidence of p that a belief is true does not necessarily entail a confidence of (1 – p) that it is false. [Shortliffe and Buchanan, 1985] cites such an example, where a doctor diagnosing that a patient is suffering from a particular disease with a confidence of p, is reluctant to diagnose with a confidence of (1 – p) that the patient is not suffering from the disease. Dempster-Shafer Theory (DST) uses Dempster’s work on upper and lower probabilities [Dempster, 1967], extended by Shafer in [Shafer, 1976], to model beliefs. The degree of belief in P is given by the Evidential Interval EI, defined as [Bel(P), Pls(P)], where Bel(P) is the degree of belief in P, and Pls(P)is the degree of plausibility of P. This range expresses the current and maximum possible belief values in P, and the ignorance Ig(P) is given by Pls(P) – Bel(P). This is extended to the open world assumption in [Smets, 2000] to produce the Transferable Belief Model. Smets also argues that an Agent cannot effectively make decisions using belief values, and he proposes a pignistic function that transforms the belief values to a form that is more suited to decision making. In [Picard, 2000] Picard combines statistical measures with propositional logic to form the Probabilistic Argumentation System or PAS. While PAS uses single point probabilities to indicate the degree of truth of a proposition, Picard provides formulations to derive the evidential interval of the proposition, and is hence able to model ignorance. In BAFs beliefs values are separated into two measures. ϕTP is the supporting measure, which indicates how much an agent believes that P is true. ϕFP is the refuting measure, which measures how much an agent believes that P is false. The primary purpose of separating these values is to allow separate chains of reasoning for and against P. A system of logic called BAF-Logic is defined to reason over these values. These values are in turn held within frames [Minsky, 1975], which provide a knowledge representation structure to the beliefs. Combining frames with belief measures also

allows us to formalize the semantics of these beliefs within a knowledge representation structure, which will be covered in a later work.

2

An Introduction to Belief Augmented Frames

We now formally look at Belief Augmented Frames. We begin with some definitions related to BAFs, and we look at BAF-Logic, the reasoning system for BAFs. We will refer to our BAF agent as “Fred”.

2.1 Definitions A BAF state S is defined by a tuple (F, R), where F is the set of frames representing all the concepts and objects known to Fred, and R is the set of relations between the frames in F. Each frame fA∈F is represented as fA, where fA is the name of the frame, and ϕTA and ϕFA are the supporting and refuting belief values that Fred holds for fA. fA can take the value if fA which means that Fred is fully confident that the object or concept represented by fA exists or is true, or it can take a value of fA, which means that Fred is fully confident that it is false. Both ϕTA and ϕFA may also take intermediate values in the range of [0, 1], which then represents various degrees of belief for and against the existence of the object or the truth of the concept. In BAFs however, generally ϕTA + ϕFA ≠ 1. I.e. ϕTA and ϕFA may be independent of each other, freeing us from the classic probability relation that ϕFA = 1 − ϕFA and leaving us space to model ignorance. While ϕTA, and ϕFA measure how much Fred believes that A is true or false, his overall belief in A is given by the Degree of Inclination DI:

DI A = ϕ TA − ϕ AF

(1)

Since 0 ≤ ϕTA, ϕFA ≤ 1, therefore –1 ≤ DIA ≤ 1, with DI = -1 represented absolute falsehood, DI = 1 representing absolute truth, and intermediate values of DI representing various degrees of truth and falsehood. A possible interpretation for DI is shown in Table 1 Value Interpretation -1 FALSE -0.75 Most probably false -0.5 Likely false -0.25 Somewhat false 0.0 Undecided 0.25 Somewhat true 0.5 Likely true 0.75 Most probably true 1 TRUE Table 1. Possible Interpretations for DIA.

We can manipulate IgA to give:

Ig A = 1 − (ϕ TA + ϕ AF )

(4)

In this form the semantics of IgA are clarified: The ignorance IgA measures the amount in which the support and refutation belief values fail to fully explain A. The utility function UA maps DIA to the range [0, 1] to allow it to be used conveniently for decision making: (5) UA = (1+DIA)/2 UA can be used as a probability measure if all the utility functions are normalized to sum to 1. Therefore the utility function UA also serves to bridge between the belief functions in BAFs and standard probabilistic models. A relationship r(A, B) is represented by a slot-value pair within the frame fA, relating it to frame fB by the relationship r. ϕTr(a,b), and ϕFr(a,b) are the supporting and refuting values for this relationship. Again, a value of r(A,B) indicates the Fred is absolutely certain that the relationship r between fA and fB is true, while r(A, B) indicates that he is absolutely certain that it is false. Analogous versions of equations 1 to 5 are defined on these values. A BAF-State S may also be understood as Fred’s epistemic state, since it contains all the concepts that Fred knows about, and the relationships between them.

2.2 BAF-Logic BAF-Logic defines a number of inference rules to perform reasoning over the belief values. Let B be the set of BAFLogic statements. Then: i) ii) iii)

fA∈ B r(A, B)∈ B If P∈B and Q∈B, then P∧Q ∈ B, P∨Q ∈ B and ¬P ∈ B.

For the rest of this section we will assume that P and Q are valid BAF-Logic statements. The soundness of these rules is proven in [Tan 2003]. The inferred support and refutation values for a BAF-Logic statement is marked with a “*”, for e.g. ϕTP, *, to differentiate it from an already known belief value ϕTP for P. ¬-Introduction P ¬P,* T

The plausibility of A is given by:

Pl A = 1 − ϕ AF

This means that the maximum belief that Fred is willing to place on A is inversely related to his disbelief in A. While the plausibility tells us the maximum belief that Fred is willing to place on A, the ignorance in A is given difference between this and ϕTA, the actual belief that Fred has in A: (3) Ig A = Pl A − ϕ TA

(2)

∨-Elimination-1 This rule specifies how negation is to be carried out in BAFLogic.

P∨Q P,* Q,*

¬-Elmination P ¬¬P,* T

This is the double-negation rule, and can be trivially proven from the ¬-Introduction inference rule.

This inference rule is a mirror of ∧-elimination, and follows directly from the ∨-introduction rule. If an existing Q This means that when Fred needs to rely on two propositions P and Q to be true, the maximum belief he will place on P ∧ Q is just the minimum belief in P and Q, since once either P or Q is false, P∧Q is false immediately. Consequently the belief he places in P∧Q being false is the maximum of the refutation values of P and Q. ∧-Elimination

From the ∨-introduction inference rule, ϕTP∨Q = max(ϕTP, ϕ and ϕFP∨Q = min(ϕFP, ϕFP). Since DIQ≤ -0.5, we know that ϕTQ < ϕFQ. However DIP∨Q ≥ 0.5, thus ϕTP∨Q ≥ ϕFP∨Q. Thus it is not possible that ϕTP∨Q = ϕTQ and ϕFP∨Q = ϕFQ, and we can infer that in this situation, ϕTP, * = ϕTP∨Q and ϕFP, * = ϕFP∨Q. T Q)

Modus Ponens P→Q P DIP→Q ≥ 0.5 DIP ≥ 0.5 T Q,*

P∧Q P,* Q,* T P∧Q,

F

Since ϕTP∧Q = min(ϕTP, ϕTQ) from the ∧-introduction rule, it follows that ϕTP, * and ϕTQ, * are at least as large as ϕTP∧Q. Likewise since ϕFP∧Q = max(ϕFP, ϕFQ), it follows that ϕFP, * and ϕFQ, * are at most as large as ϕFP∧Q. If Fred already knows P with belief P and if T ϕ P and ϕFP fall outside the range set by ϕTP,* and ϕFP,*, Fred’s knowledge is said to be inconsistent. Similarly if the product DIP.DIP, * < 0, Fred’s knowledge is said to be contradictory as the degrees of inclination if P and P,* are of different signs and hence of different truth values. We will deal with inconsistencies and contradictions later in this paper.

Q is true only if P and P→Q are both true. Fred’s belief that Q is true is thus only as strong as the weakest of P and P→Q. Modus Tollens P→Q Q DIP→Q ≥ 0.5 DIQ ≤ -0.5 P,*

∨-Introduction P Q P∨Q,* P∨Q becomes true as long as either P or Q is true, and hence Fred’s belief in P∨Q is as strong as the strongest of the two BAF statements P and Q. Similarly for P∨Q to be false, both P and Q must also be false. Fred’s confidence that P∨Q is false is thus as strong as the weakest confidence that P or Q is false.

From Modus Ponens, we have ϕTQ = min(ϕTP→Q, ϕTP) and ϕ Q = max(ϕFP→Q, ϕFP). Since DIQ≤-0.5, we see that min(ϕTP→Q, ϕTP) < max(ϕFP→Q, ϕFP). However we know that ϕTP→Q > ϕFP→Q since DIP→Q ≥ 0.5 and therefore for DIQ≤ 0.5 to be possible, min(ϕTP→Q, ϕTP) = ϕTP and max(ϕFP→Q, ϕFP)=ϕFP. We thus conclude that ϕTP,* = ϕTQ and ϕFP,* = ϕFQ. Since propositions in BAF-Logic have two independent belief and refutation values, BAF-Logic is best suited for applications where arguments may be formulated for an against a proposition. BAF-Logic is then able to reason over both chains of arguments to derive a conclusion. F

2.3 Inconsistencies and Contradictions In a non-monotonic BAF-Agent like Fred, an inconsistency between an inferred belief P,* and a known belief P occurs when DIP.DIP,* ≥ 0, but ϕTP and ϕFP fall outside the range imposed by ϕTP,* and ϕFP,*. For example, suppose we are given the following: P, P→Q, Q This is an inconsistency, as the Modus Ponens inference rule will give us a supporting belief ϕTQ = 0.8, which is bigger than ϕTQ,* = min(0.7, 0.6) = 0.6. Likewise the refuting belief ϕFQ = 0.0, which is less than ϕFQ,* = max(0.2, 0.1) = 0.2. We propose that BAF Agents like Fred are conservative believers who will take the lesser of the two inconsistent beliefs (i.e. the less optimistic one) as being the correct one. Formally: Erosion-1 P P,* ϕTP,* ≥ ϕTP ϕTP,* = ϕTP Erosion-2 P P,* ϕ FP, * ≤ ϕ FP ϕFP,* = ϕFP Erosion-3 P P,* ϕTP,* ≤ ϕTP ϕTP = ϕTP,* Erosion-4 P P,* ϕ FP, * ≥ ϕ FP ϕFP =ϕFP,* These four erosion rules, taken together result in the more optimistic version of P (whether the P already stored in the knowledge base, or P,* obtained from the inference rules above) being reduced to the less optimistic value. Contradictions are dealt with by the belief updated function which we will now look at.

2.4 Belief Update

To illustrate the belief update function, we now augment all belief masses with a time stamp t representing the current time, and t+1 representing the time of the next belief value. Thus Pt is the belief in P at the present time, while Pt+1 is the new belief in P after incorporating the newly inferred P,*. We can obtain the new belief masses Pt+1 by using:

ϕ PT , t +1 = min(ϕ PT , t + αϕ PT ,* ,1) ϕ PF,t +1 = max(ϕ PF,t − αϕ PT ,* ,0) ϕ PT , t +1 = max(ϕ PT , t − αϕ PF,* ,0)

ϕ

F P , t +1

= min(ϕ

F P ,t

+ αϕ ,1) F P ,*

(6a) DI P ,* > 0

(6b) (7a)

DI P ,* < 0

(7b)

These equations reinforce the belief in P when DIP,* is greater than 0, and erode it when DIP,* is less than 0. When DIP,* = 0, P. I.e. if the inferred proposition has an undecided truth value, it is ignored. To make the reinforcement/erosion proportional to the truth value DIP,*, we set α = |DIP,*|. This will update the belief values proportionally to Fred’s confidence in P,*. After seeing sufficient evidence of P being true or false, this function will result in Fred eventually believing that P is completely true or completely false. Contradictions (defined to be when DIP,* . DIP < 0) are handled automatically by our belief update function. When DIP > 0 and DIP,* < 0, ϕTP is reduced while ϕFP is increased, making DIP decrease towards the negative range consistent with DIP,*. Eventually after several updates, DIP will be less than 0, and DIP,*.DIP will be a positive number. The contradiction is thus eliminated. When DIP < 0 and DIP,* > 0, ϕTP is increased while ϕFP is decreased , making DIP increase towards the positive range consistent with DIP,*. The contradiction is similarly eliminated after several updates. This may not always be the best way to deal with a contradiction; Fred does have other avenues, like re-confirming a conflicting conclusion with another Agent, before choosing to update his beliefs.

2.5 Frame Operations Several operations are defined on BAFs. As with the case of the inference rules stated earlier, all operations may have logical consequences. This will be dealt with in the next section. Add a New Frame/Update Frame The operation +fA adds a new frame fA to the frame set F if it does not exist, or changes the belief values of fA if it does. The belief values are updated to the values

provided by this operation. Any BAF-Logic statement involving fA will be re-evaluated with fA. Delete a Frame The operation –fA removes the frame fA from F. If the relationship r(B,A) exists, where fB ∈ F, it is automatically removed. Any BAF-Logic statement involving fA will be re-evaluated with fA, i.e. with fA undefined. Add a New Relation/Update Relation The operation +r(A,B)< ϕTr(A,B), ϕFr(A,B)> creates a relation r between frames fA and fB. If fA ∉ F, then +fA is automatically performed if DIr(a,b) ≥ 0.5, in the set of relationships R. Semantically, Fred is creating the concept of a new and previously unknown object, on the basis that since the relationship exists, the object is also assumed to exist. If the relationship already exists, its belief values are updated to the values provided. Any BAF-Logic statement involving r(A,B) will be re-evaluated with r(A,B). Delete a Relation The operation –r(A,B) removes the relation r(A,B) from R. Any BAF-Logic statement involving r(A, B) will be reevaluated with r(A,B). Generalize from a set of Frames The function gen(S) returns a frame fg with the following properties: i)

ii)

fg. Since the generalized frame fg exists if all of the frames in S exist, Fred’s confidence that fg exists is taken to be the conjunction of the existence values of all the frames in S. Similarly, r(g, B)

The operation +fg is automatically performed.

2.6 Dealing with Logical Consequence The BAF-Logic inference rules in section 2.2, erosion rules in section 2.3, and the frame operations in section 2.4 may result in many other BAF-Logic statements being reevaluated. Since Fred is a realistic resource-bounded agent, he evaluates statements lazily. For example, given the following simple relation: A→B→ C When the belief in A is modified, by modus ponens, the belief in B is also modified. Since our belief in B is modified, by modus-ponens, our belief in C is modified.

When the belief in A is modified, instead of updating the belief in B, Fred merely marks B for updating, while C is left alone. B’s belief values are updated only if they are used, and in turn C is marked for updating, but is not updated unless its belief values are actually used. This assumes that Fred has perfect recall (i.e. he can always recall past inferences). This is in itself potentially resource intensive and needs to be looked into. The epistemic and logical implications of this strategy have yet to be evaluated, and this will also be deferred to a later work.

2.7 Initialization of Belief Values Fred faces a critical issue the first time he instantiates a frame (+fA, fA ∉ F) or a relation (+r(A,B), r(A,B)∉ R) in his mind: What should he choose as initial belief values? Fortunately he has several choices: Expert Opinions. Expert opinions can be used for the initial belief scores. For example, a doctor may believe that, given no other information, a patient has a 10% chance of having cancer and an 80% chance of not having cancer. Then we can have cancer as an initial assignment. Linguistic Hedges. An agent may be “quite certain” or a “little unsure” about the truth of a proposition. We can then use fuzzy-style approximations [Zadeh, 1965] to model these linguistic hedges. This was the approach used in [Tan and Lua, 2003b] to model discourse. Sensors Suppose we have an array of n sensors to detect the presence of n different objects. Let P be the proposition that object p is present, and let sensor(x) return a number bounded by [0,1] indicating the likelihood of object x being present. The belief masses for P might be:

ϕTP = sensor(p)

ϕFP= ∑i ≠ p n

sensor(i )

(8a) (8b)

n −1

Default Values If we are not given any information on a particular proposition, we might opt to incorporate default values. To do this we can define “categorical default” rules [Antonelli, 1996]. For example: X ¬Y ∉ B Y

This means that if the set of BAF-Logic statements entails X and does not entail ¬Y, we are entitled to infer Y with belief values ϕTY, ϕFY.

4 Applications Early forms of BAF and BAF-Logic have been applied to dialog management [Tan and Lua 2003a], discourse comprehension [Tan and Lua 2003b], and text classification [Tan 2004]. [Tan 2004] is particularly interesting as experiment results presented there show that BAF-Logic is able to classify text documents better than approaches like Naive Bayes, K-Nearest neighbour, maximum entropy and expectation maximization.

5 Summary and Future Work This paper presents Belief Augmented Frames (BAFs) and a reasoning system called BAF-Logic, including the inference rules of BAF-Logic, frame operations and a proposal to minimize computational complexity. We are currently studying the epistemic implications of BAFs and BAF-Logic, especially in relation to Kripke possible world semantics, and the interaction of BAFs with right-brain techniques like neural networks and markov models. We also plan to extend BAFs and BAF-Logic to the temporal domain to allow Fred to reason over future events.

References [Alchourron, Gardenfors, Makinson, 1982] Carlos Alchourron, Peter Gardenfors, David Makinson. On the Theory of Change: Partial Meet Contraction and Revision Functions, The Journal of Symbolic Logic, (50):510-530, 1985. [Antonelli, 1996] Gian Antonelli. Defeasible Reasoning as a Cognitive Model. Uppsala Prints and Reprints in Philosophy, (18):1-13, Uppsala University, 1996. [Bacchus, 1990] Fahiem Bacchus. Probabilistic Belief Logics. In Proceedings of the European Conference n Artificial Intelligence (ECAI-90), pages 56-94, Stockholm, Sweden, 1990. [Dempster, 1967] Arthur Dempster. Upper and Lower Probabilities Induced by a Multi-Valued Mapping, Annals of Mathematical Statistics, (38):325-339, 1967. [Hintikka, 1962] Jaakko Hintikka, Knowledge and Belief, Cornell University Press, Ithaca, New York, 1962. [Kripke, 1963] Saul Kripke. Semantic Considerations on Modal Logic, Acta Philosophica Fennica, (16):83-94, 1963. [Milch and Koller, 2000] Brian Milch, Daphne Koller. Probabilistic Models for Agents’ Beliefs and Decisions In Proceedings of the 16th Annual Conference on Uncertainty in Artificial Intelligence, pages 389-396, Stanford, California, June 2000.

[Minsky, 1975] Marvin Minsky. A Framework for Representing Knowledge. Reprinted in The Psychology of Computer Vision, McGraw-Hill, 1975. [Shafer 1976] Glenn Shafer. A Mathematical Theory of Evidence, Princeton University Press, 1976. [Shortliffe and Buchanan, 1985] Edward Shortliffe, Bruce Buchanan, “A Model of Inexact Reasoning in Medicine”, Rule Based Expert Systems, pages 232-262, 1985. [Smets, 2000] Phillipe Smets. Belief Functions and the Transferrable Belief Model. Web Link:

http://ippserv.rug.ac.be/documentation/belief/beli ef.pdf, 2000. [Picard, 2000] Justin Picard, Probabilistic Argumentation Systems Applied to Information Retrieval, Doctoral Thesis, Universite de Neuchatel, Suisse, 2000. [Tan, 2003] Colin Tan, Belief Augmented Frames, Doctoral Thesis, National University of Singapore, Singapore, 2003. [Tan and Lua, 2003a] Colin Tan, Lua Kim Teng. Belief Augmented Frames for Knowledge Representation in Spoken Dialogue Systems. In Proceedings of the First Indian International Conference on Artificial Intelligence (IICAI-03), Hyderabad, India, 2003. [Tan and Lua, 2003b] Colin Tan, Lua Kim Teng. Discourse Understanding with Discourse Representation Structures and Belief Augmented Frames. In Proceedings of the 2nd International Conference on Computational Intelligence, Robotics and Autonomous Systems (CIRAS-2003), 2003, Singapore. [Tan, 2004] Colin Tan. Text Classification using Belief Augmented Frames. In Proceedings of the 8th Pacific Rim International Conference on Artificial Intelligence (PRICAI-04), Auckland, New Zealand, 2004. [Zadeh, 1965] Lofti Zadeh. Fuzzy Sets. Information Control, pages 338-353, 1965.