UnBBayes: a Java Framework for Probabilistic Models in AI Shou Matsumoto1 , Rommel N. Carvalho2 , Marcelo Ladeira1 , Paulo Costa2 , La´ecio Santos1 , Danilo Silva1 , Michael Onishi1 , Emerson Machado1 1

Departamento de Ciˆencia da Computac¸a˜ o Universidade de Bras´ılia, Brazil 2

Center of Excellence in C4I George Mason University, USA

1

Introduction Systems that think like humans, systems that act like humans, systems that think rationally, and systems that act rationally. Four categories of AI’s definitions, adapted from (Russel & Norvig, 2002).

In the past few years, the way software development is performed in research and academia has changed dramatically. One of the major driving forces causing this phenomenon is the growing complexity of Artificial Intelligence (AI) systems as they face increasingly demanding requirements for real-world application development. As usability (ease of use), productivity (engineering efficiency), portability (multi-platform compatibility) and interoperability (ability to operate harmonically) become a concern in developing AI systems, Java™ stands out as the programming language of choice in many AI applications due to its advantages to complex systems design, such as its effective implementation of the object-oriented paradigm to develop large, portable, GUI1 -enabled software projects quickly, as well as its natural support to the ubiquitous development of web technologies for information exchange. As Java™ popularity surges, several domain independent architectures supporting the design of knowledge intensive reasoning systems are being developed using the language (e.g. jCOLIBRI2 (Garcia, 2008), Prot´eg´e3 1 GUI stands for Graphical User Interface, a type of interface that allows users to interact with programs by images rather than text commands. 2 jCOLIBRI is an open-source framework for Case Based Reasoning (CBR), maintained by the Group for Artificial Intelligence Applications (GAIA), Dpt. of Software Engineering and Artificial Intelligence at Complutense University of Madrid. 3 Prot´eg´e is an open-source suite of tools to construct domain models and knowledge-based applications with ontologies (a representation of a knowledge, structured in a special manner), maintained by Stanford Center for Biomedical Informatics Research at the Stanford University School of Medicine.

UnBBayes: a Java Framework for Probabilistic Models in AI

(Noy et al., 2000)). One example is UnBBayes4 , an open-source framework that provides a graphical means of building probabilistic knowledge models for real-world applications. This Chapter invites you to understand the historical background of UnBBayes, its architectural underpinnings, and the major technical challenges behind its development. Further, it presents suggestions on its usage as a framework and API5 for domain modeling and probabilistic reasoning. This Chapter is conceptually divided in three parts: overview of the tool and its architecture, presentation of the API, and a description of the plug-in infrastructure. This initial Section outlines the challenges that originally motivated the development of UnBBayes. Section 2 describes the core structure of this tool, as well as its development milestones and overall design. Section 3 focuses on the API usage, mostly related with Bayesian Networks (BN) and Influence Diagrams (ID) modeling. Finally, Section 4 describes UnBBayes infrastructure for plug-ins, which allows the creation of extension points for unforeseen features. Some familiarity with Unified Modeling Language (UML) is desirable, in order to comprehend some design issues throughout this Chapter. All domain examples were modeled on UnBBayes. It’s worth noticing that this Chapter is not intended to presenting fundamental theoretical principles of probabilities and its underlying concepts. However, as the UnBBayes’ class structure relies on Bayesian Networks and Influence Diagrams abstractions within a few points, a basic overview of BN and ID will be introduced respectively in Subsections 1.2 and 1.3 to facilitate the comprehension of the system’s overall architecture. Hopefully, this Chapter will serve as a basis on understanding and appreciating the capabilities and applications of Java™ to research and academia. 1.1

Problem Description - Modeling Uncertainty Using Probabilities “Uncertainty is ubiquitous. Any representation scheme intended to model real-world actions and processes must be able to cope with the effects of uncertain phenomena.” (Costa, 2005)

Not surprisingly, many real-world domains are structured in a manner that information is not always complete, consistent or immutable over time, as some facts may be unknown (theoretical ignorance), unrevealed (practical ignorance, caused by the lack of suggestive evidences) or too coarse (lacking sufficient detail). Therefore, a means of dealing with uncertainty on hypotheses becomes necessary. Historically, numerical6 approaches to express uncertainty have gained attention thanks to their ability to finely grade plausibility within contexts, and to specify a reasoning process using well defined calculi. Several models have been adopted by researchers and engineers to express uncertainty, as the Dempster-Shafer’s belief functions (Dempster, 1967; Shafer, 1976) and some controlled use of rule-based systems with Certainty Factors (Buchanan & Shortliffe, 1984). Fuzzy logic models also have been increasingly popular for representing and processing degrees of truth about imprecise or vague pieces of linguistic information, and became a strong candidate for incorporating impreciseness derived from applications using natural language. Finally, probabilistic models7 are also a very strong candidate that provide sound representation and sound calculus for rational degrees of belief or likelihood of a proposition. 4 UnBBayes’

project page: http://sourceforge.net/projects/unbbayes/. stands for Application Program Interface, an interface implemented by a software program that enables it to interact with other software. 6 A numerical approach represents uncertainty using numbers (either singular values or intervals) to represent degrees of certainty (i.e. a statement may be more or less likely to be true). Normally, these values are in a range between 0 and 1 and operations are applied to allow for inferences. 7 The interpretation of probability may vary from school to school (e.g. frequentists interpret probabilities as relative frequency in a large amount of trials), but in this Chapter we refer “probability” using Bayesian evidential interpretation, which is a measure of a state of knowledge (e.g. degrees of belief). 5 API

Shou Matsumoto, et al.

The use of graphical probabilistic models has become an appealing way of enabling information systems to coherently exploit incomplete information, due to its ability to express knowledge as interrelated modular components, improving tractability and facilitating the knowledge engineering process. Bayesian Networks (BN) (Pearl, 1988), a graphical, flexible means of expressing joint probability distributions over interrelated hypotheses, is one of the most promising probabilistic approaches to representing uncertainty. UnBBayes was designed to provide a suite of tools to easily construct and manipulate domain knowledge based on BN and other related probabilistic models. Generally speaking, academic research related to information technology is well known for its evolutionary nature, where the state of art is largely formed by extensions and enhancements from previous work. AI research is no different in this aspect. If a single AI system lives for a long lifespan (including its successive enhancements), it is natural that it will attract a greater number of researchers, mentors, pupils, or developers, resulting into a very fast-paced development cycle. As new theoretical principles emerge and requirements become more specific and demanding at a higher pace, language support to archive reusability8 and variability9 with minimal learning effort becomes a major aspect when deciding which platform to use for developing an AI system. The UnBBayes’ project team has chosen Java™ as the system’s primary language due to the strong support for reusability (thanks to its implementation of the object-oriented paradigm), to the possibility of developing and executing on different computer architectures (thanks to the Java Runtime Environment), and to the availability of high-quality libraries to archive load-time or runtime variability (see Section 4). 1.2

Bayesian Networks

As UnBBayes’ class structure relies heavily on Bayesian Network’s structure at some points, it would be nice to have a brief understanding of the BN’s theoretical model from now on. A Bayesian Network (BN) (Pearl, 1988) is a graphical model based on Bayesian probability. It consists of a directed acyclic graph and a set of local probability distributions. Each node in the graph represents a random variable10 , and the edges represent direct qualitative dependencies of the random variables. The local probability distribution is a function (usually represented as a Conditional Probability Table - CPT) specifying the quantitative information about the strength of the dependencies. Each random variable has a set of possible values, and exactly one of the possible values will be the actual value at a given moment (but we are uncertain about which one). The graph and the local distributions together are a compact representation of a joint probability distribution over the random variables. Reasoning systems based on Bayesian models rely on Bayes rule, which provides a method for updating the probability of a proposition when information is acquired about a related proposition. In the Bayesian view, inference is a problem of belief updating as new information becomes available. The standard format of Bayes rule is: P(X|Y ) =

P(Y |X)P(X) , P(Y )

P(X|Y ), which is the posterior probability of event X given event Y . P(Y ) is the a priori probability of Y , and represents our belief in event Y before obtaining evidence. Similarly, P(X) is the a priori probability of X. P(Y |X) is the likelihood of event Y given that event X has occurred. Figure 1 illustrates the classic “Asia” example from the medical domain (Lauritzen & Spiegelhalter, 1988), in which the task is to identify a chest disease based on evidence from a patient. The probabilities shown at the top figure are the “a priori” probabilities, that is before any evidence. Basically, it states that (a) a resent visit 8 Reusability

is a quality of a program (or its source code) of being able to be used again with subtle or no modifications. is a quality of program (or its source code) of being subject to variation with subtle or no effort. 10 A random variable denotes a set of attributes or hypotheses which we may be uncertain. 9 Variability

UnBBayes: a Java Framework for Probabilistic Models in AI

to Asia increases the risk of tuberculosis due to a recent outbreak of that disease in the region, (b) smoking is a risk factor for bronchitis and lung cancer, (c) tuberculosis or cancer can cause positive X-Ray results, (d) all three diseases may cause dyspnoea11 , (e) and tuberculosis and cancer cannot be distinguished by the X-Ray results or the presence/absence of dyspnoea. Figure 2 specifies the CPT of HasDyspnoea, giving us a quantitative information about the effects of bronchitis, tuberculosis and/or lung cancer on dyspnoea.

Figure 1: A Bayesian network for the classical “Asia” domain (on the top). The same BN after entering evidences/findings is presented at the bottom. It was modeled on UnBBayes.

Figure 2: CPT of HasDyspnoea.

One of the key strengths of reasoning in BN is its ability to update the beliefs of each random variable via bi-directional propagation of new information through the whole structure12 . The bottom side of Figure 1 illustrates the same BN after discovering that the patient is a smoker presenting dyspnoea. The posterior probability distribution states that the patient is likely to have bronchitis (88.02%), compared to the probabilities 11 Dyspnoea

or dyspnea is also known as the shortness of breath. default, the UnBBayes’ Bayesian inference engine uses the junction tree algorithm (Jensen et al., 1990b; Jensen et al., 1990a), but further extensions can be added by using plug-ins (see Section 4.2). The latest works have also provided other inference algorithms, as Likelihood Weighting and Gibbs Sampling 12 By

Shou Matsumoto, et al.

of lung cancer (14.83%) and tuberculosis (1.54%). Bayesian networks have been successfully applied to create consistent probabilistic representations of uncertain knowledge in various fields, such as language understanding (Charniak & Goldman, 1989), image recognition (Booker & Hota, 1986), and medical diagnosis (Spiegelhalter et al., 1989). Additionally, several extensions have been proposed to boost up BN’s expressiveness (e.g. Dynamic Bayesian Networks (Murphy, 2002), Object Oriented Bayesian Networks (Koller & Pfeffer, 1997), Multi-Entity Bayesian Networks (Laskey, 2008)), suggesting BN to be a very solid basis for plausible reasoning in domain-independent knowledge systems. 1.3

Influence Diagrams “An influence diagram is a way of describing the dependencies among aleatory variables and decisions.” (Howard & Matheson, 1981)

Influence Diagrams (ID) were formally proposed in (Howard & Matheson, 1981) and are a compact graphical representation of a decision-making problem where uncertainty is modeled with probability. The technique can be used to determine the expected consequences of a series of decisions (or “actions”). It represents the domain as a directed acyclic graph (as BNs do) and comprises the following elements: • Probabilistic nodes - random variables representing uncertain information; • Decision nodes - nodes representing a set of possible choices of actions; • Utility nodes - these are nodes13 with no children and they represent utility (policy or preference) functions14 , which represent the amount of gains (or losses, if negative) that can be obtained from a particular set of actions or evidence; • Conditioning influences - arrows leading into probabilistic or utility nodes, which represent respectively conditional probability dependencies or inputs of an utility function; • Informational influences - arrows leading into a decision node, which shows exactly what variables shall be known by the decision makers at the time that decision is made15 . Figure 3 illustrates a classic ID known as “The Used-Car Buyer”, adapted from the one presented in (Shachter, 1990). This example represents a situation where a person wants to decide whether to buy a certain used car which may be a nice “peach” or an unpleasant “lemon”. He (or she) has the option of doing (or not doing) some tests beforehand (Test1 and Test2), and of buying it with a guarantee or not (Buy). The utility node (Profit) specifies an utility function which indicates the estimated gain (or loss) according to the value of InitialCondition and all three preceding actions.16 In a typical optimization problem (i.e. decision making problems following a maximum expected utility criterion), the main objective is to determine a sequence of actions that maximize the expected value of this utility function. 13 If

there are more than one utility node in an ID, they must represent additive contribution to the utility function. utility function is represented in UnBBayes as a table with a single row, in which the columns represent the combination of the possible values of a set of inputs. A value (which is a number) in a cell represents a gain (or a loss, if the gain is negative) expected for a particular combination of the input’s values. By doing this way, it is easier to specify utility functions as rules like “if my previous decisions were yes-yes-no and the value of the random variable “X” was false, then my gain is 1400”. 15 Unlike conditioning influences, the informational influences represent basic cause/effect ordering of information. 16 In this example, Profit encapsulates some rules like: “if the car is lemon and the last actions were transmission, differential and buy guaranteed, then the estimated profit is 26 units of profit”, “if the car is peach and the last actions were fuel/elect, differential and buy, then the estimated profit is 51 units of profit”, etc. 14 An

UnBBayes: a Java Framework for Probabilistic Models in AI

Figure 3: An Influence Diagram for the classical “The Used Car Buyer” domain (on the top). Test1, Test2 and Buy are the decision nodes, the larger boxes with belief bars are the probabilistic nodes, and the diamond at the lower right corner is an utility node. The same ID after entering evidence is presented at the bottom.

The evaluation (i.e. the reasoning process) of an ID is basically a combination of Bayesian inference and utility’s estimation, based on a sequence of evidences and actions17 . Table 1 shows a brief comparison between the estimated value of Profit in the initial configuration (see the top side of Figure 3) and after deciding to perform a transmission test and discovering that the car is a lemon and has one defect (see the bottom side of Figure 3). According to the resulting ID, the best actions to take (those with the highest utility) are: do a differential test and buy it guaranteed. Decision Trees (or Decision Graphs (QI & Poole, 1993)) can also represent models with decisions, consequences and probabilistic variables. Although the techniques are logically equivalent, IDs are more often chosen as a compact alternative18 , thus building up a considerable reputation in the field of decision analysis. Prospective users may feel interested in running UnBBayes19 in order to visualize the concepts presented is this Section.

17 In

ID’s terminology, an action is a decision that was made. Trees typically suffer from a problem of exponential growth of branches to each variable in the model. 19 UnBBayes can be downloaded at http://sourceforge.net/projects/unbbayes/. 18 Decision

Shou Matsumoto, et al.

Node Test1

Test2 Buy

Actions no test steering fuel/elect transmission no test differential do not buy buy buy guaranteed

Before 25.2 38 57.4 -82 57.4 50.8 57.4 50.8 30.8

After Taken -14 26 -113 -110 26

Table 1: Estimation of Profit before (top side of Figure 3) and after (bottom side of Figure 3) inserting evidences and actions.

2

Overview of UnBBayes

As it was previously stated, UnBBayes is an open-source Java™ application developed by the Artificial Intelligence Group (GIA)20 from the Computer Science Department at the Universidade de Bras´ılia21 in Brazil that provides a framework for building probabilistic graphical models and performing plausible reasoning. It features a graphical user interface (GUI), an application programming interface (API), as well as plug-in support for unforeseen extensions. It offers a comprehensive programming model that supports the exploitation of probabilistic reasoning and intrinsically provides a high degree of scalability, thus presenting a means of developing AI systems on the fly. Figure 4 displays a screen shot of the tool.

Figure 4: Screen shot of several features (e.g. internal windows) of UnBBayes. 20 GIA 21 The

stands for Grupo de Inteligˆencia Artificial. university website is http://www.unb.br/.

UnBBayes: a Java Framework for Probabilistic Models in AI

Three goals have driven the design and development of the latest versions of UnBBayes: (1) being a operative platform for the dissemination of the concepts and usefulness of probabilistic reasoning, (2) being an easy-to-use and configurable visual tool, and (3) achieving extensibility and variability. We achieve the first goal by implementing a state-of-art probabilistic graphical formalism - BN - and a default22 inference engine based on the junction tree’s algorithm (Jensen et al., 1990b; Jensen et al., 1990a; Jensen & Jensen, 1994). The second goal is reached by implementing a GUI to enable an intuitive mean for editing graphical models and to perform inferences, thus offering an integrated environment for knowledge engineering. Localization was facilitated with the use of resource files, which are chosen automatically depending on the host operational system’s locale. Finally, the third objective is accomplished by taking advantage of Java’s™ implementation of the object-oriented paradigm (thus, improving encapsulation23 and establishing well-defined extension points) and by adopting a plug-in design. The project team offers a repository that already includes plug-ins for Bayesian network (BN), Influence Diagram (ID), Multiple-Sectioned Bayesian Network (MSBN), Hybrid Bayesian Network (HBN), ObjectOriented Bayesian Network (OOBN), Probabilistic Relational Model (PRM), Multi-Entity Bayesian Network (MEBN), Probabilistic Web Ontology Language (PR-OWL), parameter learning, structure learning, and incremental learning of BNs, as well as statistical data sampling, classification performance evaluation, data mining, and several other algorithms for Bayesian inference. Figure 5 illustrates a common usage flow of UnBBayes’ core24 . 2.1

A Brief History

Although the Java™ version of UnBBayes has officially started back in 2001, it was originally written in Delphi™ and its root was leading back to year 2000, when a series of studies had been conducted by GIA/UnB25 (the Artificial Intelligence Group at the University of Brasilia) aimed at creating a support system for medical diagnosis. The following chronological list summarizes the milestones of this project. 2000 Project SARA (Sa´ude Apoiada em Racioc´ınio Automatizado - Health Supported by Automated Reasoning) has started in order to support medical diagnosis in the hospital of University of Brasilia. 2000 The initial version of UnBBayes (Soares, 2000) was implemented in Delphi™ 5.0 (Imprise Corp) in order to provide a computational tool for project SARA. 2001 A Java translation of UnBBayes started (Carvalho et al., 2002). 2003 BN learning algorithms were included to UnBBayes (da Silva, 2003). 2003 ID and MSBN (Xiang et al., 1993) support were included (Carvalho & Onishi, 2003). 2005 UnBMiner (Ladeira et al., 2005) - a variation of UnBBayes for data mining purposes - was published officially. 2006 Medical Metaphor26 - an interface designed for medical diagnosis - was finished. 2007 MEBN (Laskey, 2006) support was included (Carvalho et al., 2007). 22 There

are other inference algorithms implemented in UnBBayes as well. is an information hiding mechanism that allows us to limit the impact of a program change by restricting access to a potion of code or data which is likely to change. 24 The core is the application with minimum features (i.e. no connected plug-ins). 25 GIA/UnB stands for Grupo de Inteligˆencia Artificial / Universidade de Bras´ılia. 26 A Metaphor is a simpler user-interface that hides some details of a BN model, but allows the insertion of evidences and verification of the results. 23 Encapsulation

Shou Matsumoto, et al.

Figure 5: A sample usage flow of UnBBayes’ core (BN module) and the GUI.

2008 Human Identification Metaphor - an interface designed for “Human Identification” domain (a domain in which the program must estimate the sex of a person through the characteristics of the skull) - was included. 2008 OOBN (Koller & Pfeffer, 1997) support was included. 2010 Plug-in support was included. All previous variations of UnBBayes were refactored as new plug-ins. 2010 PRM (Getoor et al., 2001) support has started. As mentioned in the above chronological list, the UnBBayes’ team has been working on incorporating new BN formalisms into the tool lately, as new BN extensions were proposed time after time.

UnBBayes: a Java Framework for Probabilistic Models in AI

2.2

The Overall Design

UnBBayes is mostly structured in a Model-View-Controller (MVC27 ) design pattern28 , which explicitly separates the program’s elements into three distinct roles, in order to provide separation of concern (i.e. the software is separated into three different set of classes with minimum overlap of functionality). Figure 6 illustrates the overall design of UnBBayes in a MVC style.

Figure 6: A UML class diagram illustrating the MVC design of UnBBayes, in package level.

Although UnBBayes is written in Java™, which is a typical object-oriented (OO) programming language29 , a “pure” object-orientation (which is a design where programs are composed by a set of autonomous and fairly complete elements representing real or imaginary objects, performing communication by exchanging messages) is rarely applied in UnBBayes’ overall design. Instead, UnBBayes follows an adaptation of componentoriented paradigm, which organizes pieces of codes (or a set of Java objects/classes) into a set of somehow detachable and replaceable elements - i.e. “components” (this is similar to electronic components - e.g. an IC30 chip in a circuit board). A single component is usually related to a behavior of a piece of the software (thus, it is not always an intuitive representation of something in our problem’s domain) and then different components can be composed together in order to create a bigger entity representing an observable feature in a system (e.g. the composition of a network’s renderer, a network’s data structure, a network’s data entry form, and some other components can result together in a network’s editor). Designs based on component-orientation provides high level of variability and reusability, which are the major requirements mentioned in Subsection 1.1. Customizations in component-oriented models can be performed just by substituting the components (usually a set of objects or classes) to different ones (e.g. if we need to change the shape of a node, we should simply replace the object that renders it), thus there is no need to “open” a class and change lines of code. 27 A

MVC design isolates logic and data from the user interface, by separating the components into three independent categories: Model (data and operations), View (user interface) and Controller (mostly, a mediator, scheduler, or moderator of other classes) (Buschmann et al., 1996). 28 Design patterns are a set of generic approaches aiming to avoid known problems in software engineering (Gamma et al., 1994). 29 The concept of Object Oriented (OO) programming was originally introduced in Smalltalk (Goldberg & Kay, 1976). 30 IC stands for Integrated Circuit.

Shou Matsumoto, et al.

Figure 7 illustrates a symbolic difference between a “pure” object-orientation and a component-orientation approach in representing a simplified network editor. This concept of writing classes that are less resilient to changes (of lines of code), but more open to extensions (i.e. a structured substitution without having to access the original source code) is sometimes called open-closed principle (Meyer, 1997) and it is a fundamental basis for the adoption of component-based design in object-oriented languages. If the open-closed principle must be extended to a run-time binding, then a plug-in31 design can be adopted (see Section 4 for details about the plug-in design of UnBBayes).

Figure 7: Differences between “pure” object-oriented design (in the left) and component-oriented design (in the right). As component-oriented design focuses on reuse, objects/classes in the model may not be very intuitive.

The core module of UnBBayes provides two major functionalities: a base infrastructure (e.g. windowing support, storage access, preference management, plug-in support) and basic BN manipulation functionalities (e.g. storage, inference, and evaluation). The following packages compose the core module of UnBBayes (Figure 6 has presented their overall dependencies): • unbbayes.controller: it contains controller classes of MVC model. • unbbayes.draw: this package contains adapters that determine how a graph (e.g. nodes, edges) is drawn in a canvas. It enables several drawing functionalities, including resizing, coloring, shading, moving, etc. • unbbayes.evaluation: it contains classes for evaluating BN. 31 In

this chapter, the term “plug-in” has the same meaning specified in (Olshansky, 2007).

UnBBayes: a Java Framework for Probabilistic Models in AI

• unbbayes.gui: it contains GUI classes (e.g. panels, windows, forms, tables). • unbbayes.io: it contains classes for storage. E.g. storing BN, configurations, preferences or logs as files. • unbbayes.prs: it contains classes representing data structures and algorithms (i.e. model classes in MVC design). Conceptual elements (e.g. Node, Edge, Network, JunctionTree, PotentialTable) and operations on them are placed in this package. That is why it is called PRS, from Probabilistic Reasoning System. Classes in unbbayes.prs.bn represent a BN, classes in unbbayes.prs.id represents an ID, and classes in unbbayes.prs.hybridbn represent a hybrid BN, which includes continuous nodes (a random variable with continuous numeric value). • unbbayes.example: it contains classes exemplifying the usage of UnBBayes as API. See Section 3 for details. • unbbayes.util: this package unifies utility classes, which are general purpose classes used through the entire program. Classes placed here provide features like: advanced collection manipulation, debugging, and abstract classes for design patterns. • *.resources: packages with this name pattern contain the resource classes, which provide localization functionality. The UnBBayes’ class loader32 can automatically select the appropriate resource class, depending on the host computer’s configuration. See Subsection 4.4 for further details. • *.extension: packages with this name pattern contain classes implementing functionalities related to the plug-in infrastructure. See Section 4 for more details. • *.exception: packages with this name pattern contain classes representing exception classes (Sun Microsystems, Inc., 2000). Subsections 2.2.1 and 2.2.2 provide diagrams detailing the class structure in a MVC context. Irrelevant classes were hidden from these models for sake of visualization. 2.2.1

View and Controller Classes

The following list gives a brief description of the classes illustrated in Figure 8: • NamedWindowBuilder: this is a builder33 which instantiates an internal window. Each window instantiated by these builders will have an identifier (name) and it will be placed in MDIDesktopPane. This class eliminates direct dependencies from MainController to the classes in the BN module (e.g. NetworkWindow). • NetworkWindowBuilder: a builder for NetworkWindow. • MainController: this is a controller responsible for creating, loading and saving (general) networks supported by UnBBayes. It also handles system configuration. • UnBBayesFrame: this is an extension of swing34 JFrame responsible for the UnBBayes main panel (window/frame). 32 A Java class loader is an object that is responsible for locating or generating data that constitutes a definition for the class. It determines how Java virtual machine dynamically loads classes. 33 A builder is a interface (or class) following Builder design pattern (Gamma et al., 1994) to abstract the creation of objects of another classes. 34 Swing is a Java graphical framework used by UnBBayes’ GUI. See http://download.oracle.com/javase/tutorial/ uiswing/ for more details.

Shou Matsumoto, et al.

Figure 8: Classes of UnBBayes (core) implementing the View and Controller roles in an MVC model.

• MDIDesktopPane: this is an extension of swing JDesktopPane that supports the often used Multiple Document Interface35 (MDI) functionality (i.e. multiple inner windows within a single parent window). This class also handles setting scroll bars for when internal windows move too far to the left or bottom of the main window. • NetworkController: this is a controller responsible for triggering operations in the BN module (e.g. node insertion, evidence propagation). • NetworkWindow: this is an internal frame where BN edition is performed. • PNCompilationPane: this is a panel in NetworkWindow responsible for displaying a compiled BN and offering a means to insert and propagate evidences. • GraphPane: this is a panel in NetworkWindow responsible for displaying a graph representing the currently edited BN. • PNEditionPane: this is a panel in NetworkWindow responsible for offering tools to edit the BN. • UShape: this class is responsible for rendering a node in a canvas. Thus, it encapsulates the model class Node (see Subsection 2.2.2 for the model classes). 35 The

name “Multiple Document Interface” is typically used in Microsoft Windows™ environments.

UnBBayes: a Java Framework for Probabilistic Models in AI

• UCanvas: this class represents a canvas where graphs are painted. • UShapeLine: this class is responsible for rendering an edge in a canvas. Thus it encapsulates the model class Edge (see Subsection 2.2.2 for the model classes). 2.2.2

Model Classes

Classes of this category do not have routines to render themselves graphically (thus, these classes represent in-memory data). The following list gives a brief description of the classes in Figure 9:

Figure 9: Classes of UnBBayes (core) implementing the Model (data types and its operations) roles in a MVC model.

• Graph: common interface for a graph built on a set of nodes and edges • Network: concrete implementation of a generic network. If a network is composed of probabilistic nodes, using a ProbabilisticNetwork (an extension of Network) would be useful. • Edge: this is a class representing an edge between two nodes. By modeling relationships as a separate class, it becomes possible to use attributes, thus allowing differential treatment by other classes (e.g. how the GUI should display a relationship between two nodes depending on the edge’s attributes). Technically, it is possible to create relationships (dependencies) between instances of Node without using instances of Edge, but in such case no arcs linking these two nodes will be rendered in the canvas. By doing so, it becomes possible to model “hidden” relationships. • INode: interface representing a generic node. • Node: abstract class representing a node. It contains additional information, such as: name, label, description, coordinates, flags36 . 36 Flags

are boolean variables representing a state of an event (if it happened or not).

Shou Matsumoto, et al.

• TreeVariable: this is an abstract class for variables that will be displayed in the left side’s swing JTree in the compilation panel. After compiling a BN, nodes that are not extending this class will be partially ignored in PNCompilationPane. • ProbabilisticNode: it represents a probabilistic node (i.e. a node having a probabilistic assignment of values). • UtilityNode: this is an ID’s utility node. • UtilityTable: this is a table representing an ID’s utility function (which is represented as a table in UnBBayes’ model). • DecisionNode: this is an ID’s decision node. • IProbabilityFunction: this is a common interface for objects specifying a node’s probabilistic distribution. • PotentialTable: this is an abstract class representing IProbabilityFunction in a table-like format. • ProbabilisticTable: this class represents a BN’s CPT. Objects (instances) of unbbayes.gui.table.GUIPotentialTable can be used to render objects of this class graphically.

3

Application Program Interface Overview

Java™ offers an easy mechanism to reuse classes. By incorporating Java bytecode files (i.e. “.class” files or their packaged “.jar” files) into another program’s class path, public classes and methods become available to that program as a library. As UnBBayes is released as a “.jar” file, adding it to another program’s class path will enable access to its API37 . The following Subsections (3.1 and 3.2) present a sample usage of UnBBayes as an API for manipulating BN and ID respectively. The classes presented in the previous sections (mostly in Subsection 2.2.2) are applied in this section. A complete description of the API can be found as a Javadoc (Sun Microsystems, Inc., 2004) documentation in http://unbbayes.sourceforge.net/apidocs/. 3.1

The API for BN

The sample program38 in Listing 1 illustrates the usage of UnBBayes’ API as a BN manipulation tool. Initially, a BN is loaded from a NET file (Hugin Expert, 2010) (the obtained BN represents the “Asia” domain presented in Subsection 1.2), then a node and an edge are inserted in the model. Subsequently, the network is compiled in order to become ready for evidence (finding) insertion and propagation. The program prints out to the console a list of all nodes in the Asia example and the respective probabilities of their states, before and after the propagation of evidence. Readers may want to refer again to Subsection 1.2 for a theoretical review of BN. The following classes were borrowed from UnBBayes’ API (in appearance order): unbbayes.prs.bn.ProbabilisticNetwork (a structure representing a probabilistic network model), unbbayes.io.NetIO (responsible for loading NET format files), unbbayes.prs.bn.ProbabilisticNode (it represents a node with associated probability distribution), unbbayes.prs.bn.PotentialTable (a CPT), unbbayes.prs.Edge (a directed link between nodes), unbbayes.prs.bn.JunctionTreeAlgorithm (it contains routines implementing junction tree algorithm), and unbbayes.prs.Node (a generic node). 37 Artifacts 38 A

(classes or JAR files) distributed as plug-ins can also be used in a similar fashion to offer an API-like access point. more complete version of this sample code can be found in the package unbbayes.example.

UnBBayes: a Java Framework for Probabilistic Models in AI

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52

p u b l i c s t a t i c v o i d main ( S t r i n g [ ] a r g s ) throws E x c e p t i o n { / / open a . n e t f i l e ProbabilisticNetwork net = ( P r o b a b i l i s t i c N e t w o r k ) new NetIO ( ) . l o a d ( new F i l e ( ” . / e x a m p l e s / bn / n e t / a s i a . n e t ” ) ) ; / / a d d i n g a new node m a n u a l l y P r o b a b i l i s t i c N o d e newNode = new P r o b a b i l i s t i c N o d e ( ) ; newNode . setName ( ”K” ) ; newNode . s e t D e s c r i p t i o n ( ”A t e s t node ” ) ; newNode . a p p e n d S t a t e ( ” S t a t e 0 ” ) ; newNode . a p p e n d S t a t e ( ” S t a t e 1 ” ) ; P o t e n t i a l T a b l e auxCPT = newNode . g e t P r o b a b i l i t y F u n c t i o n ( ) ; auxCPT . a d d V a r i a b l e ( newNode ) ; n e t . addNode ( newNode ) ; / / search for ”HasVisitedAsia” P r o b a b i l i s t i c N o d e asiaNode = ( P r o b a b i l i s t i c N o d e ) n e t . getNode ( ” H a s V i s i t e d A s i a ” ) ; / / a d d i n g a new e d g e f r o m ” H a s V i s i t e d A s i a ” t o new node n e t . addEdge ( new Edge ( a s i a N o d e , newNode ) ) ; / / f i l l i n g CPT o f newNode auxCPT . a d d V a l u e A t ( 0 , 0 . 9 9 f ) ; auxCPT . a d d V a l u e A t ( 1 , 0 . 0 1 f ) ; auxCPT . a d d V a l u e A t ( 2 , 0 . 1 f ) ; auxCPT . a d d V a l u e A t ( 3 , 0 . 9 f ) ; / / prepare the algorithm to compile network J u n c t i o n T r e e A l g o r i t h m a l g = new J u n c t i o n T r e e A l g o r i t h m ( ) ; alg . setNetwork ( net ) ; alg . run ( ) ; / / p r i n t node ’ s i n i t i a l s t a t e s f o r ( Node node : n e t . g e t N o d e s ( ) ) { System . o u t . p r i n t l n ( node . g e t D e s c r i p t i o n ( ) ) ; f o r ( i n t i = 0 ; i < node . g e t S t a t e s S i z e ( ) ; i ++) { System . o u t . p r i n t l n ( node . g e t S t a t e A t ( i ) + ” : ” + ( ( P r o b a b i l i s t i c N o d e ) node ) . g e t M a r g i n a l A t ( i ) ) ; } } / / i n s e r t e v i d e n c e ( f i n d i n g ) t o t h e 1 s t node o f ” n e t ” P r o b a b i l i s t i c N o d e findingNode = ( P r o b a b i l i s t i c N o d e ) net . getNodes ( ) . get ( 0 ) ; f i n d i n g N o d e . a d d F i n d i n g ( 0 ) ; / / t h e 1 s t s t a t e i s now 100% // insert likelihood f l o a t l i k e l i h o o d [ ] = new f l o a t [ newNode . g e t S t a t e s S i z e ( ) ] ; likelihood [0] = 1f ; likelihood [1] = 0.8 f ; newNode . a d d L i k e l i H o o d ( l i k e l i h o o d ) ; / / propagate evidence net . updateEvidences ( ) ; / / p r i n t u p d a t e d node ’ s s t a t e s System . o u t . p r i n t l n ( ” **** U p d a t e d **** ” ) ; f o r ( Node node : n e t . g e t N o d e s ( ) ) { System . o u t . p r i n t l n ( node . g e t D e s c r i p t i o n ( ) ) ; f o r ( i n t i = 0 ; i < node . g e t S t a t e s S i z e ( ) ; i ++) { System . o u t . p r i n t l n ( node . g e t S t a t e A t ( i ) + ” : ” + ( ( P r o b a b i l i s t i c N o d e ) node ) . g e t M a r g i n a l A t ( i ) ) ; } } }

Listing 1: An example of API usage for manipulating BN.

Shou Matsumoto, et al.

3.2

The API for ID

UnBBayes also offers an API for modeling and reasoning over ID39 . Listing 2 illustrates the creation of a very simple ID from scratch. The resulting ID contains one decision node, one probabilistic node, one utility node, an edge connecting the decision node to the probabilistic node, and another edge connecting the probabilistic node to the utility node. The constructed network will then be compiled and an action will be performed. The program prints out to the console the values of the probabilistic node (the probability of each possible value) and the decision node (estimated value returned by the utility function). 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40

p u b l i c s t a t i c v o i d main ( S t r i n g [ ] a r g s ) throws E x c e p t i o n { / / c r e a t e a network from s c r a t c h P r o b a b i l i s t i c N e t w o r k n e t = new P r o b a b i l i s t i c N e t w o r k ( ” IDSample ” ) ; / / a d d i n g a new d e c i s i o n node D e c i s i o n N o d e d e c i s i o n = new D e c i s i o n N o d e ( ) ; d e c i s i o n . setName ( ” MyDecision ” ) ; d e c i s i o n . s e t D e s c r i p t i o n ( ”A d e c i s i o n node ” ) ; d e c i s i o n . appendState ( ” Action 0” ) ; d e c i s i o n . appendState ( ” Action 1” ) ; ; n e t . addNode ( d e c i s i o n ) ; / / a d d i n g a new p r o b a b i l i s t i c node P r o b a b i l i s t i c N o d e probNode = new P r o b a b i l i s t i c N o d e ( ) ; probNode . setName ( ” MyProbNode ” ) ; probNode . s e t D e s c r i p t i o n ( ”A p r o b a b i l s t i c node ” ) ; probNode . a p p e n d S t a t e ( ” S t a t e 0 ” ) ; probNode . a p p e n d S t a t e ( ” S t a t e 1 ” ) ; P o t e n t i a l T a b l e auxCPT = probNode . g e t P r o b a b i l i t y F u n c t i o n ( ) ; auxCPT . a d d V a r i a b l e ( probNode ) ; n e t . addNode ( probNode ) ; / / a d d i n g an e d g e f r o m d e c i s i o n node t o p r o b a b i l i s t i c node n e t . addEdge ( new Edge ( d e c i s i o n , probNode ) ) ; / / s p e c i f y i n g CPT auxCPT . a d d V a l u e A t ( 0 , 0 . 9 f ) ; auxCPT . a d d V a l u e A t ( 1 , 0 . 1 f ) ; auxCPT . a d d V a l u e A t ( 2 , 0 . 2 f ) ; auxCPT . a d d V a l u e A t ( 3 , 0 . 8 f ) ; / / a d d i n g a new u t i l i t y node U t i l i t y N o d e u t i l i t y = new U t i l i t y N o d e ( ) ; u t i l i t y . setName ( ” M y U t i l i t y N o d e ” ) ; u t i l i t y . s e t D e s c r i p t i o n ( ”An u t i l i t y node ” ) ; u t i l i t y . getProbabilityFunction ( ) . addVariable ( u t i l i t y ) ; n e t . addNode ( u t i l i t y ) ; / / a d d i n g an e d g e f r o m p r o b a b i l i s t i c node t o u t i l i t y node n e t . addEdge ( new Edge ( probNode , u t i l i t y ) ) ; // editing u t i l i t y function u t i l i t y . getProbabilityFunction ( ) . setValue (0 , 50); u t i l i t y . g e t P r o b a b i l i t y F u n c t i o n ( ) . s e t V a l u e ( 1 , −100); / / prepare the algorithm to compile network J u n c t i o n T r e e A l g o r i t h m a l g = new J u n c t i o n T r e e A l g o r i t h m ( ) ; alg . setNetwork ( net ) ;

39 By default, the UnBBayes’ ID reasoner uses an adapted version of the junction tree algorithm (Jensen & Jensen, 1994). As mentioned previously in Subsection 1.3, IDs are, in short, an extension of BN to offer a probabilistic decision support tool. As in BN, further algorithms can be added by implementing plug-ins (see Subsection 4.2).

UnBBayes: a Java Framework for Probabilistic Models in AI

41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67

alg . run ( ) ; / / p r i n t node ’ s i n i t i a l s t a t e s f o r ( Node node : n e t . g e t N o d e s ( ) ) { System . o u t . p r i n t l n ( node . g e t D e s c r i p t i o n ( ) ) ; f o r ( i n t i = 0 ; i < node . g e t S t a t e s S i z e ( ) ; i ++) { i f ( node i n s t a n c e o f T r e e V a r i a b l e ) { System . o u t . p r i n t l n ( node . g e t S t a t e A t ( i ) + ” : ” + ( ( T r e e V a r i a b l e ) node ) . g e t M a r g i n a l A t ( i ) ) ; } } } / / i n s e r t decision ( take the 1 st action ) ( ( T r e e V a r i a b l e ) n e t . g e t N o d e ( ” MyDecision ” ) ) . a d d F i n d i n g ( 0 ) ; / / propagate evidence net . updateEvidences ( ) ; / / p r i n t u p d a t e d node ’ s s t a t e s System . o u t . p r i n t l n ( ” *** P r o p a g a t i o n *** ” ) ; f o r ( Node node : n e t . g e t N o d e s ( ) ) { System . o u t . p r i n t l n ( node . g e t D e s c r i p t i o n ( ) ) ; f o r ( i n t i = 0 ; i < node . g e t S t a t e s S i z e ( ) ; i ++) { i f ( node i n s t a n c e o f T r e e V a r i a b l e ) { System . o u t . p r i n t l n ( node . g e t S t a t e A t ( i ) + ” : ” + ( ( T r e e V a r i a b l e ) node ) . g e t M a r g i n a l A t ( i ) ) ; } } } }

Listing 2: An example of API usage for manipulating ID.

Any operation in ProbabilisticNode (e.g. evidence or likelihood entry) can be performed as shown previously in Subsection 3.1. The following classes are the new ones borrowed from UnBBayes’ API in order to implement ID functionalities (shown in the same order they appear): unbbayes.prs.id.DecisionNode (the decision node), unbbayes.prs.id.UtilityNode (the utility node), and unbbayes.prs.bn.TreeVariable (previously explained in Subsection 2.2.2). Nodes extending the latter class can have their values derived using the method getMarginalAt(int).

4

The Plug-in Infrastructure

Unlike APIs, plug-ins offer a means to run new code inside the UnBBayes’ runtime environment. A plug-in is a program that interacts with a host application (a core) to provide a given function (usually very specific) “on demand”. The binding between a plug-in and a core application usually happens at loading time (when the application starts up) or at runtime. Therefore, a plug-in oriented design offers a practical and flexible environment for enabling software variability, since no modifications in the host application are required. A system providing plug-in extension points (i.e. portions of the program that can be extended by plug-ins) offers the following advantages: • it enables third-party developers to extend an application; • it enables support for unforeseen features;

Shou Matsumoto, et al.

• it reduces the size of each application (because unwanted parts can be left out); • it promotes modularization/componentization40 , because plug-ins’ extension points are well defined interfaces. In UnBBayes, a plug-in is implemented as a folder, a ZIP or a JAR file containing the following elements: (a) a plug-in descriptor file41 (a XML file containing metadata about the plug-in itself), (b) classes (the Java program itself - it can be a set of “.class” files or a packaged JAR file), and (c) resources (e.g. images, icons, message files, mark-up text). Samples of plug-in descriptors can be found through Subsections 4.2, 4.3 and 4.4. In order to provide a flexible and easy plug-in environment, UnBBayes offers the following base features: • Plug-in dependency management: a plug-in importing another plug-in (i.e. plug-ins that reuse classes from a preexisting one) can be developed. It speeds development and produces smaller, less redundant programs, but it can also cause what is colloquially called “dependency hell”42 . A built-in integrity check is performed in order to verify plug-in consistency. • Plug-in version control: the application can consistently load and manage different versions of a plug-in in the same environment. Therefore, there is no need to overwrite files in order to overwrite features. • Plug-ins of Plug-ins: there can be nested plug-ins (i.e. plug-ins adding functionalities to other plug-ins). • Hot-plug: plug-ins can be reloaded without restarting the application, thus it provides a flexible mechanism for runtime variability. • Flexible I/O handler: Input/Output requests (i.e. read/write requests to files) can be multiplexed to plugins according to the target’s file extension. If a conflict (a situation in which two or more plug-ins claim to be compatible with a file) occurs, the application asks for the user to choose the desired plug-in. Any plug-in offering a functionality to UnBBayes must be placed in the “plugins” folder, as illustrated in Figure 10. A default plug-in is a simpler descriptor (XML file) exposing some extension point specification (a specification of how plug-ins must be implemented in order to extend a functionality). These descriptors are needed because plug-ins can only be loaded by UnBBayes if they implement an extension point.

Figure 10: This is a typical UnBBayes folder structure. 40 The major advantages of modularization/componentization are encapsulation (some information is hidden to limit the impact of code changes) and faster development (modules/components can be developed in parallel). 41 A plug-in descriptor file is both the main and the minimal content of a UnBBayes plug-in, thus one can create a plug-in composed only by a sole descriptor file. 42 A dependency hell happens when program parts (e.g. DLL, library, JAR) have a too complex interdependency, making maintenance unfeasible because of eventual incompatibilities.

UnBBayes: a Java Framework for Probabilistic Models in AI

4.1

Architectural Overview

UnBBayes currently relies on Java Plugin Framework43 (JPF) version 1.5.1 to provide a flexible plug-in environment. JPF is an open source, LGPL44 licensed plug-in infrastructure framework for building scalable Java projects with low maintenance cost, providing a runtime engine that can dynamically discover and load plug-ins on-the-fly. The activation process (i.e. the class loading process) is done in a lazy manner, so plug-in classes are loaded into memory only when they are needed. From an architectural point of view, the main advantage provided by JPF is class loader isolation. Each JPF plug-in has its own associated class loader, which in turn is responsible for managing its own resources (classes and related files). Class loaders are organized in a graph-like hierarchic dependency, so that resource lookup requests are delegated from descendant class loaders to the ancestors45 . Although JPF is conceptually a framework, UnBBayes uses it mostly as a library46 in order to avoid excessive architectural dependency to the JPF. This is because: (1) UnBBayes is intended to be an autonomous application/framework and thus must not depend on other software’s design decisions, and (2) the plug-in infrastructure itself is susceptible to changes (it is a variation/extension point) and is likely to be changed in future releases. Figure 11 illustrates the differences between the common usage of JPF as a framework and how it is applied in UnBBayes. The “initialize main application” step works virtually as an interface between the JPF Boot Library and an application (that’s why the dashed line is overlapping it), because this step is started by JPF Boot Library, but most of the functionalities are provided by the application. Similarly, the “load JPF (PluginManager)” step is started and controlled by UnBBayes, but the functionality itself is provided by the JPF library. JPF maintains a registry of available plug-ins and its extension points in a class called org.java.plugin.PluginManager. In JPF’s vocabulary, an extension point is a portion of a program where its functionality can be extended by plug-ins, and the actual implementation of an extension point is called extension. There is an adapter class (unbbayes.util.extension.manager.UnBBayesPluginContextHolder) offered by UnBBayes to provide an unified access to PluginManager, so developers needing access to JPF functionalities (i.e. extension points and their extensions) through UnBBayes plug-in infrastructure should primarily access UnBBayesPluginContextHolder. 4.2

Plug-ins for the Core Module

Figure 12 captures some plug-ins that extend functionalities of the BN module (i.e. core). Currently, there are 4 extension points with such purpose: InferenceAlgorithm (it adds new BN inference algorithms), PNIO (it adds new I/O classes to store and retrieve Probabilistic Networks in general), PluginNode (a means to add new types of nodes in a BN), and ProbabilityFunctionPanel (a means to add new forms to edit a conditional probability distribution in existing nodes). All metadata about a plug-in is specified in a plug-in descriptor file (plugin.xml). This metadata include specifications of what extension points are made available from that plug-in, and which extension points the plug-in is implementing. The extension-point tag marks the available extension points, while the implemented extension points are tagged by the extension tag. Listings 3, 4, 5, and 6 illustrate the usage of extension-point tag to specify BN extension points, respectively for new inference algorithms, new I/O formats, new types of nodes, and new panels/forms to edit a conditional probability distribution. Listing 7 presents a relatively complete 43 JPF

homepage: http://jpf.sourceforge.net/ stands for GNU Lesser General Public License. 45 The resource lookup order can be inverted by changing configurations. 46 The main difference between a library and a framework is the direction of program calls. If X is a framework and Y adheres to it, then X calls routines of Y (X controls the overall program flow). If X is a library and Y uses it, then Y calls routines of X (Y controls the program flow). 44 LGPL

Shou Matsumoto, et al.

Figure 11: A comparison between JPF’s boot flowchart as a framework (on the left) and UnBBayes’ boot flowchart using JPF as a library (on the right).

sample descriptor file implementing the PluginNode extension point (i.e. using an extension tag). 1 2 3 4 5



Listing 3: Extension point specification for inference algorithm plug-in.

The parameter-def tag defines what parameters should be provided by an extension implementing the referred extension point (those parameters are mandatory if its multiplicity is not made explicit). Figures 13 and 14 illustrate the class structure of the previously declared extension points. The following list gives a brief explanation of the main classes: • InferenceAlgorithmOptionPanel: this is a swing JPanel to configure an inference algorithm (i.e. an object of IInferenceAlgorithm). • IInferenceAlgorithm: classes implementing this interface provide functionalities to compile and propagate evidence of a BN.

UnBBayes: a Java Framework for Probabilistic Models in AI

Figure 12: The core (BN) module has some extension points to extend BN functionalities. 1 2 3 4



Listing 4: Extension point specification for I/O plug-in.

• BaseIO: classes implementing this interface offer functionalities to load and save probabilistic networks in general. • IPluginUShape: classes implementing this interface are adapter classes providing functionalities to draw a IPluginNode in a canvas. • IPluginNode: classes implementing this interface represent BN nodes. • IProbabilityFunctionPanelBuilder: classes implementing this interface create instances of JPanel that serve to specify a conditional probability distribution of a node (i.e. a probability distribution of a node given the values of its parents). • *Builder: classes having this name pattern are builder classes, which create instances of other classes. Due to the limitations of the plug-in infrastructure, classes specified by parameter-def tags must contain a default constructor with no parameters. Since this is not always feasible (e.g. a default constructor is marked as private, denying its access), a builder (with a default constructor with no parameters) can be provided to serve as an initializer.

Shou Matsumoto, et al.

1 2 3 4 5 6 7 8 9 10 11



Listing 5: Extension point specification for a new type of node.

1 2 3 4 5 6 7 8



Listing 6: Extension point specification for a new panel/form to edit a conditional probability distribution.

• New*: classes having this name pattern are external classes and should be provided by plug-ins (i.e. are not implemented in the main program).

Figure 13: This class diagram indicates the classes that should be extended in order to create a plug-in for inference algorithm and I/O. See Listing 4.

UnBBayes: a Java Framework for Probabilistic Models in AI

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

< p l u g i n PUBLIC ” −// JPF / / J a v a Plug −i n M a n i f e s t 1 . 0 ” ” h t t p : / / j p f . s o u r c e f o r g e . n e t / p l u g i n 1 0 . d t d ”> < r e q u i r e s> < l i b r a r y i d =” P r o b a b i l i s t i c N o d e P l u g i n S t u b ” p a t h =” c l a s s e s / ” t y p e =” c o d e ”> < l i b r a r y t y p e =” r e s o u r c e s ” p a t h =” i c o n s / ” i d =” P r o b a b i l i s t i c N o d e P l u g i n S t u b i c o n s ” />

Listing 7: A sample plug-in descriptor implementing an extension point for new types of nodes.

4.3

Module Plug-ins

Figure 15 displays some screenshots of UnBBayes module plug-ins. Module plug-ins provide a means to create a relatively self-sufficient feature in UnBBayes (e.g. new formalisms or completely new applications). In UnBBayes vocabulary, modules are basically new internal frames (i.e. instances of swing JInternalFrame) that are initialized when toolbars or menu buttons are activated. Those internal frames do not need to be always visible, so one can create modules that add new functionalities to the application without displaying any actual “internal” frame (wizards or pop-ups can be emulated this way). The XML code of Listing 8 defines a module extension point: Figure 16 illustrates the main classes of a module plug-in. UnBBayesModule is the most important class of a module and it is an internal frame (thus, it is a subclass of swing JInternalFrame). Classes implementing IPersistenceAwareWindow are GUI classes containing a reference to an I/O class, and because UnBBayesModule implements IPersistenceAwareWindow, a module should be aware of what kind of files it can handle (so that UnBBayes can consistently delegate I/O requests to the right modules). NewModulePlugin and NewModulePluginBuilder are just placeholders representing classes that should be provided by plug-ins. The builder is necessary only if NewModulePlugin does not provide a default constructor with no parameters. The XML code in Listing 9 exemplifies a module plug-in descriptor47 : 47 The samples provided in this Chapter are simplifications. A complete plug-in descriptor sample can be found at project home page (http://sourceforge.net/projects/unbbayes/).

Shou Matsumoto, et al.

Figure 14: This class diagram indicates the classes that should be extended in order to create a plug-in for new nodes or new forms to edit a conditional probability distribution. See Listings 5 and 6.

1 2 3 4 5 6 7 8 9 10 11



Listing 8: Extension point specification for a module plug-in.

As the above descriptor states, a plug-in can declare its own extension points (see MEBNEditorPanel extension point). By doing so, it becomes possible to create a hierarchy of extensions. Such hierarchy creates a graph-like dependency between plug-ins, which is specified with the requires tag. Figure 17 contains a screenshot of a plug-in implementing the MEBNEditorPanel extension point.

UnBBayes: a Java Framework for Probabilistic Models in AI

Figure 15: Screen shots of some module plug-ins: Multi-Entity Bayesian Network (top left), UnBMiner (top right), Human Identification Metaphor (bottom left), and Object-Oriented Bayesian Network (bottom right).

4.4

Resource Plug-ins

Resource plug-ins can be used to provide localization. They differ substantially from other plug-ins because they combine the plug-in infrastructure and the Java resource bundle mechanism48 together to archive automatic loading of classes based on the host system’s default locale. The main advantage is that there is no need to explicitly declare a plug-in extension point to enable variability of resource classes. UnBBayes uses a feature provided by Java class loaders to automatically select the appropriate resource bundle, depending on the host computer’s locale (region) configuration. This ability is also present in class loaders responsible for loading plug-ins, thus the same mechanism can be applied to load resource classes from plug-ins as well. UnBBayes integrates the scope of both class loaders (core and plug-ins) in a single class (unbbayes.util.ResourceController), which uses chain of responsibility49 to correctly choose a class loader that can load the most appropriate resource file. As a result, new resources can be added to any object without even specifying extension points for plug-ins. In spite of its enhanced flexibility, classes composing resource plug-ins are given some restrictions inher48 Details about the Java resource bundle mechanism are beyond our scope and will not be explained. However, interested readers may want to check the Java documentation (http://download.oracle.com/javase/1.5.0/docs/api/java/util/ ResourceBundle.html). 49 Chain of responsibility is a design pattern consisting of a series of processing objects containing a logic that describes the types of operations that it can handle, and how to pass off those that it cannot handle to the next processing object in the chain (Gamma et al., 1994).

Shou Matsumoto, et al.

Figure 16: This class diagram indicates the classes that should be extended in order to create a module plug-in. See Listing 8.

ited from the Java resource bundle mechanisms. Except for the default resource class, which is loaded in absence of a better option, the names of a resource class must follow a special pattern: < Name o f the class > < locale speci f ier > . E.g. ClassName en, ClassName jp, ClassName pt br. These resource classes are basically an array of texts indexed by keys. Given the resource classes of Listing 10, the lines of code in Listing 11 will output “Ola Mundo” to the console in a Portuguese environment, and “Hello World” in any other environment (due to the lack of an exact locale match). The XML code in Listing 12 illustrates the plug-in descriptor of a sample resource plug-in. A more complete sample can be found at the project’s home page by looking for a plug-in called unbbayes.oobn.resources.ja (this is a Japanese localization plug-in for Object-Oriented Bayesian Network module).

5

Related Work

Tools for manipulating probabilistic graphical models are not new, thus several similar approaches were developed over the past few years. Bodies of such related works can be classified into (but not limited to) APIs, development environments featuring a GUI, implementations of models with higher level of expressiveness, and data mining tools. We discuss each in turn. Free-BN50 is an open-source Bayesian Network structure learning API written in Java, licensed under the Apache 2.0 license. 50 http://vangjee.wordpress.com/2009/04/09/open-source-bayesian-network-structural-learning-api-free-bn/

UnBBayes: a Java Framework for Probabilistic Models in AI

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

< p l u g i n PUBLIC ” −// JPF / / J a v a Plug −i n M a n i f e s t 1 . 0 ” ” h t t p : / / j p f . s o u r c e f o r g e . n e t / p l u g i n 1 0 . d t d ”> < r e q u i r e s> < l i b r a r y i d =” u n b b a y e s . p r s . mebn ” p a t h =” u n b b a y e s . p r s . mebn − 1 . 7 . 0 . j a r ” t y p e =” c o d e ” >

Listing 9: A sample (but syntactically complete) plug-in descriptor implementing a module and specifying another extension point simultaneously.

1 2 3 4 5 6 7 8 9

c l a s s MyResources e x t e n d s j a v a . u t i l . L i s t R e s o u r c e B u n d l e { p u b l i c s t a t i c O b j e c t [ ] [ ] c o n t e n t s = {{ ”myKey” , ” H e l l o World ” } } ; public Object [ ] [ ] getContents ( ) { return c o n t e n t s ;} } c l a s s MyResources pt extends java . u t i l . ListResourceBundle { p u b l i c s t a t i c O b j e c t [ ] [ ] c o n t e n t s = {{ ”myKey” , ” Ola Mundo” } } ; public Object [ ] [ ] getContents ( ) { return c o n t e n t s ;} }

Listing 10: A simple resource class and its Portuguese version.

1 2 3 4 5

java . u t i l . ResourceBundle r e s o u r c e = unbbayes . u t i l . R e s o u r c e C o n t r o l l e r . newInstance ( ) . getBundle ( MyResources . c l a s s . getName ( ) ); System . o u t . p r i n t l n ( r e s o u r c e . g e t S t r i n g ( ”myKey” ) ) ;

Listing 11: A sample usage of a resource class.

Shou Matsumoto, et al.

Figure 17: A screen shot of a plug-in for a plug-in (a plug-in offering features to another plug-in). This one provides some panels to allow editing of ontologies within the context of Multi-Entity Bayesian Network module (in a Prot´eg´e-like fashion) and implements the “MEBNEditorPanel” extension point of Listing 9.

In contrast to Free-BN, which is meant to be an API, UnBBayes additionally offers a GUI to help users manipulate Bayesian networks with minimum effort. This approach may lead to a lower performance or a bigger program size, but it does not rely on user’s programming skills in order to be executed. Of course, other approaches also provide a development environment with a GUI, as follows: GeNIe51 is a development environment for building graphical decision-theoretic models. It has been developed at the Decision Systems Laboratory, University of Pittsburgh, and it is implemented in Visual C++. Comparing to Java implementations, implementations in C++ usually run faster, but they are not easily portable. In case of GeNIe, its heavy dependency on Microsoft’s libraries makes it even harder to run on different platforms. Because UnBBayes is written in Java, portability is not a big issue compared to C++ implementations. Of course, there are other Java alternatives for probabilistic graphical models, as follows: BNJ52 or “Bayesian Network tools in Java” is an open-source suite for probabilistic graphical models. It is published by the Kansas State University Laboratory for Knowledge Discovery in Databases (KDD). Carmen53 is a free, open-source Java software for probabilistic graphical models. It is developed by UNED (Madrid, Spain) and it offers a set of tools to manipulate Bayesian networks and influence diagrams. 51 http://genie.sis.pitt.edu/ 52 http://sourceforge.net/projects/bnj/ 53 http://www.cisiad.uned.es/carmen/index.html

UnBBayes: a Java Framework for Probabilistic Models in AI

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

< p l u g i n PUBLIC ” −// JPF / / J a v a Plug −i n M a n i f e s t 1 . 0 ” ” h t t p : / / j p f . s o u r c e f o r g e . n e t / p l u g i n 1 0 . d t d ”> < r e q u i r e s> < l i b r a r y i d =” u n b b a y e s . oobn . r e s o u r c e s . j a ” p a t h =” u n b b a y e s . oobn . r e s o u r c e s . j a − 1 . 1 . 0 . j a r ” t y p e =” c o d e ” >

Listing 12: A fairly complete sample of a resource plug-in descriptor.

SamIam54 is a comprehensive tool for modeling and reasoning with Bayesian networks, developed in Java by the Automated Reasoning Group of Professor Adnan Darwiche at UCLA. There are commercial implementations of Bayesian Networks as well: HUGIN55 is an advanced tool for decision support using Bayesian networks and influence diagrams. This is a commercial software product, but a limited version is also available for downloading. Netica56 is a powerful program for working with belief networks and influence diagrams. It is a commercial software, but a free demo version is also available for downloading. APIs are also available to several platforms, like Java, C, C#, Visual Basic, C++, Matlab, and CLisp. Commercial implementations offer a rich repository of features that matches real demands from the market, which UnBBayes cannot offer. However, because of some licensing policies, access to its source code is usually very difficult. Because UnBBayes is an open-source framework (licensed under GPL57 ), its source-code is freely available. There are tools that focus mainly on highly expressive probabilistic models, as follows: 54 http://reasoning.cs.ucla.edu/samiam/ 55 http://www.hugin.com/ 56 http://www.norsys.com/netica.html 57 GNU

General Public License.

Shou Matsumoto, et al.

Alchemy58 is a software package providing a series of algorithms for statistical relational learning and probabilistic logic inference, based on the Markov Logic Networks (MLN) (Richardson & Domingos, 2004). It is mostly implemented in C++. PROXIMITY59 is an open-source Java platform for relational data sets and knowledge discovery using probabilistic models. It was designed and implemented by the Knowledge Discovery Laboratory in the Department of Computer Science at the University of Massachusetts Amherst. These tools are meant to perform probabilistic inferences in domains which demand extra expressiveness. UnBBayes does not offer support for either MLN or full support for relational data sets. However, it offers support for other types of highly expressive probabilistic models, like Object-Oriented Bayesian Networks (OOBN) (Koller & Pfeffer, 1997), Multi-Entity Bayesian Networks (MEBN) (Laskey, 2006), and a limited set of Probabilistic Relational Models (PRM) (Getoor et al., 2001). Finally, there are approaches that focus on data-mining. Although some of the previous approaches also offer data-driven model construction, data-mining tools are specialized in massive data manipulation. Weka60 is an open-source data mining software written in Java, and it offers a collection of machine learning algorithms for data mining tasks. UnBBayes is not a data-mining tool itself, but a plug-in called UnBMiner (Ladeira et al., 2005) can add a collection of features for massive data manipulation and machine learning. The most important thing to notice is that UnBBayes is a framework and a software environment, thus it is not limited to a specific model or aimed to a single purpose. In such sense, it can be compared to softwares like Prot´eg´e61 (Noy et al., 2000) or even Eclipse62 , if the discussion point is about multipurpose development environment supporting plug-ins. UnBBayes is not intended to compete with such well conceived products, though. UnBBayes’ focus is to provide a plug-in framework like Prot´eg´e and Eclipse, but for a specific niche, probabilistic relational models.

6

Conclusion

Java™ is increasingly becoming a de facto standard for developing complex and portable systems. In parallel, interest in principled representation and reasoning with uncertainty is experiencing full momentum in various areas of research and academia. UnBBayes can be seen as a result of both phenomena, leveraging the main advantages of Java to provide a flexible, versatile framework supporting diverse approaches to knowledge representation and reasoning with graphical models. This Chapter has provided three different visions of the tool: UnBBayes as an GUI application, UnBBayes as an API, and UnBBayes as a plug-in enabled software environment. As a probabilistic framework, UnBBayes has been experimentally applied in the context of Semantic Web to support plausible reasoning with probabilistic ontologies using Multi-Entity Bayesian Network (MEBN) as its underlying logic (Carvalho et al., 2010). Likewise, several Bayesian formalisms have been successfully implemented in this framework (e.g. Object-Oriented Bayesian Networks, Multiply Sectioned Bayesian Networks, 58 http://alchemy.cs.washington.edu/ 59 http://kdl.cs.umass.edu/proximity/proximity.html 60 http://www.cs.waikato.ac.nz/ml/weka/ 61 http://protege.stanford.edu/ 62 Eclipse

is a general-purpose Integrated Development Environment (IDE). Home page: http://www.eclipse.org.

UnBBayes: a Java Framework for Probabilistic Models in AI

Probabilistic Relational Models) in the last few years, proving UnBBayes to provide enough extensibility to speed up development of different Bayesian formalisms. As an API and software environment, UnBBayes was successfully adopted by the PRobabilistic OntoloGies for Netcentric Operational Systems (PROGNOS) (Costa et al., 2009; ?) 63 ) project team to implement Conditional Probability Scripts (CPS), a means for specifying a conditional probability distribution in Bayesian Networks using a script-like language. As shown by the PROGNOS team, UnBBayes’ plug-in infrastructure has also proven to offer substantial improvements in software reusability and variability. If you have Java and a software design using its features into its maximum extent, then the sky is the limit for what you can develop!

References Booker, L. B. & Hota, N. (1986). Probabilistic reasoning about ship images. In Second Annual Conference on Uncertainty in Artificial Intelligence Philadelphia, PA. Buchanan, B. G. & Shortliffe, E. H. (1984). Rule-based expert systems: The MYCIN. In Experiments of the Stanford Heuristic Programming Project MA, USA: Addison-Wesley. Buschmann, F., Meunier, R., Rohnert, H., Sommerlad, P., & Stal, M. (1996). Pattern-Oriented Software Architecture Volume 1: A System of Patterns. Chichester, England: John Wiley & Sons. Carvalho, R., Santos, L., Ladeira, M., & Costa, P. (2007). A GUI Tool for Plausible Reasoning in the Semantic Web using MEBN. In Proceedings of the Seventh International Conference on Intelligent Systems Design and Applications Rio de Janeiro, Brazil: IEEE Press. Carvalho, R., Santos, L., Matsumoto, S., Ladeira, M., Costa, P., & Laskey, K. (2010). UnBBayes: Modeling Uncertainty for Plausible Reasoning in the Semantic Web, (pp. 1–28). IntechWeb, Croatia. Carvalho, R. N. & Onishi, M. S. (2003). Framework e API para construc¸a˜ o de sistemas inteligentes baseados em diagrama de influˆencias e rede bayesiana m´ultipla secionada. Carvalho, R. N., Onishi, M. S., & Ladeira, M. (2002). Development of the java version of the unbbayes framework for probabilistic reasoning. In Congresso de Iniciacao Cientifica da UnB University of Brasilia: Brasilia, DF, Brazil. Charniak, E. & Goldman, R. P. (1989). A semantics for probabilistic quantifier-free first-order languages with particular application to story understanding. In Eleventh International Joint Conference on Artificial Intelligence Detroit, Michigan, USA. Costa, P. (2005). Bayesian Semantics for the Semantic Web. PhD thesis, Department of Systems Engineering and Operational Research, George Mason University. Costa, P., Chang, K., Laskey, K., & Carvalho, R. N. (2009). A multi-disciplinary approach to high level fusion in predictive situational awareness. In Proceedings of the 12th International Conference on Information Fusion (pp. 248–255). Seattle, Washington, USA. da Silva, D. C. (2003). Inclus˜ao do m´odulo de aprendizagem no framework UnBBayes. Dempster, A. P. (1967). Upper and lower probabilities induced by a multivalued mapping. Annals of Mathematical Statistics, 38, 325–339. Gamma, E., Helm, R., Johnson, R., & Vlissides, J. M. (1994). Design Patterns: Elements of Reusable Object-Oriented Software. USA: Addison-Wesley. Garcia, J. A. R. (2008). jCOLIBRI: A multi-level platform for building and generating CBR systems. PhD thesis, Department of Software Engineering and Artificial Intelligence Facultad de Informatica Universidad Complutense de Madrid. 63 PROGNOS

project page: http://sourceforge.net/projects/prognos.

Shou Matsumoto, et al.

Getoor, L., Friedman, N., Koller, D., & Pfeffer, A. (2001). Learning probabilistic relational models. In S. Dzeroski & N. Lavrac (Eds.), Relational Data Mining New York, USA: Springer-Verlag. Goldberg, A. & Kay, A. (1976). Smalltalk-72 instruction manual. Technical Report SSL 76-6. Technical report, Learning Research Group, Xerox Palo, Alto Research Center. Howard, R. & Matheson, J. (1981). Influence diagrams. Decision Analysis, 2, 127–143. Hugin Expert (2010). HUGIN API Reference Manual, version 7.4. Hugin Expert A/S. Chapter 12 - The NET Language. Jensen, F., Lauritzen, S., & Olsen, K. (1990a). Bayesian updating in causal probabilistic networks by local computations. Computational Statistics Quarterly, 4, 269–282. Jensen, F. V. & Jensen, F. (1994). Optimal junction trees. In In: Proceedings of the Conference on Uncertainty in Artificial Intelligence (pp. 360–366). San Francisco, USA. Jensen, F. V., Olsen, K. G., & Andersen, S. K. (1990b). An algebra of bayesian belief universes for knowledge-based systems. Networks, 20, 637–659. Koller, D. & Pfeffer, A. (1997). Object-Oriented Bayesian Networks. In Thirteenth Conference on Uncertainty in Artificial Intelligence (UAI-97) San Francisco, CA, USA. Ladeira, M., Vieira, M., Prado, H., Noivo, R., & Castanheira, D. (2005). UnBMiner - ferramenta aberta para minerac¸a˜ o de dados. Revista Tecnologia da Informac¸a˜ o, 5(1), 45–63. Laskey, K. (2008). MEBN: A Language for First-Order Bayesian Knowledge Bases. Artificial Intelligence, (pp. 172(2–3)). Laskey, K. B. (2006). First-Order Bayesian Logic. PhD thesis, George Mason University. Lauritzen, S. L. & Spiegelhalter, D. J. (1988). Local computations with probabilities on graphical structures and their application to expert systems (with discussion). Journal Royal Statistical Society, 50(2), 425–448. Meyer, B. (1997). Object-oriented software construction. Prentice-hall International Series in Computer Science. Prentice Hall PTR. Murphy, K. P. (2002). Dynamic bayesian networks: Representation, inference and learning. Noy, N. F., Fergerson, R., & Musen, M. A. (2000). The knowledge model of protege-2000: Combining interoperability and flexibility. In 2th International Conference on Knowledge Engineering and Knowledge Management (EKAW’2000) Juan-lesPins, France. Olshansky, D. (2007). Java Plug-in Framework (JPF) project page. http://jpf.sourceforge.net/. Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Mateo, CA, USA: Morgan Kaufmann Publishers. QI, R. & Poole, D. (1993). Decision Graph Search. Technical Report 93-3, Department of Computer Science of the University of British Columbia, Vancouver. Richardson, M. & Domingos, P. (2004). Markov Logic Networks. Technical report, University of Washington, Seattle, WA. Russel, S. & Norvig, P. (2002). Artificial Intelligence: a Modern Approach. New Jersey, USA: Prentice-Hall. Shachter, R. D. (1990). An Ordered Examination of Influence Diagrams. NETWORKS, 20. Shafer, G. (1976). A Mathematical Theory of Evidence. Princeton, NJ, USA: University Press. Soares, A. M. M. (2000). UnBBayes: Framework para redes probabil´ısticas. Spiegelhalter, D. J., Franklin, R., & Bull, K. (1989). Assessment, criticism, and improvement of imprecise probabilities for a medical expert system. In Fifth Conference on Uncertainty in Artificial Intelligence (pp. 257–285). Mountain View, CA.

UnBBayes: a Java Framework for Probabilistic Models in AI

Sun

Microsystems, Inc. (2000). Java language specification, second http://java.sun.com/docs/books/jls/second edition/html/exceptions.doc.html.

edition,

Chapter

11

-

Exceptions.

Sun

Microsystems, Inc. (2004). How to Write Doc Comments for the Javadoc http://www.oracle.com/technetwork/java/javase/documentation/index-137868.html. Oracle Technology Network.

Tool.

Xiang, Y., Poole, D., & Beddoes, M. P. (1993). Multiply sectioned bayesian networks and junction forests for large knowledge based systems. In Computational Intelligence, volume 9(2) (pp. 171–220). Cambridge.