G Model ECOMOD-6010;

No. of Pages 12

ARTICLE IN PRESS Ecological Modelling xxx (2010) xxx–xxx

Contents lists available at ScienceDirect

Ecological Modelling journal homepage: www.elsevier.com/locate/ecolmodel

Combining state and transition models with dynamic Bayesian networks夽 Ann E. Nicholson a,∗ , M. Julia Flores b a b

Faculty of Information Technology, Monash University, Australia Departamento de Sistemas Informáticos SIMD i3 A, Universidad de Castilla-La Mancha, Campus Universitario s/n, Albacete 02071, Spain

a r t i c l e

i n f o

Article history: Received 25 June 2010 Received in revised form 2 October 2010 Accepted 6 October 2010 Available online xxx Keywords: Rangeland management Bayesian networks Dynamic Bayesian networks State-and-transition models System dynamics

a b s t r a c t Bashari et al. (2009) propose combining state and transition models (STMs) with Bayesian networks for decision support tools where the focus is on modelling the system dynamics. There is already an extension of Bayesian networks – so-called dynamic Bayesian networks (DBNs) – for explicitly modelling systems that change over time, that has also been applied in ecological modelling. In this paper we propose a combination of STMs and DBNs that overcome some of the limitations of Bashari et al.’s approach including providing an explicit representation of the next state, while retaining its advantages, such an the explicit representation of transitions. We then show that the new model can be applied iteratively to predict into the future consistently with different time frames. We use Bashari et al.’s rangeland management problem as an illustrative case study. We present a comparative complexity analysis of the different approaches, based on the structure inherent in the problem being modelled. This analysis showed that any models that explicitly represent all the transitions only remain tractable when there are natural constraints in the domain. Thus we recommend modellers should analyse these aspects of their problem before deciding whether to use the framework. © 2010 Elsevier B.V. All rights reserved.

1. Introduction Environmental management involves making decisions that will impact on the ecological system. Examples include whether to control exotic flora or fauna, restrict farming or forestry practices, or change the landscape to alter water flow or usage. Any useful environmental decision support tool must model changes in the ecological system over time, particularly those that are the result of human activities. The so-called state-and-transition model (STM) has been used to model such changes over time in systems that have clear transitions between distinct states of a physical environment, in particular rangeland vegetation (Stringham et al., 2003; Bestelmeyer et al., 2003; Sadler et al., 2010), but also other ecological and environmental domains (e.g., Saatkamp et al., 1996). The STM framework facilitates the organisation of information for management purposes. STMs are mainly based on the state/transition/threshold relationships determined by the resilience and resistance of the

夽 This work has been partially supported by the Spanish Ministerio de Educación y Tecnología under Project TIN2007-67418-C03-01 and UCLM under Project PL20091291. ∗ Corresponding author at: Clayton School of IT, Monash University, Wellington Rd, Clayton, VIC 3800, Australia. Tel.: +61 399055211; fax: +61 399055146. E-mail addresses: [email protected] (A.E. Nicholson), [email protected] (M.J. Flores).

ecosystems’ primary ecological processes. They combine the graphical depiction of transitions and their causal factors with tables of qualitative descriptions of the transitions. Bayesian networks (BNs) are an increasingly popular paradigm for reasoning under uncertainty. A Bayesian network (Pearl, 1988; Jensen and Nielsen, 2007) is a directed, acyclic graph whose nodes represent the random variables in the problem. A set of directed edges connect pairs of vertices, representing the direct dependencies (which are often causal connections) between variables. The set of nodes pointing to X are called its parents, and is denoted pa(X). The relationship between variables is quantified by conditional probability tables (CPTs) associated with each node, namely P(X|pa(X)). The CPTs together compactly represent the full joint distribution. Users can set the values of any combination of nodes in the network that they have observed. This evidence, e, propagates through the network, producing a new posterior probability distribution P(X|e) for each variable in the network. There are a number of efficient exact and approximate inference algorithms for performing this probabilistic updating, providing a powerful combination of predictive, diagnostic and explanatory reasoning. Fig. 1 gives an example BN for a simple artificial ecological problem, to illustrate the components (structure and the CPTs) together with several reasoning scenarios, using screenshots from the Netica BN software (Norsys, 1994–2010). The complexity of a BN model is naturally the number of parameters, typically the size of the CPTs. However, often there is so-called “local structure” in the relationship between parent and

0304-3800/$ – see front matter © 2010 Elsevier B.V. All rights reserved. doi:10.1016/j.ecolmodel.2010.10.010

Please cite this article in press as: Nicholson, A.E., Flores, M.J., Combining state and transition models with dynamic Bayesian networks. Ecol. Model. (2010), doi:10.1016/j.ecolmodel.2010.10.010

G Model ECOMOD-6010; No. of Pages 12 2

ARTICLE IN PRESS A.E. Nicholson, M.J. Flores / Ecological Modelling xxx (2010) xxx–xxx

Fig. 1. BN example—“Native Fish” (Nicholson and Woodberry, 2010).

child nodes, so the full CPT does not need to be specified, instead more compact “context sensitive” representations can be used (e.g., Boutilier et al., 1996). In addition, the most efficient inference algorithms do not work directly on the BN graph, but instead compile it into a so-called junction tree. The complexity of the inference then depends on the structure and size of this compiled form. Over the past 10 years, BNs have been widely used in ecological modelling (see Section 5.2.3 in Korb and Nicholson, 2010 for a survey), with a number of modelling guidelines published (e.g., Varis and Kuikka, 1999; Borsuk et al., 2004; Renken and Mumby, 2009), while Uusitalo (2007) reviews their features and use in modelling environmental applications. Bashari et al. (2009) suggested combining STMs and BNs to obtain the advantages of both, namely the STM’s graphical depiction of transitions with the BN’s quantitative representation of the uncertainty using probabilities. They describe an approach to rangeland management decision support that combines a state

and transition model with a Bayesian network to provide a relatively simple and updatable rangeland dynamics model that can accommodate uncertainty and be used for scenario, diagnostic, and sensitivity analysis. In this paper we begin with a detailed analysis of Bashari et al.’s framework, then formalise and modify it to overcome most of the limitations identified (Section 2). The crucial weakness in their framework, however, is that it does not explicitly model the “next” state of the system, a key component in any dynamic modelling framework. We then show how an existing variant of Bayesian networks – so-called dynamic Bayesian networks (DBNs) – can be used to overcome this problem (Section 3). DBNs are a long-established extension to ordinary BNs that allow explicit modelling of changes over time (e.g., Dean and Kanazawa, 1989; Kjærulff, 1992; Nicholson, 1992). They have been used in a range of applications such as robot monitoring (Forbes et al., 1995; Dean and Wellman, 1991; Nicholson and Brady, 1992),

Please cite this article in press as: Nicholson, A.E., Flores, M.J., Combining state and transition models with dynamic Bayesian networks. Ecol. Model. (2010), doi:10.1016/j.ecolmodel.2010.10.010

G Model ECOMOD-6010;

No. of Pages 12

ARTICLE IN PRESS A.E. Nicholson, M.J. Flores / Ecological Modelling xxx (2010) xxx–xxx

oil price forecasting (Dagum et al., 1992) and modelling sleep apnea (Dagum and Galper, 1993), but surprisingly there have been only a few environmental and ecological modelling applications (Shihab and Chalabi, 2007; Dawsey et al., 2007; Shihab, 2008), perhaps because they are perceived to be “very tedious” (Uusitalo, 2007). We show how ordinary DBNs can be used for modelling state transitions, but have the disadvantages of (1) not providing a visualisation of the transitions in the network structure, and (2) being inefficient by not representing natural constraints on possible transitions in the system. Instead we advocate a combination of STMs with dynamic Bayesian networks, retaining the advantages of Bashari et al.’s framework (combining STMs and ordinary BNs) while providing the additional advantage of better predictive modelling, as detailed in Section 4. We also present a complexity analysis (Section 5) comparing the three different modelling approaches: (1) STMs and BNs, (2) ordinary DBNs and (3) our proposed combination of STMs and DBNs. We show that the complexity of each depends on the inherent structure in the problem being modelled. In particular, for the models to be tractable, the number of transitions from each state needs to be limited, and only influenced by a small number of causal factors. If all transitions between states are possible and if a large number of causal factors influence all the transitions, the models explicitly representing state and transition models become intractable. Thus we recommend performing this analysis to determine which model is most suitable to represent a problem, before beginning any detailed modelling. Throughout this paper, we use Bashari et al.’s rangeland management case study. Note however, that we present this research as a generic approach for modelling change over time in any application domain. 2. Bashari et al.’s combination of STMs and BNs 2.1. STMs State and transition models (STMs) are a qualitative descriptive method for modelling ecological processes (Stringham et al., 2003). The system dynamics are described using diagrams that position states along several axes, typically representing environment or management gradients. Possible transitions between these states are represented using arrows, and the conditions and thresholds

3

Fig. 2. State and transition model for the Rangeland example (Bashari et al., 2009, Figure 1). PTG stands for palatable tall grasses and UPTG refers to unpalatable tall grasses. Table 1 Catalogue of vegetation states, Table 2 from Bashari et al. (2009). State no.

State description

I II III IV V

Palatable tall tussock grasses Unpalatable tall tussock grasses Short sward and sparse tall grasses Short sward Lawn

under which these transitions can occur are recorded in a transition table, the so-called catalogue of conditions. Fig. 2 shows an example state and transition model for the cleared Ironbark-spotted gum woodland in south-east Queensland, Australia – henceforth called the Rangeland example – presented in Bashari et al. (2009). Here, the state is the grassland vegetation (see Table 1), while the catalogue of transition is given in Table 2. STMs have been advocated for rangeland and other ecological management, as a low cost, simple yet versatile technique (Bestelmeyer et al., 2003). In particular, the diagrammatic visualisation has been found to assist with manager’s understanding and participation in the development of the model.

Table 2 Catalogue of vegetation transitions, Table 3 from Bashari et al. (2009). Transition

Main causes

Qualitative probability

Time frame (years)

I, II I, III I, IV I, V

SG = high, GP = low SG = lowa , GP = moderate GP = high GP = high, SNC = aboveAvg

High High High High

2–5 2–5 2–5 2–5

II, I II, III II, IV II, V

GP = none, SG = none, FTP = freq GP = high, SG = low, FTP = inFreq GP = high, FTP = freq GP = high, SNC = aboveAvg

High High High High

2–5 2–10 2–5 2–5

III, I III, II III, IV

GP = none, SG = none, GS = freq SG = moderate, GP = moderate, GS = freq GP = high, SG = none, GS = inFreq

High High High

2–5 2–5 2–5

IV, I IV, II IV, III IV, V

GP = none, GS = freq GP = low, GS = freq GS = freq, GP = none SNC = aboveAvg, GP = high

Low Low Moderate High

1–10 1–10 >5 2–5

V, I V, II

GP = none, SNC = avg, GS = freq GP = none, SNC = avg, GS = freq

Low Low

>5 >5

Main causes: GP = Grazing Pressure, SG = Selective Grazing, SNC = Soil Nutrient Content, FTP = Fire in Time Period, and GS = Good Season. a Changed from erroneous “high” in original paper (C. Smith, personnel communication, June 2010).

Please cite this article in press as: Nicholson, A.E., Flores, M.J., Combining state and transition models with dynamic Bayesian networks. Ecol. Model. (2010), doi:10.1016/j.ecolmodel.2010.10.010

G Model ECOMOD-6010; No. of Pages 12 4

ARTICLE IN PRESS A.E. Nicholson, M.J. Flores / Ecological Modelling xxx (2010) xxx–xxx

1–10 years, while IV to II, V to I and V to II are expected to occur over the unbounded “>5 years”. 2.3. Analysis of Bashari et al.’s approach

Fig. 3. Bashari et al.’s framework for constructing the BN structure from an STM (Bashari et al., 2009, Figure 2).

2.2. Combining STMs and BNs Bashari et al. (2009) describe a method for transforming an STM into a Bayesian network. Their aim was to develop rangeland management decision support tools that combine the benefits of STMs (diagrammatic, low cost, flexible, suiting participatory development) with the advantages of Bayesian networks—using probabilities to represent uncertainty, the ability to quantify relationships between variables, predictive and scenario analysis capabilities, as well as adaptive management capabilities. Bashari et al.’s original framework1 is shown in Fig. 3. Their generic BN has a “current state” node representing the variable of interest (e.g., the vegetation on the cleared woodland), with a oneto-one mapping between the possible states in the STM and the BN node states. Next, there are nodes to represent the transitions, with one node for each of the possible current states. Each transition node in turn has all states it might transition to, from that particular current state. All the environmental and management factors that may influence the transitions in the system are divided into the main factors, which are the parents of the transition nodes, and sub-factors which influence the main factors. Fig. 4 shows Bashari et al.’s Bayesian network for the Rangeland example, for one scenario. This is a screenshot from the Netica BN software, where the nodes for which evidence has been added are shaded, and the observed state is indicated by 100% (bar and number). Thus we can see the starting state is CurrentState=Tall Palatable Grasses, GoodSeasons=Infrequent, and with observations of nearly all the sub-factors. Bashari et al. obtained the parameters of this model by eliciting subjective probability estimates for some of the CPT rows, and interpolating between those to get the remaining entries (Cain, 2001). We note that Bashari et al. include a time frame node in their example that was not in their generic framework. The two time frames are “Two to Five Years” and “Five to Ten Years”, with the latter used for the example scenario in Fig. 4. This means that the mapping from the STM catalogue into the BN is only approximate, as several transitions in the catalogue were for other time periods; II to III was for time frame 2–10 years, IV to I and IV to II were for

1 We note in passing that while they refer to the structure of their model as an “influence diagram”, it is not an influence diagram with decision and utility nodes added to the chance nodes of the Bayesian networks defined by Howard and Matheson (1981).

First we describe Bashari et al.’s framework more formally, to facilitate our analysis and later comparisons. We recast their generic structure as shown in Fig. 5; we will refer to this as the ST-BN. In our generic ST-BN, ST represents the current state at time T, which has n possible states, the same one-to-one mapping between the n states in the STM and the BN node states, {s1 , s2 , . . . si , . . ., sn }. There are then n transition nodes, ST1 , . . . STi , . . ., STn to represent the transitions from each state si . The possible states for the STi transition node are then si (if no change is also possible),2 and one state sj for each state j of the ni states the system could transition to from state i. This means that if it were possible to transition to all other states from i, STi would have n states. But of course STMs are intended to describe systems where only some transitions are possible, in which case the STi nodes in the BN would have much fewer states. The environmental and management factors that may influence the transitions in the system are divided into m main factors, F1 , . . ., Fm , which are the parents of the transition nodes, and other sub-factors, X1 , . . ., Xr , which influence the main factors. While in the most complex case all m main factors could be parents of each transition node, in practice, often only a subset of the main factors triggers each individual transition, which limits the number of parents for each transition node STi . An example ST-BN, where not all transitions are possible, and not all the main factors influence all the transitions, is given in Fig. 6. Finally, we include in the generic ST-BN a node ıT. Although not included in Bashari et al.’s framework (Fig. 3), such a node is included in their example BN (Fig. 4). The ıT node explicitly represents the time frame over which the transition is being modelled, and makes it easy to switch between different time frames. Since in practice one time frame will always be specified (by adding evidence for that node), it is just a convenient way of switching between BNs that predict at different times into the future. Bashari et al.’s framework has several benefits. The transitions from each state are clearly visible, through the use of a transition node. The main factors and sub-factors influencing each transition are also clear from the structure of the BN—they are just the parents and other ancestors of each transition node. The model is also compact in the sense that only possible transitions are represented. We have also identified the following problems with their framework: 1. Sub-factors are strictly parents of the main factors. While Bashari et al. have the subfactors being parents of the main factors, in general the BN can model any relationships between the X and F nodes, as long as the F are not ancestors of the X. We depict this more general form in Fig. 5 with the dashed rectangular boxes around the two groups of nodes. Subject to the constraint that the overall network must be acyclic, there can be arcs between any pair of X nodes: any X node can be a parent of any F node, and there may also be arcs between the F nodes.3 2. Factors influencing transitions are independent of the current state. In Bashari et al.’s framework, there are no arcs from the current state ST to the factors or sub-factors influencing the transitions. This undoubtedly reflects how the STM modelling was done,

2 While Bashari et al.’s framework did not show the ‘no change’ state, it was clearly part of their actual example, see Fig. 4. 3 We note that Bashari et al.’s rangeland BN (Fig. 4) has this richer graph structure.

Please cite this article in press as: Nicholson, A.E., Flores, M.J., Combining state and transition models with dynamic Bayesian networks. Ecol. Model. (2010), doi:10.1016/j.ecolmodel.2010.10.010

G Model ECOMOD-6010;

No. of Pages 12

ARTICLE IN PRESS A.E. Nicholson, M.J. Flores / Ecological Modelling xxx (2010) xxx–xxx

5

Fig. 4. Bashari et al.’s BN representing the Rangeland example STM (Bashari et al., 2009, Figure 3).

that is, the relationships between the current state and the factors influencing or triggering transitions were not part of STM. Also, if the ST-BN is used only for scenario analysis where the current state ST is known with 100% certainty, and evidence is entered for all the root nodes (as in the scenario in Fig. 4), then any relationships between the known current state and the observed influencing factors become irrelevant. However, by allowing more complex interactions the model can be used for prediction when the current state or the influence factors are not known with certainty. Thus our formalisation in Fig. 5 indicates that current state ST can be a parent of any of the F or X nodes within the dashed rectangular box. 3. No explicit modelling of impossible transitions. If the current state is si , then of course all the transitions from other states sj (j = / i) are impossible. The CPTs in Bashari et al.’s BN correctly contain zero probabilities to represent this, namely P(STj = sk |ST = si ) = 0, ∀ j = / i, ∀ k. However for these cases, their / i. That is, all the probabilCPT contains P(STj = sj |ST = si ) = 1, ∀ j =

ity mass goes to the “no change” case, and there is no explicit representation of “no transition possible”. So when evidence is added for ST = si (as in Fig. 4) we can see that all the transition nodes other than the one for si show 100% “no change”, which in this case means “impossible”. However, this becomes more problematic if, instead, we look at the network’s predictions using uniform priors for ST , shown in Fig. 7. We can see more clearly the problem with the semantics of the “no change” state of the transition nodes: all the “no change” state of the transition nodes have posterior probabilities of 90% or more, with the real “no change” probability mass being combined with the “impossible” transition situation. We solve this problem by extending the state space of each transition node ST, with an additional state Impossible. The CPT is then modified with: P(STj = Impossible|S T = si ) = 1, ∀j = / i / i, ∀k P(STj = sk |S T = si ) = 0, ∀j = 4. No explicit next state. The next problem is that there is no explicit representation of the “next state” of the system, which

Fig. 5. The generic ST-BN: a more general formalisation of Bashari et al.’s model. ST , the current state, may directly influence any of the environmental and management factors, which are divided into m main factors, F1 , . . ., Fm (which directly influence transitions) and other sub-factors, X1 , . . ., Xr (which influence the main factors). The transition nodes, ST1 , . . . STi , . . ., STn represent the transitions from each state si , each with at most n + 1 values, one for each state plus “impossible”, giving explicit modelling of impossible transitions. ıT represents the time frame.

Fig. 6. An example ST-BN, showing that not all transitions are possible, and that only some of the main factors influence each transition.

Please cite this article in press as: Nicholson, A.E., Flores, M.J., Combining state and transition models with dynamic Bayesian networks. Ecol. Model. (2010), doi:10.1016/j.ecolmodel.2010.10.010

G Model ECOMOD-6010; No. of Pages 12 6

ARTICLE IN PRESS A.E. Nicholson, M.J. Flores / Ecological Modelling xxx (2010) xxx–xxx

Fig. 7. Left: Example showing the problem with “no change” also representing impossible transitions in Bashari et al.’s Rangeland ST-BN. Right: The problem is solved by adding a explicit “impossible” state.

Fig. 8. General structure of a DBN (Figure 4.10 from Korb and Nicholson, 2010).

we will call ST+1 . It is in Bashari et al.’s model only in a distributed form via the states of the state transition node; that is, while we want to compute the new posterior probability P(S T +1 = sj |S T , evidence for F and X nodes), we have only the computed probabilities for each transition node P(STk = sj |S T , evidence for F and X nodes), for k = 1 . . . n.

changes in a process or system over time. The general structure of a DBN is shown in Fig. 8 (Korb and Nicholson, 2004; Figure 4.10). In a DBN, for each domain variable Xi , there is one node for each time step of interest: XiT , XiT +1 , XiT +2 , etc. Each time step is called a time-slice. The relationships between variables at successive time steps are represented by so-called temporal arcs, including relationships between (i) the same variable over time, XiT → XiT +1 , and

As discussed in Section 1, dynamic Bayesian networks (DBNs) are a variant of ordinary BNs that allow explicit modelling of

(ii) different variables over time, XiT → XjT +1 . Note that in the generic structure there are no arcs that span more than a single time step; this reflects the so-called Markov assumption that the state of the world at a particular time depends only on the previous state and any action taken in it. Given the typical restriction that both the structure and the CPTs are unchanging, a DBN can be specified very compactly.5 Fig. 9 shows an ordinary generic DBN structure modelling the change from ST to ST+1 , with the same factors and sub-factors that may influence the transition. Both ST and ST+1 have n possible states, {s1 , . . ., sn }. It is clear that the two time-slice DBN provides the “next” state missing from Bashari et al.’s framework. However, this

4 We note that Rumpff et al.’s BN does not follow Bashari et al.’s framework; it does not have nodes representing the transition and the native woodland state is a function of a larger number of state description nodes.

5 Some BN software packages (e.g. Hugin, GeNIe, Netica) provide a facility to specify a DBN compactly.

This limitation can be solved by extending the model with a “next” or “future” state; Rumpff et al. (in press) have such a node in their BN for state and transition modelling.4 We propose incorporating this “next” state more formally, as ST+1 , that is, by making the network into a dynamic Bayesian network. 3. Combining STMs and DBNS 3.1. Ordinary DBNs

Please cite this article in press as: Nicholson, A.E., Flores, M.J., Combining state and transition models with dynamic Bayesian networks. Ecol. Model. (2010), doi:10.1016/j.ecolmodel.2010.10.010

G Model ECOMOD-6010;

No. of Pages 12

ARTICLE IN PRESS A.E. Nicholson, M.J. Flores / Ecological Modelling xxx (2010) xxx–xxx

7

Fig. 10. The generic ST-DBN: combining STMs with DBNs. T

T+1

Fig. 9. An ordinary two time-slice DBN modelling the transition from S to S , with main factors F1 , . . ., Fm , which are in turn influenced by other factors X1 , . . ., Xr .

generic DBN has an obvious major deficiency—its structure does not show that different background factors may trigger different transitions, or that most transitions are impossible. Of course these aspects can be modelled in the DBN, but only within the CPT for ST+1 , as follows: • If some transitions from ST = si are independent of some factors: P(S T +1 = sj |S T = si , F1 , . . . Fm ) = P(S T +1 = sj |S T = si , G) where G ⊂ F. • If a transition from ST = si to ST+1 = sj is impossible for some combination of main factors F1 = fi, F2 = f2 , . . ., Fm = fm : P(S T +1 = sj |S T = si , f1 , . . . fm ) = 0 • If a transition from ST = si to ST+1 = sj is always impossible: P(S T +1 = sj |S T = si , F1 , . . . Fm ) = P(S T +1 = sj |S T = si ) = 0

So while the ordinary generic DBN has a relatively simple structure, the constraints on transitions, and conditions triggering them, are not shown in that structure, but rather hidden in the CPTs. This is a clear disadvantage from a modelling perspective. In addition, the ordinary DBN may well have many arcs going into the “next” states, as any factor that influences even one of the transitions must be a parent of ST+1 . This often leads to models with a very large number of parameters, many of which may be zero. 3.2. ST-DBNs The obvious solution is therefore to combine STMs with DBNs. We can think of this in two ways: (1) we can start with the ST-BN and add the “next state”, ST+1 ; or (2) we can start with a generic DBN and add n explicit transition nodes, ST1 , . . . STn . Either way, the result is the generic structure shown in Fig. 10, which we call a ST-DBN. As with the ST-BN, there is one transition node for each of the n starting states. Each transition node ST has only some of the causal

factors as parents. The CPT for the ST node is just a partition of the corresponding ordinary DBN CPT. Clearly the next state node, ST+1 , has to combine the results of all the different transition nodes, which means it has n parents. However, the relationship between the transition nodes and ST+1 is deterministic, whose CPT can be generated from a straightforward equation: P(S T +1 = si |S T = si , STi = si ) = 1 / i P(S T +1 = sj |S T = si , STi = sj ) = 1, ∀j = / i= / k P(S T +1 = sj |S T = si , STi = sk ) = 0, ∀j = Note that most of the combinations of the parents are impossi/ i are impossible and ble. For example, if ST = si , then all STj = sk , j = there is no need to specify a conditional probability P(ST+1 |ST , . . ., STj = sk ). Note also that we have not modelled the ıT (between T and T + 1) with a time variable. If the time step is constant, the DBN is less cluttered using ıT as a parameter in the CPTs for each STI , that is P(STi |ST ) = f(ıT). An explicit time node can be added if the time step has to be varied (e.g., Kanazawa, 1992). However, we prefer an alternative method that uses the ST-DBN to extend the prediction further into the future, as we explain in Section 4. 3.3. Example: the ST-DBN for the Rangeland example Fig. 11 shows the ST-DBN for the Rangeland with evidence entered for the same scenario used by Bashari et al. for their Rangeland ST-BN (see Fig. 4), but with the 2–5 years time frame. The nodes where we have added evidence are shaded, with a 100% belief bar next to the observed state corresponding to a probability of 1. Once evidence is added, the BN software updates the posterior probability distributions for all the other nodes. We can see, for example, that the highest probability for the From Palatable To Tall Grasses To transition is 35.9% for Short Sward Sparse Tall Grass, while the least likely is Unpalatable Tall Grasses (7.37%). Note that these probabilities differ from those in Fig. 4 because of the different time frames applied; the parameters in the CPTs are unchanged and the models give the same predictions over the same time frame. As expected, given this scenario and starting in state Palatable Tall Grasses, all the other transition nodes show 100% for Impossible, compared to the less informative No Change state in the ST-BN model (see Fig. 4).

Please cite this article in press as: Nicholson, A.E., Flores, M.J., Combining state and transition models with dynamic Bayesian networks. Ecol. Model. (2010), doi:10.1016/j.ecolmodel.2010.10.010

G Model ECOMOD-6010; No. of Pages 12 8

ARTICLE IN PRESS A.E. Nicholson, M.J. Flores / Ecological Modelling xxx (2010) xxx–xxx

Fig. 11. ST-DBN for the Rangeland example for the same scenario as given in Fig. 4 for the ST-BN (ıT = 2–5 years).

Fig. 12. Generic ST-DBN “rolled” out over multiple time slices.

Note that DBNs are typically used with a point value for ıT, rather than an interval. The uncertainty in the transition process is then represented in the CPTs, not in the value for ıT itself. However, since we want to replicate the Rangeland example, we retain Bashari et al.’s time range. 4. Using the ST-DBN for temporal reasoning Recall that in our generic ST-DBN there was an implied ıT for the transition time from the current state ST to the next state ST+1 . An advantage of modelling with DBNs is that once the model has been specified for the two time-slices, ST and ST+1 , it is straightforward to automatically “roll-out” the network with additional time-slices to reason predictively into the future.6

6 As mentioned above, typically the rolled-out DBN has the same CPTs for each time slice, that is, it assumes the underlying process is not changing. This assumption can be removed but then additional modelling is required to specify the new CPTs, / P(STiT |SiT , GT ). e.g. P(STiT +1 |SiT +1 , GT+1 ) =

Fig. 12 shows the generic roll-out of the ST-DBN. We can see that a new problem has arisen, namely, how do we treat the influencing factors – the X and F nodes – in the rolled out DBN? In many cases they will be system variables or management actions that are related over time. This can be modelled in the DBN, but requires the addition of many new temporal arcs, and the elicitation of many more conditional probabilities. If these temporal dependencies are ignored then obviously the model will be a simpler but less accurate model of the system (in the same way that Bashari et al.’s framework did not include arcs from the current state to the influencing factors). However, there is an established alternative to rolling out the DBN for many steps into the future. Instead, a fixed size, sliding “window” of time slices – often only two time-slices – is maintained. As the reasoning process moves forward with time, one older time slice is dropped off the DBN, while another is added—so-called “roll-up”. A version of the standard DBN updating process is given in Korb and Nicholson (2004); in practice it can be applied to the two time-slice ST-DBN as follows:

Please cite this article in press as: Nicholson, A.E., Flores, M.J., Combining state and transition models with dynamic Bayesian networks. Ecol. Model. (2010), doi:10.1016/j.ecolmodel.2010.10.010

G Model ECOMOD-6010;

No. of Pages 12

ARTICLE IN PRESS A.E. Nicholson, M.J. Flores / Ecological Modelling xxx (2010) xxx–xxx

9

Fig. 13. ST-DBN for the Rangeland example after rollup following the prediction step in Fig. 11 (ıT = 2–5 years): prior probabilities for “Current State” have been set to posterior probabilities from Future State node in Fig. 11.

1. Prediction (a) Add evidence (exact or uncertain) E for any of ST , or the F and X nodes. (b) Perform inference in the ST-DBN to compute the predicted probabilities for P(ST+1 |E). 2. Rollup (a) Use the predictions for ST+1 slice as the new prior by setting ST to that computed P(ST+1 |E): P(S T ) → P(S T +1 |E) (b) Remove all evidence E.

Let us consider how this will work for our Rangeland ST-DBN. Fig. 11 shows the first prediction step, while Fig. 13 shows the network after the priors for ST have been updated. We note that the ST-DBN updating process allows us to assess the internal consistency between the 2–5- and 5–10-year time frames. That is, using ıT = 2–5 years we can set the start state, add evidence for the factors and then compute a probability distribution over ST+1 . We then make that the new prior for ST , add the same set of evidence and compute again ST+1 , which is now the prediction for 4–10 years. We then compared this to prediction made by

Fig. 14. Comparison of predictions for ST+1 for using Rangeland ST-DBN from starting state Tall Palatable Grasses: (a) ıT = 2–5 years (1st prediction), (b) ıT = 2–5 years (2nd prediction), and (c) ıT = 5–10 years.

Please cite this article in press as: Nicholson, A.E., Flores, M.J., Combining state and transition models with dynamic Bayesian networks. Ecol. Model. (2010), doi:10.1016/j.ecolmodel.2010.10.010

G Model ECOMOD-6010; No. of Pages 12

ARTICLE IN PRESS

10

A.E. Nicholson, M.J. Flores / Ecological Modelling xxx (2010) xxx–xxx

Table 3 Number of probabilities in the CPTs of the ST nodes for the Rangeland ST-BN. STi

|˝(ST)|

|˝(SG)|

|˝(GP)|

STI STII STIII STIV STV

6 6 5 6 4

4 4 4

4 4 4 4 4

|˝(FTP)|

|˝(GS)|

|˝(SNC)|

C(STi )

2 2

3 3 3 3

192 576 240 144 96

2 2 Total

the ST-DBN for ST+1 given the same starting state and evidence, using ıT = 5–10 years. The results for ST = Tall Palatable Grasses and with the same evidence are shown in Fig. 14. Even considering the slightly different time frames, these suggest an inconsistency in the modelling of the two time frames in Bashari et al.’s original BN.

5. Comparative complexity analysis We now have three different BN structures for modelling system state transitions over time: the ST-BN (a generalised version of Bashari et al.’s framework), an ordinary DBN (without an explicit state transition nodes), and the ST-DBN. As we have seen, the ST-BN and the ST-DBN have the advantage of modelling the state transitions explicitly in the network structure. In this section we compare the complexity of the different structures, in terms of the number of probabilities in the CPTs. For this metric is an important modelling consideration, in terms of both the elicitation burden on the experts (who have to provide the probabilities) and computational burden (storage and propagation). For each structure, we analyse the complexity of the generic case and then compute the actual numbers for the Rangeland example.

5.1. Notation and method

statespace size: C(S T +1 ) = |˝(S T +1 )| × |˝(S T )| ×

C(Y ) = |˝(Y )| ×

m 

The size of the CPTs for the X and F nodes are the same for all three generic models. Hence to compare the three models, we only need to look at the size of the CPTs for the next state node, ST+1 , and the transition nodes, ST, that is C(ST+1 ) and C(STi ), for 1 ≤ i ≤ n.

5.2. Complexity of the ordinary DBN Consider the generic ordinary DBN shown in Fig. 9, with m main influencing factors, the set of nodes F = F1 , . . ., Fm . Suppose that each influencing factor Fk has statespace ˝(Fk ), with the size of that statespace |˝(Fk )| = fk . The size of the CPT for ST+1 is then the product of the size of statespaces of all its parents (the F nodes, plus ST ), times its own

m 

k=1

k=1

|˝(Fk )| = n2 ×

fk

5.3. Complexity of ST-BN Consider the generic ST-BN shown in Fig. 5. Suppose that each transition node STi has ni + 1 states, made up of ni of the original n states, plus 1 for the impossible state. Suppose also it has mi parents, G1 , . . . , Gmi , and that the size of the statespace of each Gk is |˝(Gk )| = gk . Then the number of probabilities to represent the transitions in the ST-BN is the sum of the size of the CPTs of each of the n transition nodes, which is, in turn, the product of the statespace sizes of the ST node itself and its parents. n 

C(STi ) =

n  i=1

=

n 

 |˝(STi )| × |˝(S T )| ×



mi 

n(ni + 1)

i=1



mi 



|˝(Gk )|

k=1

gk

k=1

Hence, in the worst case, if ni = n and G = F then the size of the CPT for each STi is (slightly) more than for ST+1 in the ordinary DBN, and the use of the ST nodes has increased the number of probabilities nfold. However if, for example, each transition has only 2 influencing factors (i.e. mi = 2) and all factors are binary (i.e. gk = 2), then the number of probabilities is:

|˝(Zk )|

k=1

m 

The key point to note is that this is exponential in m, the number of main influencing factors. If then, for example, all the main factors were binary nodes, the number of probabilities in the CPTs will be n2 2m . For the Rangeland example, with n = 5, and with the state space sizes of the 5 main influencing factors being GP = 4, SG = 4, FTP = 3, GS = 3, SNC = 2, their product = 52 × 4 × 4 × 3 × 3 × 2 = 7200.

i=1

We use the following notation in our complexity analysis, which is fairly standard in the BN research literature (e.g., Dean and Wellman, 1991; Korb and Nicholson, 2010). For each node Y in the BN, we refer to the set of n values {y1 , . . ., yn } it can take as its statespace, denoted ˝(X). The size of the statespace, which is of course n, the number of values in that set, is denoted |˝(Y)|. If Y has m parents {Z1 , . . ., Zm }, then the size of the CPT for Y, denoted C(Y) is the product of the size of statespaces of all its parents, times its own statespace size:

1248

=

n 

(n(2 + 1)(2 × 2)) =

12n(n + 1) = 6n(n + 1) = O(n2 ) 2

i=1

where O(·) is the so-called “Big-O” notation from complexity, indicating here that the dominating aspect of the complexity is n2 . For the Rangeland example (where n = 5), the breakdown of the calculation is given in Table 3; the total number of probabilities in the CPT is 1248, an obvious reduction from the ordinary DBN (7200, see above). 5.4. Complexity of ST-DBN Consider the generic ST-DBN in Fig. 10. Here we must add the CPT sizes for all the state transition nodes – which is exactly the same as that given above for ST-DBN – to the size of the CPT for

Please cite this article in press as: Nicholson, A.E., Flores, M.J., Combining state and transition models with dynamic Bayesian networks. Ecol. Model. (2010), doi:10.1016/j.ecolmodel.2010.10.010

ARTICLE IN PRESS

G Model ECOMOD-6010;

No. of Pages 12

A.E. Nicholson, M.J. Flores / Ecological Modelling xxx (2010) xxx–xxx

ST+1 , which is:





i=1

i=1

n

C(S T +1 ) = |˝(S T +1 )||˝(S T )| ×

we refer to as the Rangeland example) and for his prompt and helpful responses to our queries regarding details of the model. We also thank the anonymous reviewers to their useful comments and suggestions.

n

|˝(STi )| = n2

(ni + 1)

Although at first glance the complexity, which is exponential in n, appears to be a problem, it is important to realise that no modelling or elicitation is required to generate this CPT. Rather, it is a deterministic function of the parent nodes. Of course if the CPT is too large, than it cannot be stored in memory and used by the inference algorithms. But, for example, this is not the case for the Rangeland example, where the size of the CPT for ST+1 is 52 × 6 × 6 × 5 × 6 × 4 = 108000. Thus, the overall complexity is: n 

C(STi ) + C(S

T +1

)=

i=1

n 



mi 

n(ni + 1)

k=1

i=1

and the complexity 108000 + 1248 = 109248.

gk

of

the



+ n2

n 

(ni + 1)

i=1

Rangeland

ST-DBN

11

is

6. Conclusions The starting point for this work was Bashari et al.’s framework for combining state transition models and Bayesian networks that gave the advantages of both—an STM’s graphical visualisation of both transitions and their influencing factors, together with the Bayesian network’s quantitative representation of uncertainty by probabilities. Through a process of analysis and formalisation, we identified some limitations of their framework. We made some modifications that addressed most of the problems, resulting in the generic model we called ST-BN. We showed that dynamic Bayesian networks, a variant of BNs used to explicitly model systems over time, provided the explicit representation of the “next” state missing from ST-BN. We demonstrated that ordinary DBNs also had limitations—they did not visualise either the state transitions, or how only some factors influenced individual transitions. In addition, by not representing the structure inherent in a problem, they will often have very large CPTs, containing many zeros. We then presented a generic structure that combined STMs and DBNs—the so-called ST-DBN. While retaining all the advantages of the ST-BN, the ST-DBN has the additional advantage of explicitly representing the next state. We showed that this means it can be used iteratively to predict into the future more consistently than the ST-BN with different time frames. Finally, we provided a detailed complexity analysis of each of the three structures – ST-BN, ordinary DBNs and ST-DBN – in terms of the number of states, the number of main factors and the number of possible transitions. Both ST-BN and ST-DBN – which explicitly model all the transitions – only remain tractable when there are natural constraints in the domain. Thus we strongly advise modellers to analyse these aspects of their problem first before deciding to use the framework. Next we plan to formalise the methods for both constructing ST-DBNs, and for validating them against the qualitative STM catalogue scenarios. We noted earlier that we felt it was more internally consistent to use a point value for ıT and rollout the DBN, rather than having additional uncertainty in the time interval interval. Another alternative may be to apply some form of Continuous Time Bayesian Network (CTBN) Nodelman et al. (2002); this is also a topic for future research. Acknowledgements We thank Carl Smith for providing us with the network for the cleared Ironbark-spotted gum woodland in SE Queensland (which

References Bashari, H., Smith, C., Bosch, O., 2009. Developing decision support tools for rangeland management by combining state and transition models and Bayesian belief networks. Agricultural Systems 99, 23–34. Bestelmeyer, B.T., Brown, J.R., Havstad, K.M., Alexander, R., Chavez, G., Herrick, J.E., 2003. Development and use of state-and-transition models for rangelands. Journal of Range Management 56, 114–126. Borsuk, M., Stow, C., Reckhow, K., 2004. A Bayesian network of eutrophication models for synthesis, prediction, and uncertainty analysis. Ecological Modelling 173, 219–239. Boutilier, C., Friedman, N., Goldszmidt, M., Koller, D., 1996. Context-specific independence in Bayesian networks. In: Horvitz, Jensen (Eds.), UAI96—Proceedings of the Twelfth Conference on Uncertainty in Artificial Intelligence, pp. 115–123. Cain, J., 2001. Planning improvements in natural resources management: guidelines for using Bayesian networks to support the planning and management of development programmes in the water sector and beyond. Technical Report. Centre for Ecology and Hydrology, Crowmarsh Gifford, Wallingford, Oxon, UK. Dagum, P., Galper, A., 1993. Forecasting sleep apnea with dynamic network models. In: UAI93—Proc of the Ninth Conference on Uncertainty in Artificial Intelligence, San Mateo, CA, pp. 65–71. Dagum, P., Galper, A., Horvitz, E., 1992. Dynamic network models for forecasting. In: Dubois, D., Wellman, M., D’Ambrosio, B., Smets, P. (Eds.), UAI92—Proceedings of the Eighth Conference on Uncertainty in Artificial Intelligence. San Mateo, CA, pp. 41–48. Dawsey, W.J., Minsker, B.S., Amir, E., 2007. Real time assessment of drinking water systems using a dynamic Bayesian network. In: World Environmental and Water Resources Congress. Dean, T., Kanazawa, K., 1989. A model for reasoning about persistence and causation. Computational Intelligence 5, 142–150. Dean, T., Wellman, M.P., 1991. Planning and Control. Morgan Kaufman Publishers, San Mateo, CA. Forbes, J., Huang, T., Kanazawa, K., Russell, S., 1995. The BATmobile: towards a Bayesian automated taxi. In: Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence (IJCAI’95), pp. 1878–1885. Howard, R., Matheson, J., 1981. Influence diagrams. In: Howard, R., Matheson, J. (Eds.), Readings in Decision Analysis. Strategic Decisions Group, Menlo Park, CA, pp. 763–771. Jensen, F.V., Nielsen, T.D., 2007. Bayesian Networks and Decision Graphs, 2nd edition. Springer Verlag, New York. Kanazawa, K., 1992. Probability, time, and action. Ph.D. Thesis. Brown University, Providence, RI. Kjærulff, U., 1992. A computational scheme for reasoning in dynamic probabilistic networks. In: UAI92—Proceedings of the Eighth Conference on Uncertainty in Artificial Intelligence, San Mateo, CA, pp. 121–129. Korb, K.B., Nicholson, A.E., 2004. Bayesian Artificial Intelligence. Chapman & Hall/CRC. Korb, K.B., Nicholson, A.E., 2010. Bayesian Artificial Intelligence, 2nd edition. Chapman & Hall/CRC. Nicholson, A., Woodberry, O., 2010. The “Native Fish” Bayesian networks. Technical Report. 2010/3, Bayesian Intelligence. http://www.bayesianintelligence.com/publications.php. Nicholson, A.E., 1992. Monitoring discrete environments using dynamic belief networks. Ph.D. Thesis. Department of Engineering Sciences, Oxford. Nicholson, A.E., Brady, J.M., 1992. The data association problem when monitoring robot vehicles using dynamic belief networks. In: ECAI92—Proceedings of the Tenth European Conference on Artificial Intelligence, pp. 689–693. Nodelman, U., Shelton, C., Koller, D., 2002. Continuous time Bayesian networks. In: UAI02—Proceedings of the Eighteenth Conference on Uncertainty in Artificial Intelligence, San Francisco, pp. 378–387. Norsys, 1994–2010. Netica., http://www.norsys.com. Pearl, J., 1988. Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Mateo, CA. Renken, H., Mumby, P.J., 2009. Modelling the dynamics of coral reef macroalgae using a Bayesian belief network approach. Ecological Modelling 220, 1305– 1314. Rumpff, L., Duncan, D., Vesk, P., Keith, D., Wintle, B., in press. State-and-transition modelling for adaptive management of native woodlands. Biological Conservation. Saatkamp, H.W., Huirne, R.B.M., Geers, R., Dijkhuizen, A.A., Noordhuizen, J.P.T.M., Goedseels, V., 1996. State-transition modelling of classical swine fever to evaluate national identification and recording systems—general aspects and model description. Agricultural Systems 51, 215–236. Sadler, R.J., Hazelton, M., Boer, M.M., Grierson, P.F., 2010. Deriving state-andtransition models from an image series of grassland pattern dynamics. Ecological Modelling 221, 433–444. Shihab, K., 2008. Dynamic modeling of groundwater pollutants with Bayesian networks. Applied Artificial Intelligence 22, 352–376.

Please cite this article in press as: Nicholson, A.E., Flores, M.J., Combining state and transition models with dynamic Bayesian networks. Ecol. Model. (2010), doi:10.1016/j.ecolmodel.2010.10.010

G Model ECOMOD-6010; No. of Pages 12 12

ARTICLE IN PRESS A.E. Nicholson, M.J. Flores / Ecological Modelling xxx (2010) xxx–xxx

Shihab, K., Chalabi, N., 2007. Dynamic modeling of ground-water quality using Bayesian techniques. Journal of the American Water Resources Association (JAWRA) 43, 664–674. Stringham, T.K., Krueger, W.C., Shaver, P.L., 2003. State and transition modeling: an ecological process approach. Journal of Range Management 5, 106–113.

Uusitalo, L., 2007. Advantages and challenges of Bayesian networks in environmental modelling. Ecological Modelling 203, 312–318. Varis, O., Kuikka, S., 1999. Learning Bayesian decision analysis by doing: lessons from environmental and natural resources management. Ecological Modelling 119, 177–195 (19).

Please cite this article in press as: Nicholson, A.E., Flores, M.J., Combining state and transition models with dynamic Bayesian networks. Ecol. Model. (2010), doi:10.1016/j.ecolmodel.2010.10.010