Motion Graphs in Character Animation Pascal Pohl [email protected] Matrikelnummer: 2501337 Saarland University SS08, Character Animation Seminar Supervisor: Alexis Heloir

Abstract. This paper introduces the use of motion graphs in the field of character animation. First we will see how motion capture works. The paper then shows an approach how one can improve the use of motion capturing data to gain more control over the character. It will be explaind what are motion graphs and how to build them out of a set of different motion clips. After that it is shown how the resulting motion graph is used to generate new animation clips. It points out the means of control offered to the user and in what way these influence the result. At the end a more recent approach is briefly described which uses motion graph but uses some other techniques to improve of user control.

1

Introduction

When thinking about to animating a virtual character, one has to decide what technique to use. There are severeal approaches, each with its pros and cons. Some approaches yield to good quality in the resulting animation, others allow more control to the user. The aim is to achieve a good quality in both goals. Since the motion capture process yields to a set of very good animations it is desirable to expand the restricted use of these data. After a motion capturing process one gets a set of single motion clip, each recorded in the studio. When animating a virtual character one can just use these clips one after another. Every scene you wish to be performed by the character must be played be the actor. But this prohibits interactive control of the animation. The idea is now, to add more connections between the clips and especally between short parts of the clips. This offers the user the possibility to use the small parts of the recorded clips in a lot of new sequences. We will also see that with the use of motion graphs it is possible to lead a character (interactive) along a desired path. In the first section we give a short overview over what motion capture is. There we will see how motion capture works and why its it a nice thing when creating animation where a good quality is neccesery. But as we will recognize, motion capture is not flexible. Next is to indrocude the idea of [1] and to explain what motion graphs are and why they are useful. Then we show how to build a motion graph out of the data gained by motion capture. Then it is shown how to use that structed data. It will be explained how to get new animation clips by

2

Motion Graphs in Character Animations

searching the motion graph. At the end of the paper we introduce some further work which is based on motion graphs and adds more control to the animation process. They used an additional structure to give the user more (interactive) freedom and easier control in the animation.

2

Character Animation - Motion Capture

There exist many approaches dedicated to the animation of a virtual character. According to the requirment, one has to choose that technique that yields the best result. Motion Capture means the process of recording real actors. These actors are marked with many reflexive dots and filmed by several cameras from diffrent positions. The gained informations are used to create corresponding motion clips for the virtual character. The movement of the markers define the motion of the joints in the animation. Each recording of a scene provides the data to create exactly the same motion. This means that all scenes that the animator wants to use, must be recorded previously. Giving the user the option to control the character freely would mean to keep a giantic amount of clips. Working with raw mocap data implicates generating animations by stringing single but complete clips together. But the big advantage is that this ensures realy good quality since all motions come from real actors and are hence realistic.

3

The Idea of Motion Graphs

As we have seen, motion capture results in high quality animations. But we have also seen that the dataset is very restricted in sense of mixing the recorded motion. It is very difficult to generate additional motion, it only possible to modify only very small parts. So when thought about a possibility of improving this, they developed the idea of motion graphs. They searched for a new way to reorganize the stored data. As there exists several ideas of data structures such as trees, graphs or stacks, there are also more approaches on structuring the motion data. [1] decided to use a directed graph to make the data more easier to handle. They split the recorded motion data into small clips of a couple of frames. Then they use this motion clips as edges in the directed graph. They then try to find clips which include a similiar posture and use them as nodes. All motion clips starting and ending respectively in that pose get connected to this node. So the node serves as a choice point between all incoming and outgoing motion clips. So we can use two different transitions from two separate recorded scenes. But this also means that the graph also contains pieces of automatically generated transitions. As we obtain a directed graph from this process, the generation of new animations can benefit from algrorithms from AI tasks and graph theory. One way to organize the ”knowledge” of an artificial intelligens is with the help of directed graphs. The research in this fields revealed a lot of algorithms on evaluating graphs. Since motion graphs represent the data in the same way transfering the algotithms is feasible. There can be used to search for graph walks leading to animations satisfying user inputs. First we describe in detail,

Motion Graphs in Character Animation

3

how such a motion graph is built. The goal is to get a good deal between quality and a goog connectivity of the graph. After explaning the contruction of the motion graph we show how one gets back new animations out of a graph walk. Then we show how motion generation can be parametrized.

4

How to Build a Motion Graph

Since the basis of motion graph is motion capturing, it starts with a database of several motion streams. These contain the single recorded motions. The first step when constructing the graph, is to split these streams into small clips of several frames. These motion clips serve as our transition edges. This is shown in Fig.1. The nodes of the graphs are the connections between these motion clips and the points at which one can choose between several outgoing motion clips. When later animating a character transiton after transition is taken and strung together to the completed animation. After the first step of building the graph consists only of simple parallel graphs where every node has one incomming and one outgoing transition (motion clip). But that means that nothing is gained so far. A more useful graph for motion synthesis must contain many more transitions. More outgoing transitions in a node means that there have to be also more incoming edges. The goal is to find such points where more transitions can be connected. The problem now is that in general the motion of different clips is not sufficiently similiar enough to get connected directly. This means that in order to achieve seamless transitions new motion must be generated. But we will first explain how to find candidate points for new transitions. Then we pay attention on generating these transitions and finally there will be some work to finalize the motion graph.

Fig. 1. Starting with two single clips, first step is to split them into more clips and add corresponding node in the graph

4.1

Finding Candidate Transition Points

As we have said, the points to create new transitions must contain sufficiently similar postures. But what means sufficiently similiar in detail? A frame of mo-

4

Motion Graphs in Character Animations

tion capture data contains vectors, representing the root position of the skeleton and the additional joint rotations. A naive way to solve this problem would be to compare some vector norm of the two frames. But this would go wrong since some important points are not respected: 1. When just regarding vector norms there is no possibility to add some wheights to the different joints of the character. That system has also to be variable, differnt poses demand different wheighting. 2. The posture of the character is only defined to a rigid 2D coordinate transition. It plays no role where in space the character performs a motion, it remains unchanged when translating or rotating in the room. 3. The comparision only of pairs of frames would not contain informations like speed and accelerations.

Fig. 2. Two frames showing a similar pose

This means that we have to find another measure for the distances. And there is a very simple approach. Just see the character in the frame as what it is, a cloud of joint in the relative cartesian space. Additional labels are used to store some more information in the frame (kind of movement e.g. running, sneaking). To compare two of the frames now, one has just to compute the weighted sum of the distances of the single joints in both frames. That leads to the following equation: X dist(Ai , Bi ) = ωi kpi − TΘ,x0 ,zo p0i k2 i

Motion Graphs in Character Animation

5

Fig. 3. For each joint we compute the dislocation of the joints

The Tθ,x0 ,z0 is the linear transformation to solve the problem of the rigid coordinate system and aligns the Bj to the Ai . The weights ωi can be choosen to get more importance on some joints as requiered. Fig.3 illustrates computing of the dislocations for each joint. If we only compare each two frames, we still have the problem that nothing can be said about velocities and accelerations. For example walking and running are very simmilar at some points of movement but vary in the amount of displacement of the joint from frame i to i+1. To get rid of that problem, not only two frames are compared but two windows of frames of a certain length k. Two motions of different speed will then have more differences in the position of the joints. So it is posssible to take more parameter into account. When we now calculate the differences of the frames, we will get a 2D error function which we then have to minimize in attemp to find good transition points. In Fig.4 you see a typical result for such a error function. In x-direction we have frames AI and in y-direction frames Bj . Pixel (i,j) denotes then the error for the corresponding frames. All local minima of these error function seem to be good candidates for starting new transitions. But these are local minima what means that they are better then ohter points. To guarentee that they are really good enought for transitions, one has to ensure that the dist(Ai , Bj ) is smaller than a threshold defiened by the user. This threshold deals between a good quality of the generated transitions (small error) and a high connectivity of the graph. Regarding the needs of the motions in the clips, this threshold can be variable. Smooth and slow motions need a better blending as fast, bold motions.

6

Motion Graphs in Character Animations

Fig. 4. Example for a typical 2D error function, white means lower error, red dots mark good spots for transitions

4.2

Create new transitions

After the points for the new transitions are found, new motion clips must be generated. This is done by blending between the frames of the two windows of frames. For Ai , Bj fulfilling the requirements, the first step is to translate the frames of B to the position of A again. Then we perform a linear transition between the position of the root joints and a spherical interpolation between the rotations of the ohter joints. The interpolation of the frames is done between the frames Ai to Ai+k−1 and Bj−k+1 to Bj in the following: Rp = α(p)RAi+p + [1 − α(p)]RBj−k+1+p where Rp means the position of the root joint in frame p of the transition. The interpolation between the rotations is performed by: qpi = slerp(qAi+p , qBj−k+1+p , α(p)) The weight function of the interpolation process must fullfill the conditions that α(p) = 1f orp ≤ −1, α(p) = 0f orp ≥ k in order to maintain continuity: α(p) = 2(

p+1 2 p+1 3 ) − 3( ) + 1, −1 < p < k k k

The interpolation may hurt the constraints of the original motion clips. In the new transition contraits come up by their own ”interpolation”. If a constrain is active in the first but not in the second clip, it is active for the first half of the new transition frames and passive for the second half.

Motion Graphs in Character Animation

4.3

7

Pruning the Graph

When working with the generated graph it is not said that it is possible to create infinitly long animations. The contruction of the motion graph may produce some artefacts. It could contains some nodes and edges which impede any further proceed of the generation process. There are three kinds of troubling nodes: – Deadends: when entering a dead end, no further motion is possible because no outgoing transition exists. Nodes that are not part of any cycle are the start of such dead ends – Sinks: after enterign a sink, not all other nodes of the graph are any longer reachable. There may be still a possibility to produce infinite long motion by running through loops but is not possible to perform all motions in the graph. – Problems with the labeling: the generation process may produce some nodes where an edge with label A is incomming but there are only outgoing transitions labeled B. That means that an action of style A always has to be followed by a motion of style B (e.g. a fighting clip always has to end in a huging scene) Pruning now means that before using the motion graph we remove all these nodes. Ohterwise automatically computed graph walks would contain nonrealistic locomotions or restrict the possibilities of the user. The aim of the pruning process is to enable arbitrary long graph walking starting in any node. First step of the pruning is to associate every frame of the original motion clips with a set of labels. This set may also be empty. Say there are n unique sets. So we have to build a subgraph for each of this sets, containing all edges whose frames are labeled exactly the same way. Computing the stronly connected components (SCCs) is up next. A SCC is a maximal set of nodes such that there is a path from u to v for every pair of nodes (u,v) in this subgraph. We now have to remove all those edges in the subgraph who not attach two nodes in the largest SCC of every subgraph. At least we reject all nodes without edges. Problems occur if the SCC contains too few frames or some SCCs have no transitions in between. The pruning process is done automatically but not sufficient. We then prune it manually which may take some time. Increasing the thresholds in the transition building process in Fig.4.1 would solve this problem.

5

Producing Motion from a Graph Walk

If we have a complete path from one node of the graph to an other, it is very easy to generate the corresponding motion of the character along this path. The path of the graph walk consists of a sequence of transitions. But in our motion graph these transitions correspond to motion clips. This means that to generate the animation we have to join these clips together. The last thing to do is to translate the motion clips to the right position and orientation with an appropriate translation. These translation are multiplied from edge to edge when passing the path.

8

5.1

Motion Graphs in Character Animations

Searching for motion

The more interesting task is to find such graph walks according to the users inputs. We are working on a directed graph so we can use to solve this problem algorithms from graph theory and AI-tasks. A na¨ıve approach is to use a cost function g(w,e) which computes the error of taking transition e after path w. We are able to compute the total error of a path w by f (w) = f ([e1 , e2 , . . . , ei ]) =

X

g([e1 , . . . , ei−1 ], ei )

i

The function g(w,e) must be nonnegative so that we can’t have negative cost by edding an additional edge. The user also has to define a halting condition. This condition specifies if a graph walk is complete or not. It determines if the result of the graph search is good enough to satisfy the user. f (w) ≤ δ The goal now to find a graph walk which minimizes f(w) and fullfills the user inputs. There are several possibilties to perform a graph search,[1] used a depth first search with a branch and bound strategy. Branch and bound means that branches of the search that could notlead to any improvement of the path get eliminated. Since our g(w,e) never decreases, the computed f(w) is a lower bound to the cost of any path (w+v). So after finding the first complete path walk we store the cost of this w. From that point on we safe always the best path wo pt with cost f (wo pt) an look whenever adding an additional edge e to the current path w if f (w) + g(w, e) < f (wo pt). If this condition is violated, we halt the search on this branch and proceed with the next one.

5.2

Path Synthesis

In the last chapter we have seen how the user can determine criterias for a graph search. We take the example of path synthesis to show how this works in detail. If we want our virtual character to walh along a certian path, we have to find a corrsponding weight function g(w,e) and a threshold δ as halting condition. The goal is to fit the desired path as good as possible so a usefull weight can be found be calculating differences between the actual position of the character and the wanted. A simple way to compare both paths P (original) and P’ (the generated) is to project the root joint of the character on the floor at each frame yielding a linear curve. Then it is easy to calculate g(w, e) =

n X

kP(0 s(ei )) − P (s(ei ))k2

i=1

The completeness condition of P’ is if P’ is suffcient long enough or exceeds the length of P.

Motion Graphs in Character Animation

6

9

Remarks on Searching Algorithms

Since the searching for motion in the motion graphs always is the problem to define a cost function w, there is the question what good canditdates for such a function are. If we for example decide to give the algorithm just a starting posture at point A and an ending posture at point B, we would get an animation that looks not very pretty. Because nothing is said about the motion between those two postures, the algorithm just confines to finding an animation that fits close to the given endposition. So it may be sufficient just to find a certain style of motion instead of one specific pose. For example if we want the character to jump at the end of the animation not one special frame with the character in the air should be the end pose but any motion clip labeled ”jump”. This leads us to two important hints: – The function g should give some guideline through the whole animation not just at the beginning and the end. – One should give the algorithm as much freedom as possible to make it easier to complete the animation

7

Conclusion

We have seen that motions graphs are a good concept to preserve the good quality of motion capture. We can now produce good animations but the higher connectivity of the graph (in comparision to the set of the origial captured clips) gives the user more control. Because the main work (to build and prune the graph) has to be done before generating the animations, interactive control of the virtual character is possible. But although we can now combine the motion clips in many different orders, the animation is still bound to the recorded scenes. The computed blending do just connect two slightly different original clips. So it is still quiet expansive to get the basic data set. With the size of the database also the amount of work to do increases since every pair of frames in the database must be compared which each other. One good aspect is then enlarging the dataset by adding new frames, work can be kept and only the new frames must be compared. The number of new transition will also be rather small in relation to the amount of clips in the database. So the idea of motion graphs would also for large data work. We have seen that different weigth(functions) lead to different paths and animations. On thing to improve would be to find a way to modify these parameters and thesholds automatically. We can’t fix the parameter because different scenes need differnt values to get an optimal result. Perspective The simple use of motion graph is low level, this means we create our motion from path walks over the graph. But people want somehow behavioral control making it easier to describe the desired animation in a more natural way. To do so, Lee et al. introduced in [2] a two layer control structure. The lower layer is a motion graph and keeps close contact to the motion details. The additional higher layer consists of a behavioral build structure. In a statistical

10

Motion Graphs in Character Animations

modell different kinds of character behavior is organized such that we can choose the most probable motion for the next frame. In [2] they are using this structure to offer two ways of control. First possibility is Choice Control, the animation software offers the user a limited number of new positions and poses the character can reach in the next step, as the avatar moves the programm adaptes the eligible postures. Second method is Vision Control, this means that an actor is filmed and the recorded motions gets compared with the content of the database. The high level structure helps to simplify the search for similar motion. So we see that motion graphs are a good basic structure. But when working with large databases it helps to have a second structure which provides a more abstract way to specify the behavior.

References 1. L. Kovar, M. Gleicher, and F. Pighin. Motion graphs. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, 2002. 2. J. Lee, J. Chai, P. Reitsma, J. Hodgins, and N. Pollard. Interactive control of avatars animated with human motion data. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, 2002.