3D Model Retrieval Based on 2D Slice Similarity Measurements Pu Jiantao1, Liu Yi1, Xin Guyu1, Zha Hongbin1, Liu Weibin2, Yusuke Uehara3 National laboratory on machine perception, Peking University, Beijing 1008711 Internet Application Lab., Fujitsu R&D Center Co., LTD2 Language Processing Lab., Information Technology Media Lab., Fujitsu Laboratories LTD.3 [email protected] Abstract In this paper, we present an approach based on 2D slices for measuring similarity between 3D models. The key idea is to represent the 3D model by a series of slices along certain directions so that the shape-matching problem between 3D models is transformed into similarity measuring between 2D slices. Here, we have to deal with the following problems: selection of cutting directions, cutting methods, and similarity measuring. To solve these problems, some strategies and rules are proposed. Firstly, a maximum normal distribution method is presented to get three ortho-axes that coincide better with human visual perception mechanism. Secondly, a cutting method is given which can be used to get a series of slices composed of a set of closed polygons. Thirdly, on the basis of 3D shape distribution method presented by Robert et al., we develop a 2D shape distribution method to measure the similarity between the 2D slices. Some experiments are given in this paper to show the validity of this method for 3D model retrieval.

1. Introduction 3D models are now playing an important role in many applications such as mechanical manufacture, games, biochemistry, E-business, art, virtual reality, etc., and the number of them is increasing greatly. How to find out the desired models quickly and correctly from a 3D model library has become a practical problem. As the most intuitive way to describe 3D models from the point of view of human perception, shape matching is the key of 3D model retrieval and its efficiency has a direct impact on the performance of 3D model retrieval systems. However, it is a very difficult problem, and many wellknown institutions and universities in the world are dedicating themselves to this research field. In this paper, we propose a method based on 2D slices to do 3D model retrieval. The key idea is to represent the 3D model by a series of slices along certain directions so that the shape matching between 3D models is

transformed into similarity measuring between 2D slices. In this process, three problems are involved: selection of cutting directions, cutting methods, and similarity measuring. To solve these problems, some strategies and rules are presented. Firstly, a maximum normal distribution method is proposed to get three ortho-axes that coincide better with human visual perception mechanism. Secondly, a cutting method is given which can be used to get a series of slices composed of a set of closed polygons, even though some models are formed by degenerated polygons. Thirdly, on the basis of 3D shape distribution method presented by Robert et al. [1], a method based on 2D shape distribution is presented to measure the similarity between 2D slices. This paper is organized as follows. Firstly, some related works are summarized in Section 2. Then, the basic idea of our method is explained in Section 3 with the help of slice-based shape representation. For the involved problems mentioned above, some approaches and rules are explained in Section 4, 5, 6 respectively. Finally, some retrieved examples are presented to analyze the validity of our method. At the same time, the future works are also discussed briefly.

2. Related works For the 3D shape-matching problem, many approaches have been proposed. Motofumi [2] described a web-based retrieval system for 3D polygonal models, in which a 3D polygonal model is described by the combination of several features. Examples of these feature descriptors include tensors, normals, volumes, polygonal vertices and polygonal faces. Zhang et al. [3] proposed a new set of region-based features for representing 3D models. Features such as volume-surface ratios, moment invariants and Fourier transform coefficients can be efficiently calculated from the mesh model directly. However, when a large number of feature descriptors are used for the query, the system may not be able to respond quickly because more computations are needed.

From the view point of statistics, Mihael et al. [4] described a method that characterizes areas of intersections with a collection of concentric spheres. The distribution of areas is normalized so that the overall volume is one and two distributions can be compared by computing their L2 difference. Horn et al. [5] characterized a 3D model in terms of its distribution of surface normal vectors. They aligned the EGI (Extended Gaussian Images) for each model based on its principal axes, and then compared two aligned EGIs by computing their L2 difference. In addition, Robert et al. [1] proposed a shape distribution method to measure the similarity between two arbitrary 3D polygonal models. The key idea is to represent the features of an object as a shape distribution sampled from a shape function measuring global geometric properties of an object. Its motivation is to simplify the shape matching problem to the comparison of probability distributions, which is simpler than other shape matching methods that require pose registration, feature correspondence, or model fitting. However, these methods could not depict the topology structure of 3D models, and the local shape features are not measured explicitly. Hilaga et al. [6] proposed a technique named “Topology Matching” in which similarity between polyhedral models is calculated by comparing skeletal and topological structures. Sundar et al. [7] described a novel method that encodes the geometric and topological information in the form of a skeletal graph and uses graph-matching techniques to match the skeletons and to compare them. The skeletal graphs can be manually annotated to refine or restructure the search. These topology-based methods are very intuitive and invariant to translation, rotation and scaling. However, they need a large amount of computation resources and can not be applied to real-time applications easily. In addition, how to register the correspondence of nodes of skeleton graph accurately and robustly is not an easy task. The methods mentioned above and some others [8, 9, 10, 11, 12] have their own advantages and limitations. However, they have one common characteristic: the original models could not be reconstructed precisely from the extracted features. It means that some shape features are ignored to some extent. The aim of the method proposed in this paper is to preserve the shape features as much as possible.

3. Slice-based shape representation The key of shape matching is to find out a kind of representation that can describe the shape precisely. The slice-base method proposed in this paper is to represent a 3D shape as a series of 2D slices. It arises from such a fact as described in Figure 1: since 3D mesh models can

be cut into a series of slices with polygon contours, why not the reverse? Could the reverse procedure be used to do shape matching? In this figure, the middle shape consists of 33 slices, while the right one consists of 100 slices. It is shown that a 3D model can be reconstructed precisely by overlapping a series of slices that represent the local contour of 3D model. The larger the number of slices is, the more precise the final 3D model will be.

Figure 1. Slice-based shape representation. The shape on the right is reconstructed using more slices than the middle one. As compared with other methods, this representation contains almost all features of 3D shape because the original shape can be reconstructed by these slices completely. In this way, the shape-matching problem could be transformed into the similarity measuring between 2D slices. To do this, however, several problems need to be addressed. (1) Cutting directions. Different cutting direction will lead to different slice sets that represent the same shape in different forms. Due to the fact that the similarity is a kind of human experience, a good slice set should be in accordance with human intuition of 3D shape. Furthermore, the selection of cutting directions have to be determined uniquely so that the similarity measuring could be done on the basis of one standard direction. Otherwise, it makes no sense to measure the similarity between two slice sets because these slices are not obtained along axes with the same visual standard. As shown in Figure 2, it is not proper to do similarity measuring along the side directions of two different bounding boxes.

Figure 2. Different bounding boxes for the same model. (2) Cutting algorithm. This step is to use a plane to cut the model into a series of slices along some axis. The procedure can be regarded as the intersection between polygon mesh and plane. However, it is not enough to use the intersection points to represent the slices. They should be organized properly to represent the topology of these slices for the sake of similarity computing. For example, the slices can be represented by a series of polygon loops

so that the slices can be described by some parameters, such as areas, contour lengths, etc., as shown in Figure 3. It can be seen that the slice on the right can reflect the geometric shape precisely at the cutting position. Thus, how to do this efficiently becomes the key of the cutting algorithm.

Figure 3. A slice example: the slice on the right is obtained from the model on the left at some position. (3) Similarity measuring between slices. After the slices of two models are obtained, the next step is to measure the similarity between them. In this measuring process, three problems are involved. Firstly, the two slice sets must have the same cutting direction so that the similarity measuring is feasible. For example, when similarity measuring is done between two models with the same shape, the cutting direction must be the same. Secondly, some parameters should be found out to represent the 2D shape of slices. As mentioned above, the slices consist of several polygons that reflect some local shape. Finally, the similarity for 3D shape can be obtained by calculating all these 2D shape similarities.

4. The selection of cutting directions One intuitive way to determine the cutting direction is to find out a unique bounding box that encloses the 3D model tightly. In this way, the three side-directions can be used as the cutting directions. In this paper, we propose a new method and some rules to determine the cutting direction.

(1)

(2)

(3)

(4)

(5)

(6)

Figure 4. Bounding box eamples. The bounding boxes shown in (1), (2) and (3) are obtained by inertial

principal axis method, while the boxes shown in (4), (5) and (6) by the maximum normal distribution method. Until now, many box-creation methods have been proposed [13, 14, 15, 16, 17]. The simplest bounding box is AABB [15], but it is not unique because the side directions of the box are determined by the axes of universal coordinate system. Gottschalk has presented an inertial principal axis method [15, 17] to compute a goodfit bounding box using a statistical method. By Computing the eigenvctors of a 3×3 covariance matrix, the direction vectors for a good-fit box can be obtained. In Figure 4, the bounding boxes shown in (1), (2) and (3) are some examples obtained by this method. Despite the fact that the inertial principal axis method can obtain three ortho-axes uniquely, they are still not ideal for our purpose because the three directions are not in accordance with our visual perception mechanism. As shown in Figure 4, the bottom two bounding boxes are better than the top two. Therefore, we propose another method named maximum normal distribution to compute a unique principal axis set. As the name explains, we use the maximum normal distribution as one of the principal axis. Firstly, we must compute the normal direction N for each triangle Δ pqr and normalize it. It is the cross product of any two edges as pq × qr (1) N= pq × qr Secondly, the area of each triangle a is calculated and the areas of all triangles with same or opposite normals are added. Here we think the normals that are located in the same direction belong to a same distribution. Next step is to determine the three principal axes. From all normal distributions, the normal with the maximum area is selected as the first principal axis bu. To get the next principal axis bv, we can search from the remaining normal distributions and find out the normal that satisfy two conditions: (1) with the maximum area, (2) orthogonal to the first normal. Naturally, the third axis can be obtained by doing cross product between bu and bv: b w = bu × bv .

Figure 5. An example for the bounding box of a face model, in which the maximum normal distribution method fails. The bounding box on the left is obtained

by the maximum normal distribution method, while the one on the right by the inertial principal method. To find the center and the half-length of the bounding box, we project the points of the polygon mesh onto the direction vector and find the minimum and maximum along each direction. Finally, the positive direction for each principal axis has to be decided. For this purpose, we propose a rule: the further side from the centroid is the positive direction. In Figure 4, the boxes shown in (4), (5) and (6) are obtained by the maximum normal distribution method, and they looks much better than (1), (2) and (3). For models with obvious normal distributions, such as CAD models, the maximum normal distribution method is better than the inertial principal axis method. However, for models without obvious normal distributions, as shown in Figure 5, the former method will fail because the normal distribution has random property for this case. From Figure 5, we can observe that the inertial principal axis method is good in describing the mass distribution of 3D models, and it can find out the symmetric axes according to the mass distributions. Therefore, to overcome this limitation and make full use of the merits of the two methods, we propose a rule to combine the two methods: select the bounding box with smaller volume as the final box. Its validity has been proved by a large amount of models in our 3D library consisting of more than 2,700 models.

to it and mark this point as visited. In other words, if two intersected points are located on the separate edges of a triangle in SIT, then we can determine that the two points are adjacent to each other and they could be connected directly as the neighbor vertexes of a polygon. Thirdly, choose the first selected point as the base point (i.e., the point mentioned in the first step.) and repeat the second step until no points can satisfy the conditions in step two. Until this stage, all visited points are formed into a loop and can be regarded as the vertices of a polygon. The aim of this step is to maximize the polygon as large as possible, otherwise many small polygons will be generated and intersections between them will occur. Finally, check whether there is unvisited point in SIP. If yes, repeat above steps from step one again. Otherwise, the task of polygon generation is ended, and a set of polygons will be formed, and then we can step into the next stage. Step 3: Eliminate unclosed polygons. It is well known that a polygon could not be formed by less than three points. Therefore, if there exists a polygon that contains a

1

2

b 4 3′ 3 c

d

a

6

5

5. Cutting method The cutting process is to perform the intersection between a cutting plane and a model, and it will obtain a series of points. Due to the reason that these discrete points have no explicit connection relationship, we have to link these points reasonably according to their intrinsic neighborhood. Since the model is a polygon mesh, a natural way to describe the slices is to use polygons. Therefore, the task of the cutting algorithm is to obtain a series of slices that consist of polygon sets. For this purpose, we propose an algorithm outlined by the following processes: Step 1: Compute the intersection between the specified plane and the polygon mesh, and save the intersected points and triangles into two separate arrays named SIP and SIT respectively. It is noticed, however, that the same intersected point can not be recorded twice. Step 2: Organize the intersected points into a set of polygons according to their neighborhood-distribution on the surface of the model. Four steps are included: Firstly, select one intersected point randomly from SIP and marked it as visited. Secondly, search from the remaining unvisited intersected points to find out one that is adjacent

d

b 4 1

2

3

5

e

c

only two or one points, then delete it. Figure 6. T-vertex (left) and edge-cracking cases (right). The crossing lines represent the cutting planes. In the left, vertex b is a T-vertex, as it belongs to the polygons to the left of it but it is not a vertex of the triangle acd. In the right, the gray region is an edgecraking where the left and the right polygons don’t meet properly. After these processes, we can obtain a set of polygons. This method is only suitable for ideal models. For models with T-vertex and edge-cracking as shown in Figure 6, some additional processes should be adopted. (1) T-vertex case. In the left of Figure 6, vertex b is a T-vertex, as it belongs to the polygons to the left of it but it is not a vertex of the acd. 1, 2, 3, 4, 5 are the intersected points between the polygon mesh and the cutting plane. On the position of 3, there are two overlapped intersected points, 3 and 3ˊ. Let’s say the point 3 is the intersected point between the plane and edge bc, the point 3ˊis the intersected point between the plane and edge ac. As

3D Models

mentioned in the first process of Step 2, only one point could be reserved, while another one must be discarded. If point 3 is reserved, this algorithm will not obtain such a conclusion that 3 and 4 are neighbor vertices of the same polygon because bc and cd are not the edges of the same triangle. On the contrary, if point 3ˊis reserved, we can reach the right conclusion. To overcome this limitation, we adopt a special process: when two intersected points are the same, the two edges should be saved respectively and the point be marked. If this point is visited, the algorithm will decide which intersected edge should be checked and whether the two intersected edges are located on the same triangle. (2) Edge-cracking case. In the right of Figure 6, the gray region is an edge-cracking where the left and right polygons don’t connect seamlessly. When the point 3 is visited, the point 4 will not be chosen as a neighbor vertex because the triangle ace does not belong to the polygon mesh. To overcome this limitation, a judgment is added: check whether the intersected edges form a new triangle. If yes, then the two points are neighbor vertex of a polygon. As shown in Figure 7, some examples are listed, and they are obtained by this proposed method.

Robert et al. [1] proposed a new method to compute the shape feature of arbitrary 3D polygonal models, in which 3D shape can be represented as a probability distribution sampled by a shape function. Our motivation is to extend this method to the 2D case. In other words, the similarity between 2D polygon sets can be obtained by measuring their 2D shape distributions. Like the 3D case, the shape-matching problem between slices can be summarized by following three steps:

(a)

(b)

Figure 8. Illustration about how to measure the similarity between (a) and (b), which both consist of polygons. (1) Uniform sampling on the edges. For the polygons contained in a slice, we adopt a uniform sampling principle. If the total length of edges is L and the sampling number is n, then for one edge defined by two vertices represented by vectors Ai and Aj, the sampling number k and the sampled point Si are defined by n × Ai A j L L S i = A i + i × Di n k=

(4)

Slices

in which Di is the normalized vector from Ai to Aj. Such an example can be illustrated by Figure 9.

Figure 7. Some slice-polygon examples.

6. Similarity measuring method As shown in Figure 7, these slices represent the shape features distributed in certain direction, and hence the shape-matching problem is converted into how to measure the similarity between the corresponding polygon sets that are obtained from different models. This problem can be illustrated by Figure 8. It is not a similarity problem between two single polygons but one problem between polygon sets that might contains more than one polygon. Until now, many methods [18, 19, 20, 21] have been proposed to compute the similarity between two polygons, but they cannot be easily extended to the similarity measuring between polygon sets. In this paper, a 2D shape distribution method is presented to accomplish this task.

Figure 9. One 2D uniform sampling example. (2) Shape distribution generation. Robert et al. describe five kinds of shape functions [1] to measure geometric shape of point clouds: D1, D2, D3, D4, A3. For 2D cases, we adopt the D2 distance to measure the shape features of polygons because other distance metrics are designed specially for 3D cases. D2 distance is to measure the distance between two random points on the surface. Figure 10 shows the two distributions formed by polygons in Figure 9. The horizontal axis represents the distance between two random points, while the vertical

axis represents the number with the same distance. The black curve is the distribution of Figure 9(a), and the gray one the distribution of Figure 9(b).

and/or double surfaces. Table 1 lists some examples obtained by our prototype system. It can retrieve the queried models accurately if the queried models are in the database. As illustrated in Table 1, there are still some models that don’t coincide with human perception mechanism very well. The reason lies in the fact that this algorithm still belongs to the category of statistics method. It can only measure the similarity along some directions between models.

(3) Similarity measuring. This step is similar to 3D case, in which a normalization step should be taken before the similarity measuring is done. Generally, two methods for normalization can be adopted: (a) align the maximum D2 distance values; (b) align the average D2 distance values. For the first normalization method, the maximum values of the two shape distributions have to be adjusted to the same value, which is used to normalize the shape distribution. The other one is to use the mean value of distance to normalize the shape distribution. To alleviate the influence of noises, we adopt the second one as the normalization standard. The similarity between two slices can be obtained by calculating the difference between their distributions. The final similarity between models can be measured by 3

n

Top seven similar models

Figure 10. Two distributions formed by the two polygons in Figure 9.

Query

Table 1. Some retrieved samples

m

Similarity = ∑∑∑ ( sdij − kdij ) 2

(2)

d =1 i = 0 j = 0

where d is the number of cutting direction. There are only three choices that we can select as the cutting directions. n is the cutting number along one direction, m the divided histogram number of the shape distribution curve, s and k the probability at certain distance. This slice-similarity approach owns the same advantages as the 3D shape distribution method. It is very simple and easy to implement. It also has some unique properties: (1) insensitive to geometric noises; (2) invariant to translation, rotation and scaling; (3) no need to find out the feature correspondences between models.

7. Experimental results and conclusion The proposed method has been implemented in C++ on windows platform, and all involved basic functions are encapsulated. In order to test its validity, a database with more than 2,700 3D models is built up elaborately. It mainly consists of 3D CAD mechanical parts that are collected from the Internet. The models are not all welldefined polygon meshes; many of them contain cracks, self-intersections, missing polygons, one-sided surfaces,

Table 2. Retrieval performance Average accuracy (Percent: %) Top 5 Top 10

Average retrieval time (second)

Average preprocessin g time (second)

53.7%

1.7~6.22

4.5~300

74.1%

Table 2 shows the retrieval efficiency. It is obtained on the basis of a large 3D model library. Given a reference model, the retrieval accuracy is measured by checking the probability that the models within same group appear in higher similarity. It can be seen that one disadvantage is that it needs a long time to finish one retrieval task because the process to form the polygons in one slice

needs a long time. Therefore, a preprocessing step is needed to complete the cutting process beforehand.

8. Conclusion and future work In this paper, we present a new 3D model retrieval method based on the similarity measuring between 2D slices. As compared with other methods, its aim is to represent the 3D shape precisely. The contributions of this paper can be summarized as following: (1) a method is proposed to obtain the unique ortho-axes that are in accordance with human visual perception mechanism; (2) a cutting method is presented that can be used for arbitrary 3D models, including degenerated models; (3) a 2D shape distribution method is given to do 2D shape matching between polygon sets. However, it still could not measure the similarity between local shapes explicitly. This problem will be addressed specially in our future work, and we are now exploring one topology-based method in which the topology of 3D models is derived from the slices. Finally, how to measure the validity and efficiency of a 3D model retrieval system is also an important problem that we will explore in the future.

Acknowledgement This work is supported in part by the NSFC grant (No.60333010).

References [1] Robert, O., Thomas, F., Bernard, C., and David, D., “Shape Distribution”, ACM Transactions on Graphics, 21(4): 807-832, 2002. [2] Motofumi, T. S., “A Web-based Retrieval System for 3D Polygonal Models”, Proc. Joint 9th IFSA World Congress and 20th NAFIPS International Conference (IFSA/NAFIPS2001), pp.2271-2276, Vancouver, 2001. [3] Zhang, C., and Chen, T., “Indexing and Retrieval of 3D Models Aided by Active Learning”, Proc. ACM Multimedia 2001, pp. 615-616 Ottawa, Ontario, Canada, 2001. [4] Mihael, A., Gabi, K., Hans-Peter, K., and Thomas, S., “3D Shape Histogram for Similarity Search and Classification in Spatial Databases”, Proc. 6th International Symposium on Spatial Databases, pp.207228, HongKong, China, 1999. [5] Horn, B., “Extended Gaussian Images”, Proc. IEEE 72, 12(12), pp.1671-1686. New Orleans, USA, 1984. [6] Hilaga, M., Shinaagagawa, Y., Kohmura, T., and Kunii, T.L., “Topology Matching for Fully Automatic Similarity Estimation of 3D Shapes”, Proc. SIGGRAPH 2001, Computer Graphics Proceedings, Annual Conference Series, pp.203–212, Los Angeles, USA, 2001.

[7] Sundar, H., Silver, D., Gagvani, and Dickinson, S., “Skeleton Based Shape Matching and Retrieval”, Proc. Shape Modeling International 2003, pp.130-142, Seoul , Korea, 2003. [8] Badel, A., Mornon, J. P., and Hazout, S., “Searching for Geometric Molecular Shape Complementary using Bidimensional Surface Profiles”, Journal of Molecular Biology, 10 (12): 205-211, 1992. [9] Bardinet, E., Vidal, S. F., Arroyo, S. D., Malandain, G., and De La Blanca Capilla, N. P., “Structural Object Matching”, Tech. Report. DECSAI-000303, Dept. of Computer Science and AI, University of Granada, Spain. 2000. [10] Michael, K., Bernard, C., David, D., Thomas, F., and Szymon, R., “A Reflective Symmetry Descriptor for 3D Models”, Algorithmica, 38(1):201-225, October 2003. [11] Siddiqi, K., Shockoufandeh, A., Dickinson, S. J., and Zucker, S. W., “Shock Graphs and Shape Matching”, International Journal of Computer Vision, 35(1): 13–31. 1999. [12] Vranic, D.V., and Saupe, D., “Description of 3DShape using a Complex Function on the Sphere”, Proc. IEEE International Conference on Multimedia and Expo (ICME 2002), pp.177-180, Lausanne, Switzerland, 2002. [13] Eberly, David, 3D Game Engine Design: A Practical Approach to Real-time Computer Graphics, Morgan Kaufmann Publishers Inc., San Francisco, 2000. [14] Barequet, G., “Efficiently Approximating the Minimum-Volume Bounding Box of a Point Set in Three Dimensions”, Algorithms, 38 (1): 91-109, 1999. [15] Gottschalk, S., Collision Queries using Oriented Bounding Boxes, PhD thesis, Department of Computer Science, university of North Carolina at Chapel Hill, 1999. [16] Ritter,J., “An Efficient Bounding Sphere”, Graphics Gems, Academic Press, pp.301-303, 1990. [17] Tomas, A., and Eric, H., Real-Time Rendering (2nd edition). A K Peters, Ltd, pp.564-567, 2002. [18] Esther, M., Arkin, L., Paul, C., Daniel, P., Klara, K., and Joseph, S.B., “An Efficiently Computable Metric for Comparing Polygonal Shapes”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(3):209216, 1991. [19] Lun, H.T., and Irwin, K., “A Two-Stage Framework for Polygon Retrieval”, Multimedia Tools and Applications, 11:235-255, 2000. [20] Faouzi, A.C., Bogdan, C., Mari, P., Pasi, R., and Moncef, G., “Shape Similarity Estimation using Ordinal Measures”, Proc. Workshop on Image Analysis for Multimedia Interactive Services, pp.139-146, Tampere, Finland, 2001. [21] Avis, D., and Elgindy, H., “A Combinatorial Approach to Polygon Similarity”, IEEE Transactions On Information Theory, IT-29: 148-150, 1983.