Computers & Graphics 25 (2001) 449–461

Technical Section

Mesh refinement with color attributes$ P. Rigirolia,b, P. Campadellic, A. Pedottib, N. Alberto Borghesea,*,1 a

Laboratory of Human Motion Study and Virtual Reality, MAVR, Istituto Neuroscienze e Bioimmagini CNR, Scientific Institute H S. Raffaele, LITA, Via f.lli Cervi, 93 - 20090 Segrate, Milano, Italy b Department of Bioengineering, Centro di Bioingegneria, Fondazione ProJuventute, Politecnico di Milano, Pzza Leonardo da Vinci, 32 - 20133 Milano, Italy c Department of Computer Science, University of Milano, via Comelico, 33, 20100 Milano, Italy

Abstract Although many sophisticated solutions have been proposed to reduce the geometry complexity of a 3D mesh, few of these take into account color and texture fields, which are fundamental for a visual appearance of high quality. We propose here an innovative solution which combines the concept of hierarchy in refining a mesh with color attributes with mesh regularization. It operates subdividing recursively the faces of a 3D mesh, which was previously simplified according to geometrical criteria only. New vertices are introduced by splitting a face until a maximum color error is achieved. The error is evaluated in CIE-L
u
v
iso-perceptual metric. Splitting is carried out using a new strategy, which chooses between either binary or quaternary subdivision schema, in order to achieve 3D mesh regularization. In this respect our method improves other hierarchical subdivision techniques (applied for example to radiosity and height fields). If the unsimplified version of the mesh is available, the procedure positions the new vertices on this mesh, decreasing, the topological error besides the color one. The method presented here is quite general and allows to create models of high pictorial quality (  20,000 faces), which can be manipulated in real-time on machines with no hardware accelerator for texture mapping. Main strength of the method is locality, which allows very fast operations as well as the possibility of local refinements. Results show how, at the expense of few more polygons, the final appearance of the model can be greatly improved in a very short time. The produced models are suitable to progressive transmission: first a light mesh can be transmitted with its per-vertex colors, then, in a second stage, the position and color value of the vertices added in the refinement stage can be sent to improve the pictorial appearance. # 2001 Elsevier Science Ltd. All rights reserved. Keywords: Surface simplification; Mesh re-tiling; Texture mapping; Level of detail; CIE-L
u
v
color space

1. Introduction

$

We have termed ‘‘Tex’’ the software, which implements the algorithms described here. In case of interest for this work, Tex will be put on our ftp site under the directory pub/tex/. Tex runs under Windows 32bits. The input format for the files, the procedure for aligning the mesh and bitmap are fully described in the Readme.txt file. *Corresponding author. Tel.: +39-02-21717548; fax: +3902-21717558. E-mail address: [email protected] (N. Alberto Borghese). 1 http://www.inb.mi.cnr.it/Borghese.html

One of the main goals of Computer Graphics is the visualization of complex 3D models with very high quality. To the scope, 3D scanners and CAD systems can easily produce models constituted of millions of triangles with detailed texture, which only top-level workstations can afford to manipulate. This has stimulated the development of a variety of mesh simplifying algorithms [1]. Most of these operate only on the geometrical attributes and are aimed to preserve the topological appearance of a model [2]; texture mapping remains the primary tool to give the color attribute [3–5]. However, the widespread of Web

0097-8493/01/$ - see front matter # 2001 Elsevier Science Ltd. All rights reserved. PII: S 0 0 9 7 - 8 4 9 3 ( 0 1 ) 0 0 0 6 8 - 1

450

P. Rigiroli et al. / Computers & Graphics 25 (2001) 449–461

applications has forced to rethink this view. In the Web, in fact, the narrow bandwidth of the network along with its latencies and the lack of complex hardware in many client sides are the critical bottlenecks. Walkthrough web systems, in which texture mapping is heavily present, are still very rare [6]. This makes texture mapping prohibitive slow at the client side for medium–large textures, and the paradigm ‘‘download and play’’ failing for complex 3D scenes. A possible solution is to transmit to the client only a 2D model (the actual 2D view of the scene) and to partition texture mapping between a server and its clients [6,7]. This solution may become inefficient when high-level interactivity with the 3D models is required. A different approach, proposed here, is to send to the client a simplified model in which the color field is adequately represented as per-vertex color. In this case, texture is incorporated in a color field, leaving to the client side only Gouraud shading. It is shown here that this choice is competitive and it is winning under certain situations (mainly when soft gradients and large textures are involved). An innovative simple solution is proposed for the construction of per-vertex colored light meshes. The algorithm works as a post-processor of a 3D mesh previously simplified using only geometrical criteria. First, the mesh is re-tiled (cf. [8]) adding new vertices, such that a maximum color error is achieved over the whole mesh with Gouraud shading interpolation. The algorithm adopts an innovative splitting criterion, which is aimed to a progressive regularization of the 3D mesh. In a second step, the position of the new vertices is corrected, by placing them on the original un-simplified mesh, therefore decreasing the topological error besides the color one. This approach allows achieving the desired visual quality, guaranteeing a maximum error in both color and geometrical spaces (separately).

2. Related work and methodological background The algorithm belongs to the mesh-simplification class. Although, many simplifying techniques have been developed aimed to preserve a geometrical high quality appearance with a reduced number of polygons [1], they come short in maintaining a high pictorial quality (cf. Fig. 1c, 5d). For example, in regions where the surface is smoothly varying, geometrical simplification proceeds in a radical polygon reduction. If in these regions, large color gradients or high frequency texture are present, a blurred image results: texture variations do not imply topological variations and vice versa. Quite surprisingly, only recently, color attributes, which give to the models the highest realism [9,10] have been taken into account by a few algorithms. Hoppe [11] was probably the first to stress the importance of considering color properties in the

simplification process. In his progressive meshes, an energy value is associated to the difference between the true color and the expected one in each vertex of the mesh. This energy term is added to other energy terms associated to geometry errors and the simplification process eliminates more vertices in those regions where the global energy is smaller. A non-linear optimization problem results, which is computationally expensive to solve. Garland and Heckbert [2] proposed a faster approach based on quadratic metric applied locally to the position and color of the vertices. The simplification driven by this metric is achieved by solving many local (small) quadratic problems. Their approach has been recently refined by Hoppe [12]. These algorithms produce a 3D model which is a compromise between geometry and color fidelity. However, these algorithms do not guarantee an adequate sampling of the two attributes [13] as they do not provide a direct control on what has been termed ‘‘visual degradation’’ [9,10]. As color and texture do not have topological properties, an open problem exists with these approaches: which is the adequate relative weight of color and geometry in the error function? Or equivalently: how to compare a color error with a geometrical error? Few algorithms based on multi-resolution analysis (MRA) of meshes with color attributes have also been presented. Eck et al. [14] presented an approach to simplify arbitrary meshes and Certain et al. [15] extended it to colored meshes analyzing separately color and geometrical attributes. However, the MRA structure constraints the simplification process, and algorithms, which work on 3D meshes like the one presented here, are more flexible [13]. The models considered here are represented as triangular meshes: MðV; TÞ, which are described as a collection of NV 3D points (the vertices) VfVi g; Vi ðXi ; Yi ; Zi Þ, connected in triples to form a set of NT triangular faces: Tftj g; tj fi; j; hg; where i, j, h are the indexes of the vertices ðVi ; Vj ; Vh 2VÞ. Attributes different from geometry can be associated to the mesh as scalar fields: diffuse color, normals, texture coordinates, bumps, displacement, . . . . Each of these has its own representation. Here we will focus our attention on color. Its standard representation is offered by the RGB space where it is represented as triplets: C ¼ ½r; g; b T with 04r; g; b41. As a result, a colored point of the mesh is completely identified by a sixtuple of coordinates: P2R6 , P ¼ ½X; Y; Z; r; g; b T . Texture is represented as a color bitmap, Bðx; yjCðx; yÞÞ, plus a mapping function, pðx; yÞ ¼ jðPÞ which operates a parameterization of M over the texture bitmap.

3. Overview of the algorithm The algorithm operates in four sequential stages: geometry simplification, color application, mesh refine-

P. Rigiroli et al. / Computers & Graphics 25 (2001) 449–461

ment and vertex correction (Fig. 1). The starting point is a triangular mesh sufficiently dense, M0 ðV0 ; T0 Þ (Fig. 1a). This is simplified using one of the host of topology/geometry-based simplification algorithms [1]

451

to produce a light mesh Mf ðVf ; Tf Þ (Fig. 1b–d), which preserves the topology. At this stage, color comes into play: a color value, C, is associated to each vertex of Mf to produce, through Gouraud shading, a colored model:

Fig. 1. The 3D mesh, M0 , obtained sampling at 1 mm step, the continuos reconstruction of Stefano’s face, through a HRBF network which is constituted of 20,842 vertices (a). The 3D mesh obtained simplifying M0 according to pure geometrical criteria (fuzzy vertex clustering [24]): 2515 vertices are left (b). Gouraud shading of 3D mesh in b (c). The 3D mesh, Mf , obtained simplifying M0 through [24], down to 1014 vertices, Mf (d). The mesh obtained re-tiling d, according to a maximum color error of 0.12 (2515 vertices) (e); and its Gouraud shaded version (f ). Notice the regularization of the mesh in the colored area of (d). The azure triangle (scalene) is split according to a binary schema while the yellow one (equilateral) to a quaternary one (f ).

452

P. Rigiroli et al. / Computers & Graphics 25 (2001) 449–461

Fig. 2. The compressed 3D mesh of Lorena (a); the bitmap obtained from a 2D frontal picture (projective texture) (b); and the 3D mesh projected over the bitmap (c).

Mf ðVf ; Tf ; CðVf ÞÞ (Fig. 1c). The color is derived from a bitmap on which Mf is projected (Fig. 2). In the third stage, Mf is refined adding a set of new vertices, Vnew , along with their associated triangles, Tnew ; such that a maximum color error is guaranteed inside each triangle (Fig. 1e, f). The last stage takes care of correcting the position of fVnew g, repositioning them on the manifold identified by M0 .

4. Processing geometry and color attributes 4.1. Texture processing The mapping function p ¼ jðPÞ, which parameterizes the 3D mesh over the texture, is required to apply a given texture bitmap over the mesh. A large body of research in Computer Graphics has been devoted for finding the optimal parameterization, which would preserve a bitmap appearance of high quality [16]. However, a texture bitmap, sampled through video/ photocameras from the same object described by the mesh, has a precise location and orientation in the 3D space, which is that of the target of the camera. The transformation between the texels of the 3D model and their 2D image sampled by the camera reduces to a perspective transformation (perspective texture). In this case, jð Þ is described, in homogenous notation, simply by a matrix multiplication: p ¼ AP [17]. Few attempts to automatically determine the parameters in A have been proposed [16,17]; some good results have been achieved when structured scenes have been considered (e.g. [18]).

However, the task remains primarily manual when unstructured objects like human faces are considered. It is achieved here by an interactive graphical tool which implicitly determines the parameters in A. The projective transform allows to virtually insert the mesh in the same position and orientation in 3D space from which it was surveyed by the camera that produced the bitmap (Fig. 2c). When the mesh is full 3D, a pixel can be projected over more than one triangle of the 3D mesh. In this case it is attributed (correctly) to the lower (closer) face. A color value can be applied to each point P of the simplified mesh Mf , by sampling the color in the point p ¼ AP of the texture bitmap. Repeating this operation for all the vertices fVf g, a colored model, Mf ðVf ; Tf ; CðVf ÞÞ (cf. Fig. 1d), is obtained. We remark here that the matrix A is valid not only for Mf , but also for M0 , and for any other mesh with a different LOD derived from M0 .

4.2. Defining an error metric To evaluate the quality of the simplified mesh, Mf ðVf ; Tf ; CðVf ÞÞ, an error measure, which takes color into account, should be defined. A decoupled representation, similar to that proposed in [13], is adopted here. It allows evaluating the errors in the two spaces separately. Particular care has been used to select a proper metric to measure the color distance of two points. Texture bitmaps usually come in RGB format, as the additive color coding is suitable to computer graphics display devices [19]. This color space, however, has the drawback of being non perceptive-isometric, meaning that

P. Rigiroli et al. / Computers & Graphics 25 (2001) 449–461

the Euclidean distance between two different colors, in RGB three-dimensional space, does not reflect the visual difference perceived by the Standard Colorimetric Observer. For this reason, the CIE-L * u * v * color space, which is perceptual uniform, has been adopted [20]. The transformation between RGB and CIE-L * u * v * spaces, gð Þ, is biunivocal and is reported in Appendix B. For sake of comparison, its values are normalized to one. It results: C0 ðL * ; u * ; v * ÞT ¼ gðCðr; g; bÞT Þ, with 04L * ; u * ; v * 41.

5. Texture-driven tessellation refinement The key idea is to analyze each face of the simplified colored 3D mesh, Mf ðVf ; Tf ; CðVf ÞÞ, and to subdivide it recursively until the color error goes under threshold in each of its points (cf. Fig. 3). To the scope, the mesh is

453

projected over the bitmap (cf. Fig. 2c and Section 4.1) and the analysis of the color error is carried out on the bitmap plane. 5.1. Evaluation of color error inside each face First, for each face t, all the pixels, which fall (also partially) inside t, are determined (Fig. 3a). These pixels constitute a piece-wise color field, which changes its value at the boundaries between two adjacent pixels. We will indicate this color field as CT ðpðx; yÞÞ. It can be added or, even better, blended according to predefined parameters, with that given by other textures or any other shader or field. For example, the field generated by the diffuse color, defined as per-vertex attribute on the original mesh M0 , can be considered as well. CT ðpðx; yÞÞ is compared with the field generated through linear interpolation of the color of the face vertices (Gouraud

Fig. 3. A texture bitmap is applied onto a 2D triangular face (a). The color field obtained by interpolating the color of the triangle vertices is clearly not adequate (b). A much better result (e) is obtained after re-tiling (d). The color errors are reported, respectively, in (c) and (d).The maximum color error between (a) and (b) is 0.3343, between (d) and (e) is 0.1296.

454

P. Rigiroli et al. / Computers & Graphics 25 (2001) 449–461

shading), CG ðpðx; yÞÞ. This color is that of the pixel inside which the projection of Vit falls: Cit ðvit ðx; yÞÞ, with vit ðx; yÞ ¼ AVit , where t indicates the generic tth face. Starting from vit ðx; yjCit Þ, the color in each point, pt , internal to the tth face of the mesh, CG ðpðx; yÞÞ ¼ ½rp ; gp ; bp , can be computed as rp ¼

3 X

wi ri ;

gp ¼

3 X

i¼1

wi gi ;

i¼1

bp ¼

3 X

wi bi ;

ð1Þ

i¼1

where fwi jwi ¼ ½0 1 g are the intrinsic coordinates of p on the tth projected face, and they are a function of the vertices coordinates on the bitmap plane (bilinear interpolation, [21]): w1 ¼ 1  ½ ðx  x1 Þð y3  y2 Þ  ð y  y1 Þ ðx3  x2 Þ =ð2SÞ;

ð2aÞ

w2 ¼ 1  ½ ðx  x2 Þð y1  y3 Þ  ð y  y2 Þ ðx1  x3 Þ =ð2SÞ;

ð2bÞ

w3 ¼ 1  ½ ðx  x3 Þð y2  y1 Þ  ð y  y3 Þ ðx2  x1 Þ =ð2SÞ;

ð2cÞ

where S is the area of the triangle. It is now possible to evaluate the color difference between the two fields: CT ðpÞ and CG ðpÞ. This represents the color error: DCðpÞ ¼ jjgCT ðpÞ  gCG ðpÞÞjj, in CIE-L * u * v * space, and it is a field itself. As demonstrated in Appendix A, thanks to the nature of CT(p) and CG(p), the maximum error can be found in one of the four corners of the pixels. Therefore, sampling the color error field in these positions is sufficient to find the maximum error inside each face. This is an advantage with respect to those methods, which evaluate the error scanning the manifold [3,5,13], and require manual tuning of the sampling step. Our scanning procedure starts from the upper leftmost pixel and explores a face in rows. When the error is over-threshold, the triangle is too large to have an adequate texture representation through linear interpolation, and it is split.

5.2. Splitting a triangle In splitting a triangle, its shape should be considered to avoid degenerate situations such as one-pixel micropolygons or stripe-shaped faces [21]. An innovative strategy is proposed here. Triangles are split according to a flexible criterion, which is aimed to create a progressive regularization of its shape (Table 1). Triangles which have the longest side much larger than the smallest one (scalene) are ‘‘regularized’’ splitting them in two by a segment connecting the middle point of the longest side to the opposite vertex (Table 1, class # 1). Triangles, which are approximately equilateral, are re-tiled according to a quaternary schema [21], which preserves the original equilateral shape (Table 1, class # 2). The evaluation of the triangles type is carried out on the manifold, to avoid stretching introduced by the perspective projection. The effect of the re-tiling procedure is exemplified in Fig. 3. Here a 2D triangular face with a bitmap superimposed is analyzed (Fig. 3a). The reproduction of the color field through Gouraud shading is clearly insufficient as it is also shown by the color error reported in Fig. 3c. A much better result is obtained after retriangulation as it can be seen in Figs. 3d–f. 5.3. Vertices correction Once a new vertex, vnew , has been added to the projected mesh, its position on, Vnew , on the 3D mesh, M0 has to be found. The inverse of the transformation vnew ¼ AVnew , is not univocal as it identifies all the points on a straight line, s, through vnew and the perspective center of the camera [17], O (Fig. 4). As a first approximation the position of Vnew can be placed at the intersection of s with Mf . However, when the unsimplified mesh, M0 , is available, the 3D position of the new vertex can be computed, more correctly, as the intersection of s with M0 : V0new . Notice that Mf and M0 share the same projection matrix, A. With this procedure

Table 1 Geometry-based triangle classification and their splitting Class

Condition

Classification

Edge selection

1

One edge >150% of the shortest one

Scalene

The longest edge is split in its mid-point

2

Else

Quasi-equilateral

Every edge is split in its mid-point

Example

Split performed

P. Rigiroli et al. / Computers & Graphics 25 (2001) 449–461

Fig. 4. Determination of the 3D position of a new vertex, V0new . Its color is correctly projected (mapped) over M0 .

the re-tiled model also enjoys a better geometrical description. Being the texture projective, there is no need for correcting the position of the mapped color as, for example, in [13].

6. Results The method described here has been extensively applied to get 3D models of human faces of high pictorial quality from the data acquired through the Autoscan digitizer [19,22]. The geometrical model is constructed starting from the 3D position of an ensemble of points obtained digitizing a laser spot projected over the subject’s face by a hand-held laser pointer. The spots are recognized at sub-pixel accuracy through real-time hardware correlation and are fed to a neural network model (HRBF network [19,23]), which reconstructs the surface, SðX; Y; ZÞ, as a linear combination of clusters of Gaussians at different scales. S is then regularly sampled to produce a dense triangular mesh, M0 (cf. Fig. 1a). However, any other approach which gives a tessellation dense enough, cleaning the measurement error is suitable as well. On M0 , a simplification algorithm, based on geometrical criteria, is run, to produce a lighter mesh, Mf . Here a fuzzy vertex clustering algorithm [24], which minimizes the geometrical distortion error, has been adopted (cf. Fig. 1b–d). The texture, Bðx; yjCðx; yÞÞ, is obtained from a picture of the subject, taken frontally during the 3D-digitisation process (cf. Fig. 2b). For our scope, the use of just one frontal picture can be considered sufficient. B; M0 and Mf constitute the input to the refinement stage. As it is clear from Figs. 1c, 5a, although Mf is clearly able to represent the topology

455

of the model (Fig. 1b), the color field generated through Gouraud shading gives a poor appearance (Figs. 1d, 5d). A large color error is produced, which can be as large as 0.7 inside some faces (Fig. 5g). A much better result is obtained after re-tiling the mesh. This is exemplified in Figs. 1f and 5. In Figs. 5e, f, a maximum color error respectively of 0.12 and 0.06 is achieved over the whole mesh. The improvement is even more evident in the close-up of the eye displayed in Figs. 5h, i. The mesh in Fig. 5b is constituted of NT ¼ 4307 faces and NV ¼ 2843 vertices, about twice those constituting Mf , but still much less than those constituting M0 . A further increase in the number of faces would not produce an appreciable improvement in the quality of the reconstruction. Interestingly, about 8% faces less are required to achieve the threshold error in CIE-L * u * v * space with respect to RGB space. The number of triangles as a function of color error is reported in Tables 2 and 3. This shows that, with a little increment in the number of faces (up to 4307), a large decrease in the color error is obtained. Little more can be gained incrementing the number of faces over this level. Notice how the new triangles are inserted in the regions where the chromatic gradient is higher and variable in direction. The re-tiling procedure is fast: only 2.4 and 3.2 s are required in order to obtain respectively the medium and high quality 3D meshes on a Pentium II, 350 MHz. The same results are obtained with data from the literature. For example the re-tiled models of the globe are equivalent to those obtained by simplification in [2, Fig. 9] (see Fig. 6). The main difference is that, here, the new triangles cluster around the color discontinuities and larger patches are used to cover smooth color transitions, which allows a better definition of critical regions such as the Arabic peninsula and the Mediterranean sea. Similar results are obtained for the mandrill model [11, Fig. 8]. Here the main difference is in the triangular shape. In [11] many elongated triangles are used which adapt to the color patches of the model, while in Fig. 7, due to regularization, the triangles tend to be equilateral. Although the number of vertices is larger (683 versus 400), the number remains limited and the computational time drops from the 19 min (on SGI Indigo2, 150 MHz) declared in [11] to less than 1 s on a Pentium II, 350 MHz. Moreover, convergence problems, which often plague global optimization, are here avoided. 6.1. Quantitative evaluation A quantitative evaluation of memory occupancy is now derived. Let us call NV and NT the number of vertices and triangle, respectively, of a given 3D mesh. Its constituents are: the vertices coordinates fVg, the

456

P. Rigiroli et al. / Computers & Graphics 25 (2001) 449–461

Fig. 5. The starting 3D mesh, Mf , (2505 triangles, 1264 vertices), and the 3D mesh obtained after refinement: with a color error of 0.12 (b) and 0.06 (c). The three 3D meshes rendered with Gouraud Shading (d–f). A close-up of an eye is reported for the three 3D meshes (g–i).

457

P. Rigiroli et al. / Computers & Graphics 25 (2001) 449–461 Table 2 Number of faces, vertices, computational time, as a function of the color error

Stefano Lorena Globe Mandrill

Starting mesh

Medium quality

High quality

Faces

Vertices

Error

Faces

Vertices

Storage

Time

Error

Faces

Vertices

Storage

Time

2020 2505 840 4

1032 1264 450 5

0.12 0.12 0.55 0.25

4138 4307 3000 1081

2515 2483 1836 636

46 47 34 12

2524 ms 2433 ms 2634 ms 1002 ms

0.06 0.06 0.12 0.12

10,066 10,513 30,012 3896

6491 6758 19,160 2259

116 Kb 120 Kb 344 Kb 43 Kb

3445 ms 3225 ms 4026 ms 1512 ms

Kb Kb Kb Kb

Table 3 Number of faces as a function of the color error for Lorena. The starting mesh is constituted of 2505 faces. The data in light grey refer to the meshes reported in Fig. 5(a). The memory occupancy of the same model compared with texture of different quality mapped over a model with the same number of faces (b)

Texture mapping ( jpeg 100%) Texture mapping ( jpeg 80%) Texture mapping ( jpeg 65%) Per vertex color (0.12 error) Texture mapping ( jpeg 60%) Texture mapping ( jpeg 40%)

vertices index for each face fTg, the texture map jð Þ, the texture bitmap, B, and the vertices colors fCðVÞg. Let us assume that each coordinate of a vertex, Vi, and the vertices index of a face, TVi , can be stored in two bytes. The texture map requires also two bytes for each 3D vertex. The color of each vertex can be coded by three bytes (RGB values). Considering that the number of faces is roughly twice the number of vertices, the memory occupancy in bytes for the per-vertex color coded model is Memory Per vertex ¼ NV2 6 þ ðNV2 2Þ6 þ NV2 3 ¼ NV2 21;

ð3aÞ

while for a textured model is Memory Texture ¼ NV1 6 þ ðNV1 2Þ6 þ NV1 2 þ MðBÞ ¼ NV1 20 þ MðBÞ;

ð3bÞ

Texture (bytes) (398  617)

Model (bytes)

Total (bytes)

167,404

25,142

192,546

32,769

25,142

57,911

23,574

25,142

48,716

//

48,189

48,189

21,669

25,142

46,811

16,513

25,142

41,655

where MðBÞ is the memory occupied by the texture. NV2 > NV1 results as re-tiling is required by per-vertex coding to achieve the same pictorial quality of texture mapping. In this balance there are two free variables: the number of vertices after re-tiling ðNV2 Þ and the memory occupancy of the texture. For example in the model of Fig. 5, the re-tiled model becomes convenient when JPEG cannot be compressed over 65%. But the main advantage is when no hardware texture mapping is available at the client side. On a SGI Indigo2 Solid Impact, 150 MHz, CosmoPlayer allows a fluid manipulation of per-vertex colored meshes constituted of up to 20,000 polygons, while textured meshes of as small as 1000 faces cannot be manipulated in real-time. Although better SW schemas for representing textured polygons can be developed [6], the difference in rendering time remains extremely large.

458

P. Rigiroli et al. / Computers & Graphics 25 (2001) 449–461

Fig. 6. Globe. The starting mesh, Mf , (840 faces, 450 verrtices) (a, d). The mesh obtained after refinement with a color error of 0.55 (3000 faces, 1836 vertices) (b, e), and after refinement with a color error of 0.12 (30,012 faces, 19,160 vertices) (c, f).

Fig. 7. Mandrill. The starting mesh constituted of 4 faces only projected on the bitmap (a). The mesh obtained after refinement with a color error of 0.25 (1081 faces, 636 vertices) (b), and after refinement with a color error of 0.12 (3896 faces, 2259 vertices) (c).

P. Rigiroli et al. / Computers & Graphics 25 (2001) 449–461

7. Discussion and conclusion The procedure described here allows the maximum flexibility. The user can interactively choose the proper balance between geometry complexity and pictorial appearance by examining the two aspects separately. As the computation is carried out locally, this procedure is particularly suitable to selectively refine the mesh in critical regions at need, for example when it has to be seen from particular viewing angles. By lowering the color threshold, different sets of vertices and faces are added to the mesh. These can be stored as ‘‘refinement data’’, fRj g, which are particularly suitable in incremental transmission. First, the coarse mesh, Mf , can be transmitted with its per-vertex color values (Fig. 1b and 5d); afterwards the ‘‘refinement data’’ can be sent and combined with Mf to obtain an effective progressive mesh which improves, step by step, the visual appearance (cf. Fig. 5). As geometrical simplification has preserved the spatial details, refinement data improve only the pictorial appearance of the model. A particular set of LOD of models, or hierarchy, is therefore created where the details added refer only to the color attribute. A reduced number of triangles could be obtained by placing the new vertices in optimal position; for instance such that the error on the resulting triangles is minimal. However, this would lead to non-linear minimization, which highly increases the computational load. Although the re-tiling algorithm presented here is simple, it produces excellent results. It is reminiscent of methods for iterative refinement of terrain elevation or radiosity fields [21,25,26]. In terrain elevation, a 3D face is subdivided when the difference between the height of one of the sampled points, which fall inside it, and that obtained by linearly interpolating its vertices heights is over-threshold. This approach does not guarantee that the true maximum of the error is found; if this falls inbetween the sampled points it goes un-noticed. On the contrary, the continuity of color information (the bitmap) along with the method that we have developed to analyze the bitmap, assures that the true maximum error inside the face is found (cf. Appendix A). In radiosity solution, a hierarchical subdivision schema of a 2D image, was introduced to create 2D patches whose form factor (which represents the percentage of light arriving on the face) is under-threshold. The method yields a faster radiosity computation. However, its estimate of the form factor of each face is only approximate; as in the computation, each 2D patch of the mesh is substituted by a circular patch with cross sectional area equal to the patch area or by a circumscribing sphere [25]. Besides quantifying exactly the maximum error for each face, the algorithm proposed here allows powerful mesh regularization. It is based on a flexible splitting criterion, which adapts to the shape of the face to be split.

459

This allows avoiding the effect of fixed (e.g. ternary) subdivision which may recursively subdivide a triangle until it occupies a stripe one-pixel wide. Although skinny triangles can be very useful in particular cases, (e.g. to represent sharp transitions or stripe-shaped color constant, regions); it is necessary that their vertices are optimally positioned to be effective. If they are not, the subdivsion schema does not allow correcting the color error inside these triangles. The flexible subdivision schema, instead, allows creating, in the limit, triangles even smaller than one pixel, at a pixel border. For these reasons skinny triangles have been avoided. The progressive regularization of the mesh, obtained through a flexible subdivision schema, may suggest using this method also in terrain elevation or radiosity fields, when applied to irregular meshes. Full 3D colored models can also be re-tiled with this method. In this case, multiple bitmaps, taken from different vantage points, are required. Each bitmap will come with its own projection matrix (A in Section 4.1). Each face of the 3D mesh, is mapped over all the bitmaps, which are visible from that face and the color error analyzed as in Section 5.1. The strategy, to be used to stitch together the different bitmaps, goes beyond the scope of this paper. Carrying out the refinement of a 3D mesh, in the 2D space of the bitmap, makes the algorithm computationally efficient. In fact, this requires only that each vertex, V, of the 3D mesh be projected onto the bitmap plane, which is obtained by linearly multiplying the coordinates of V by the matrix A (Section 4.1). The re-tiling procedure could have been carried out onto the mesh, in the 3D space. This is computationally more expensive as it requires that the corners of each pixel be back-projected in 3D space and the intersections of the back-projections with the 3D mesh computed. The usage of per-vertex color to represent texture guarantees soft color transitions on the mesh in the close-ups, where the texture would give a pixellated image. It is therefore particularly suitable for scenes in which soft shadows are present [7] like those where, many unstructured objects, are present (e.g. human faces). As a result of applying this method, the graphical board is freed from texture mapping operations, saving computing power to map other scalar properties like bumps, deformation fields (displacement maps), normals,. . . [27,28]. With the new graphics architecture which are becoming available [28], one can envisage a master process which partitions the graphic attributes (diffuse color, normals, texture coordinates, bumps, displacement. . ..) among a few boards which operate field mapping and few processors which operate linear per-vertex interpolation, to achieve the maximum throughput of the model with all its attributes.

460

P. Rigiroli et al. / Computers & Graphics 25 (2001) 449–461

8. For further reading The following references are also of interest to the reader: [29]; [30]; [31].

Acknowledgements We wish to thank Federico Lo Monaco and Gianluca Costa which contributed to a preliminary phase of the present work.

obtained by subtracting to pG ðx; kÞ the color value generated by the texture: pT ðx; kÞ, which is piece-wise constant. It follows that the maximum error is achieved at the intersection of the horizontal line with the pixel borders. The same reasoning applies to vertical scanning of the bitmap. It follows that the maximum error is found in one of the four corners of the pixel. With similar reasoning it can be shown that, when a pixel is located across a triangular face side, the maximum error is obtained: at one of the pixel corners contained inside the face, or at the two extremes of the intersection of the side with the pixel area.

Appendix A Appendix B We show here that, for each face t, the maximum color error is found in correspondence of the corners of the bitmap pixels. Let us reorganise Eqs. 2(a)–(c) in w1 ¼ 1 þ a1 x þ b1 y þ c1 ;

ðA:1aÞ

w2 ¼ 1 þ a2 x þ b2 y þ c2 ;

ðA:1bÞ

w3 ¼ 1 þ a3 x þ b3 y þ c3 ;

ðA:1cÞ

where for the first equation: a1 ¼ ðy3  y2 Þ=ð2SÞ; b1 ¼ ðx3  x2 Þ=ð2SÞ; c1 ¼ ½x1 ðy3  y2 Þ þ y1 ðx3  x2 Þ =ð2SÞ: As we choose the image reference system with the axes parallel to the rows and columns of the image, the scanning will be at constant height ðy ¼ kÞ. Substituting y ¼ k into Eq. (A.1), the following set of parameters is obtained for a point, pðx; kÞ, which lies on a horizontal line inside the face t: w1 ¼ 1 þ a1 x þ c01 ;

c01 ¼ c1 þ b1 k;

ðA:2aÞ

w2 ¼ 1 þ a2 x þ c02 ;

c02 ¼ c2 þ b2 k;

ðA:2bÞ

c03

ðA:2cÞ

w3 ¼ 1 þ a3 x þ

c03 ;

¼ c3 þ b3 k

To standardize the evaluation of color differences, in 1976 the Commission International de l’Eclairage (CIE), proposed two methods (CIE-L * u * v * and CIE-L * a * b * ) to evaluate them. They are both based on a two step transformation [10]. In the first step, the RGB coordinates are linearly transformed into three primary colors, denoted X; Y and Z as: 2 3 2 32 3 R X 0:607 0:174 0:200 6 7 6 76 7 ðB:1Þ 4 Y 5 ¼ 4 0:299 0:587 0:114 54 G 5: B Z 0:000 0:066 1:116 CIE-L * u * v * coordinates are then obtained through the following non-linear transformation:  1=3 Y L * ¼ 116 16; ðB:2aÞ Yn u * ¼ 13 L * ðu0  u0n Þ;

ðB:2bÞ

v * ¼ 13 L * ðv0  v0n Þ;

ðB:2cÞ

0

0

where u ; v ;

u0n

and

v0n

are computed as follows:

and the color of a point pG ðx; kÞ, obtained through Gouraud shading, will be

u0 ¼

4X ; X þ 15Y þ 3Z

rp ¼ r1 ð1 þ a1 x þ c01 Þ þ r2 ð1 þ a2 x þ c02 Þ þ r3 ð1 þ a3 x þ c03 Þ;

ðA:3aÞ

u0n ¼

4Xn ; Xn þ 15Yn þ 3Zn

ðA:3bÞ

where Xn ; Yn ; Zn are the coordinates of a white stimulus. With this transformation the coordinate L * represents the luminance of the stimulus.

gp ¼ g1 ð1 þ a1 x þ c01 Þ þ g2 ð1 þ a2 x þ c02 Þ þ g3 ð1 þ a3 x þ c03 Þ; bp ¼ b1 ð1 þ a1 x þ c01 Þ þ b2 ð1 þ a2 x þ c02 Þ þ b3 ð1 þ a3 x þ c03 Þ;

v0 ¼

9Y ; X þ 15Y þ 3Z

v0n ¼

9Y ; Xn þ 15Yn þ 3Zn

ðA:3cÞ

which are a set of linear equations in x. For the linear Gouraud interpolation, color coordinates decrease or increase along a horizontal line, and, in particular, inside the area occupied by each pixel. The color error is

References [1] Heckbert PS, Garland M. Survey of polygonal surface simplification algorithms. Technical Report, Carnegie Mellon University, 1997.

P. Rigiroli et al. / Computers & Graphics 25 (2001) 449–461 [2] Garland M, Heckbert PS. Simplifying surfaces with color and texture using quadric error metrics. Proceedings of the IEEE Visualization, 1998. [3] Maruya M. Generating texture map from object-surface texture data. Proceedings of Eurographics’95. Computer Graphics Forum 1995;14(3):397–405. [4] Soucy M, Godin G, Rioux M. A texture-mapping approach for compression of colored 3d triangulation. The Visual Computer 1996;12:503–14. [5] Cignoni P, Montani C, Rocchini C, Scopigno R. A general method for preserving attribute values on simplified meshes. Proceedings of the IEEE Visualisation 1998; 59–66. [6] Mann Y, Cohen-Or D. Selective pixel transmission for navigating in remote virtual environments. Proceeding of Eurographics’96. Computer Graphics Forum 1997;16(3):201–6. [7] Levoy M. Polygon-assisted JPEG and MPEG compression of synthetic images. Proceedings of Siggraph’96. ACM Computer Graphics 1996;30:11–20. [8] Turk G. Re-tiling polygonal surfaces. Proceedings of Siggraph’92ACM. Computer Graphics 1992;26:55–64. [9] Reddy M. Scrooge: perceptually-driven polygon reduction. Computer Graphics Forum 1996;15(4):191–203. [10] MacDonald L. Using color effectively in computer graphics. IEEE Computer Graphics and Applications 1999;19(4):20–35. [11] Hoppe H. Progressive meshes. Proceedings of Siggraph’96. ACM Computer Graphics 1996;30:99–108. [12] Hoppe H. New quadric metric for simplifying meshes with appearance attributes. Proceedings of IEEE Visualization, 1999. [13] Cohen J, Olano M, Manocha D. Appearance-preserving simplification. Proceedings of Siggraph’98. ACM Computer Graphics 1998;32:115–22. [14] Eck, DeRose T, Duchamp T, Hoppe H, Lounsbery M, Stuetzle W. Multi-resolution analysis of arbitrary meshes. Proceedings of Siggraph95. ACM Computer Graphics 1995;29:173–182. [15] Certain, Popovic J, DeRose T, Duchamp D, Salesin D, Stuetzle W. Interactive multiresolution surface viewing. Proceedings of Siggraph’96. ACM Computer Graphics 1996;30:91–8. [16] Herman M, Kanade T. Incremental reconstruction of 3D scenes from multiple, complex images. Artificial Intelligence 1986;30:289–341.

461

[17] Weinhaus F, Devarajan V. Texture mapping 3D models of real-world scenes. ACM Computing Surveys 1997;29(4):325–65. [18] Debevec PE, Taylor CJ, Malik J. Modeling and rendering architecture from photographs: a hybrid geometry and image-based approach. Proceedings of Siggraph’96. ACM Computer Graphics 1996;30:11–20. [19] Borghese NA, Ferrari S. A portable modular system for automatic acquisition of 3D objects. IEEE Transactions on Instrumentation and Measurement 2000;49(5):1128–36. [20] Robertson PK. Perceptual color spaces. IEEE Computer Graphics and Applications 1998;18(5):50–64. [21] De Floriani L, A pyramidal data structure for trianglebased surface description. IEEE Computer Graphics and Applications 1989;32:67–78. [22] Borghese NA, Ferrigno G, Baroni G, Ferrari S, Savare R, Pedotti A. AUTOSCAN: a flexible and portable scanner of 3D surfaces. IEEE Computer Graphics and Applications 1998;18(3):38–41. [23] Borghese NA, Ferrari S. Scanning and reconstruction of human body parts. Eurographics’98, Short Presentations 1998:3.5.1–.2. [24] Ferrari S, Ferrigno G, Borghese NA. Vertex clustering and data filtering with vector quantisation, Proceedings of the International Joint Conference on Neural Networks, Como, September 2000: 512–9. [25] Hanrahan P, Salzman D, Aupperle L. A rapid hierarchical radiosity algorithm. Proceedings of Siggraph’94. ACM Computer Graphics 1991;25:55–67. [26] Cohen M, Greenberg DP, Immel DS, Brock PJ. An efficient radiosity approach for realistic image synthesis. IEEE Computer Graphics and Applications 1986;6(3):41– 50. [27] Peercy M, Airey J, Cabral B. Efficient bump mapping hardware. Proceedings of Siggraph’97. ACM Computer Graphics 1997;31:303–6. [28] Olano M, Lastra A. A shading language on graphics hardware: the pixel flow shading system. Proceedings of Siggraph’98. ACM Computer Graphics 1998;32:159–168. [29] Ronfard G, Rossignac J. Full-range approximation of triangulated polyhedra. Proceedings Eurographics’96. Computer Graphics Forum 1996;15(3):67–76. [30] Gershbein R, Schroder P, Hanrahan P. Textures and radiosity: controlling emission and reflection with texture maps. Proceedings of Siggraph’94. ACM Computer Graphics 1994; 28:171–178.