Multiple Illuminated Paper Textures for Drawing Strokes Kyoko Murakami*

Reiji Tsuruno†

Etsuo Genda‡

Graduate School of Kyushu Institute of Design

Kyushu University

Kyushu University

ABSTRACT

to generate the drawing tools (a stroke, and an eraser).

This paper presents an algorithm for generating realistic drawing strokes that can take on the appearance of pastels, charcoals, or crayons. The similarity between pigments deposited on a paper surface by a pastel stroke and on the texture of an illuminated paper surface was studied. Twelve paper textures were prepared and were illuminated from various directions in increments of 30 degrees for a stroke drawn in an arbitrary direction. After these textures were processed as if they could be used as a height field, pigments deposited on paper were calculated using the height field and pen pressure. The height field is determined by linear interpolation of multiple paper textures for drawing strokes in any direction. By using the paper texture instead of simulating the deposition of pigments, a realistic stroke with arbitrary parameters can be rendered in real time.

2. Related Works

CR Categories: I.3.4. [Computer Graphics]: Graphic Utilities

- Paint systems; I.3.3. [Computer Graphics]: Picture/Image

Generation - Display Algorithms Key words: drawing, stroke, illuminated paper, NPR 1. Introduction

The modeling of traditional drawing/painting materials and techniques is an ongoing and open-ended topic in NPR graphics. A variety of characteristics can be expressed in the final rendered images through elaborate simulations. Because most real pictures are constructed through a series of strokes, many NPR studies have focused on modeling the strokes themselves. For each individual work, the stroke that is inherent in each painting/drawing tool is generated specifically for each system and is affected by brush types, support media, and the interaction of these materials. When using drawing tools that have soft tips, like pencils, pigments are scraped from the tip and then deposited onto the surface of the support media. In this case, paper is often used as the support medium. When the tip of the drawing tool is softer than that of pencils, including pastels, charcoals, chalks, or crayons, the paper grains are not planed by the stroke and the paper texture becomes visible. Realistic art tools cannot be accurately represented without simulating the interaction between such materials. However, elaborate simulation requires considerable computation time for a drawing tool that should work in real time. In this work, realistic drawing strokes are generated in real time using twelve illuminated paper surface samples. Instead of calculating the interaction between each pigment and paper, the gradation of the illuminated paper surfaces is used as the height field, which requires less time. Kalnins's method [1] is employed *e-mail: [email protected] †e-mail: [email protected] ‡e-mail: [email protected]

In several NPR studies on drawings, the results are highly affected by the support medium because of soft-firmed pigments, including pencils [2], colored pencils [3], charcoals [4], crayons [5] and pastels [6]. A support medium, such as a paper surface, is typically simulated using a height field as detailed by Curtis et al. [7]. The height field is used in both faithful and simple simulations of various art tools. Several authors have proposed a stroke-generation method for soft tips and pigments [1], [4], [6], and [8]. Bleser et al. [4] generated a stroke from a sample image of a real pastel stroke. Scanning and regeneration methods have also been used in some studies. Kalnins et al. [1] regenerated various crayon and pencil strokes by scanning several real strokes. Durand et al. [8] generated a "threshold matrix" from a scanned image and represented various pen pressures and lap over. These texture-based methods are beneficial for both real-time rendering and generating "realistic" strokes. The textures that appear on strokes are affected by various elements, such as pen pressure variation, pen pressure distribution on the cross-section, cross-section form variation by rubbing down or collapsing tips, and stroke direction. Several stroke samples cannot adequately represent the entire span of variation. Media-simulation methods of drawing tools that account for complicated pigment behavior can sufficiently represent the variation [2], [3], [5]. In some studies, all of the simulated media suitable for the system are generated. Takagi et al. [3] used a

Figure 1. Appearance of paper texture for real pastel strokes from different directions. Arrows on each sample indicate the stroke direction. pastel

drawing direction h

light source

paper surface

x h

x

Figure 2. Paper surface illuminated by a light source and pigments deposited by a stroke. The right array shows the original paper surface (top) and the height field altered by the light (bottom): h[0 -1]. x indicates a 2D plane.

control points (spline) maximum pressure points stroke center points

of drawn pastel strokes and paper texture that is illuminated by a light. When a pastel stroke is drawn on a paper surface, pigments scraped by irregularities in the paper are deposited not on top of the convex area but on an area near the top. This causes different appearance of paper texture along the stroke directions as shown in Figure 1. In contrast, when an irregular paper surface is illuminated from one direction, causing shade and a shadow to appear, the brightest part is not the highest part, but an area slightly lower than the highest part in the direction of the light. This phenomenon is seen in many strokes drawn by tools that have soft tips and pigments, as depicted in Figure 2. Therefore, the illuminated paper texture is used as the height field for simulating pigment deposition by a stroke drawn in one direction. When a stroke is drawn, in fact, the stroke uses a height field consisting of the converted paper texture illuminated by a light coming from the same direction as the stroke.

maximum pressure line segment stroke path

cross section stroke mask

pen pressure F

pen pressure F

3. Implementation cross section

cross section

Figure 3. Stroke mask generation. Pen pressures at the cross sections are decided by the interpolation of the maximum pressure points with quadratic (bottom left) or linear (bottom right).

volumetric approach for generating paper. They approximated colored pencil tips as spheres and used “offset distance accessibility (ODA)” [9] to model pigment deposition. Sousa and Buchanan’s paper model [2] uses real paper data classified according to its height. Tips of all pencil drawing tools are defined as polygons that have the pencil pressure at the center and the vertices of the polygon. Leads are scraped when they come into contact with the higher parts of the paper surface. The NPR by Rudolf et al. [5] defined the wax deposited by crayons on paper as the overlapping cross-sections of the crayon and the paper. Murakami and Tsuruno [6] deposited particles of pigment on a paper surface to generate pastel strokes. All of these methods simulate drawing tools that allow for arbitrary stroke styles. Unfortunately, these methods also include some drawbacks. Elaborate simulation requires considerable processing time, making it difficult to draw in real time. Furthermore, suitable interaction is needed between the paper surface and the drawing tools to effectively regenerate the strokes, and numerous parameters are carefully set based on actual data to produce “realistic” results. This paper focuses on the similarity between the appearance

The final image depends on the amount of pigment deposited in a pigment buffer. The pigment buffer is the same width and height as the resulting image, and the pigments are stored within it. A stroke is defined as a stroke mask. Inside the stroke mask, the number of pigments deposited in the pigment buffer is calculated by considering the interaction between the pen pressure and the paper texture. All of the tools are produced using basically the same method. 3.1. Stroke Mask There are various methods for generating strokes [10], [11], [12]. In this study, a two-dimensional stroke mask based on that of Kalnins et al. [1] is used, which is ideally defined as a train of cross-sectional surfaces. The “base path” is a Catmull-Rom spline consisting of control points that are denoted by a mouse or tablet. The stroke contour is then defined as the stroke mask from this base path using a triangle strip as the width. It is rare that a constant pressure is applied when a stroke is drawn. The convex part at the cross-sectional surface of the drawing tool receives more pressure than the other points, causing more pigment to be deposited on the paper surface. Furthermore, most drawing tools are fragile: convex parts are immediately flattened, causing new convex points to appear. This results in variation of gradation inside a stroke. In order to accurately represent this effect, a “maximum pressure line segment”

1 2

light source

camera

12

3

11

1

2

3

4

5

6

7

8

9

10

11

12

θ= 30° 4

45°

paper surface

10 紙面

5

camera

paper surface

6

9 8

7

Figure 4. Paper capturing set-up. The paper surface is positioned under a camera and illuminated from one direction. The paper surface is turned every 30 degrees and the texture is photographed. The right images are paper textures that were captured and converted for implementation (drawing paper). The numbers below the images correspond to the light source number for the image center.

consisting of maximum pressure points at each cross section along the base path is implemented, as shown in Figure 3. At the cross-section of a stroke, a linear or a quadratic curve is used to interpolate the maximum pressure point and both end points. Controlling the coefficient and the intercept of the functions allows for variation in the stroke. 3.2. Selecting Paper Texture After the stroke mask is set, the paper texture is applied. The paper texture is tiled on the pigment buffer with a wrap-over at the edge. Its size is arbitrary and depends on the resolution of the resulting images. As described above, the paper texture that appears on a stroke varies with the stroke direction. Therefore, the appropriate drawing direction vector must be selected to obtain the appropriate paper texture. Furthermore, the pressure distribution displaces pigments inside the stroke. This is approximated by considering the movement of the maximum pressure line segments along the stroke when selecting the paper texture. Twelve surface images were prepared. For each image, the paper surface is illuminated by a single light source with azimuth θ set in increments of 30 degrees from 0 to 360 degrees. In order to obtain the illuminated paper texture suitable to distinguish paper irregularities, the elevation from the paper to the light source is set at 45 degrees, as depicted in Figure 4. In this figure, the images are converted to 0-1 gray values so that the minimum value in the original image becomes zero and the maximum value becomes 1. This value is then used as the deposit efficiency of the pigments. These converted paper surfaces are referred to as paper texture T, and each texture T has a direction vector to light source Vt. When a stroke is drawn in one direction, its direction vector V s is used to select two paper textures T 1 and T 2, which are illuminated from directions similar to Vs: Gs = t1 (1-ratio) + t2 ratio with Gs: gray value for stroke, and ratio: (Vs -Vt1) / (Vt2- Vt1). The same process using pressure movement direction vector Vp is used for selecting textures for the maximum pressure line segment and the gray value Gp. The resulting gray value G is the average of Gs and Gp, which is used as the height field. The pigments that are deposited in the pigment buffer depend on pen pressure F and the height field. THRESHOLD = 1-F if (THRESHOLD>G) deposit pigments The displayed color is equal to the original color, but independent from the quantity of pigments. To prevent aliasing when the difference between gray value G and THRESHOLD is small (defined as less than 0.02), transparency is given to the pigments according to the difference: TRANS = 1 - (THRESHOLD + 0.02 - G) DEC where DEC is the attenuation value. The transparency value TRANS varies between 0 (transparent) and 1 (opaque).

deposit new pigments because the paper irregularities are filled with the pigments on the paper surface in a real drawing. To account for this effect, the new pigments are deposited in an inverse proportion as follows: deposit pigments = (Pm-Ps)/Pm Pn with Ps: the amount of pigments stored in the pigment buffer, Pm: maximum pigments able to store in the pigment buffer, and Pn: the amount of pigments that are scraped by drawing. The proportion of the amount of deposit pigments and Pn is used as proportion of color blending. Some works use the Kubelka-Munk model that approximates translucent pigments for color blending [5], [7]. Instead, this study uses simple subtractive color mixing through the use of an OpenGL function. 3.3.2. Width Variation The widths of real strokes vary with pen pressure. In most cases,

1 texture (1)

2 textures (1, 7)

4 textures (1, 4, 7, 10)

12 textures (all)

Figure 5. The effect of multiple paper textures. The numbers following the texture numbers correspond to paper numbers shown in Figure 4 (right).

(a)

(b)

(c)

3.3. Additional Effects 3.3.1. Overlap If pigments are already deposited on a paper, it is difficult to

(d) Figure 6. The effect of a pressure interpolation.

drawing paper

water color paper type 1 (canson)

water color paper type 2 (watson)

water color paper type 3 (cotton)

water color paper type 4 (mold maid)

Figure 7. Comparison between real pastel strokes and the model: drawing paper (top) and four types of watercolor paper. The strokes on the left are real, and those on the right are made with the model.

(a)

(b)

(c)

(d)

Figure 8. Overlapping strokes. Pink strokes are drawn on blue strokes: real strokes (a)(b) and the proposed method (c)(d).

Figure 9. Resulting images using the proposed drawing application.

stroke width becomes broad when pen pressure is high, and becomes narrow when pen pressure is low. In this paper, the stroke width is alternated by multiplying the stroke width by pen pressure F. 3.3.3. Eraser Pigments on the pigment buffer can be removed by an eraser mask that is defined using the same method as the stroke mask. In real drawings, an eraser flattens the paper surface. In this study, Pm is reduced where an eraser mask is applied, making it difficult to deposit new pigments on these planes. 4. Results and Discussion All of the experiments were performed using a Pentium 4 2.0 GHz, 512 MB RAM in real time. Figure 5 shows the effects of multiple paper textures on the resulting image. The strokes are drawn using the height field with different paper texture numbers, and using the same path. From top to bottom, textures 1, 2, 4, and 12 were used for generating the height field. As the number of textures increase, the patterns on the texture decrease. This is because the present method deposits the pigments on different areas of the paper surface along the drawing directions. Stroke variation using a pen pressure interpolation is shown in Figure 6. Strokes (a) and (b) use basic linear interpolation, with (b) maintaining the maximum pressure points on one edge. Strokes (c) and (d) use quadratic curves with different coefficients for interpolation. Figure 7 shows a comparison between real strokes and virtual strokes produced by the proposed method. For this comparison, pastel strokes were used for the real drawing stroke samples because they are highly affected by paper surfaces. Realistic strokes, similar to those drawn on drawing paper and watercolor paper types 2 and 3 were generated. Multiple paper textures in a stroke reveal a “crater” that is produced by the interaction between paper and pigments. On watercolor paper types 1 and 4, however, the strokes could not accurately render the paper textures. The difference between paper textures lies in the scale of paper irregularity: the first three paper surfaces are shallower than type 4. Two solutions to this problem are considered. The first solution involves setting the converting parameters more carefully, while

the second involves adjusting the incident angle of illumination. Because the incident angle of illumination affects the paper shadow and shade, the optimum angle is different for different types of paper. Similarity between the regenerated strokes and the real pastel strokes also depends on the resolution of the paper textures. Overlapping strokes are shown in Figure 8. In both the real (a, b) and virtual (c, d) images, the pink strokes are drawn over the blue strokes. On the left (a, c), the blue strokes are applied with more pastel pressure than on the right (b, d). The figure-panels appear similar. On the left (a, c), the pink pigments could not be deposited on the paper in areas because the blue pigments have already filled the paper’s irregularity. In contrast, the pink pigments in the images on the right (b, d) could be deposited by blending with the blue pigments. Figure 9 shows some example images using the regenerated strokes. 5.

Conclusion and Future Work

This paper presented a method for regenerating the realistic strokes of an artistic tool that is highly affected by the irregularities of the support medium. The height field value, which is calculated from the gray scale of illuminated paper surfaces, is similar to a real pastel stroke drawing in any direction. This texture-based simulation generates “realistic” pastel strokes in real time. The same method was used to generate an eraser and a blender, allowing for the creation of a pastel-like drawing tool. Several issues remain with regard to the proposed pastel drawing tool. Blending is important for pastel drawing and smoother smudging should generate better resulting images. Smudging effects vary according to the blender, and artists often use pastels in a powdery state. To implement these effects, the stroke softness inherent in pastel drawings must be considered. REFERENCES [1] R.D. Kalnins, L. Markosian, B.J. Meier, M.A. Kowalski, J.C. Lee, P.L. Davidson, M. Webb, J.F. Hughes, and A. Finkelstein, "WYSIWYG NPR: drawing strokes directly on 3d models," In Proceedings of the 29th annual conference on Computer graphics and interactive techniques, pp. 755-762, 2002. [2] M.C. Sousa and J.W. Buchanan, "Observational models of graphite

pencil materials," Computer Graphics Forum, 19(1), pp. 27-49, 2000. [3] S. Takagi, M. Nakajima, and I. Fujishiro, "Volumetric modeling of colored pencil drawings," Pacific Graphics, pp. 250-258, 1999. [4] T.W. Bleser, J.L. Sibert, and P. Mcgee, "Charcoal sketching: returning control to the artist," ACM Transactions on Graphics (TOG), Volume 7(1), pp. 76 - 81, 1988. [5] D. Rudolf, D. Mould, E. Neufeld, “Simulating wax crayons,” 11th Pacific Conference on Computer Graphics and Applications (PG’03), pp. 163-173, 2003. [6] K. Murakami and R. Tsuruno, “Pastel-like Rendering Considering the Properties of Pigments and Support Medium," Conference Abstracts and Applications of SIGGRAPH 2002 (Technical Sketches), p. 227, 2002. [7] C.J. Curtis, S.E. Anderson, J.E. Seims, K.W. Fleischer, and D.H. Salesin, "Computer-generated watercolor," Proceedings of SIGGRAPH97, pp. 421-430, 1997. [8] F. Durand, V. Ostromoukhov, M. Miller, F. Duranleau, and J. Dorsey, "Decoupling strokes and high-level attributes for interactive traditional drawing," In 12th Eurographics Workshop on Rendering, pp. 71-82, 2001. [9] G. Miller, “Efficient algorithms for local and global accessibility shading,” In Proceedings of SIGGRAPH94, pp. 319-326, 1994. [10] S. Strassmann, “Hairy brushes,” In Proceedings of SIGGRAPH 86, 20(4), pp. 225-232, 1986. [11] B. Pham, “Expressive brush strokes,” Computer Vision, Graphics, and Image Processing. Graphical Models and Image Processing 53(1), pp. 1-6, 1991. [12] S.C. Hsu and I.H.H. Lee, “Drawing and animation using skeletal strokes,” In Proceedings of SIGGRAPH94, pp. 109-118, 1994.