Three Dimensional Virtual Calligraphy Simulation with Pen Tablet

Three Dimensional Virtual Calligraphy Simulation with Pen Tablet Akira Wada, Jungpil Shin Three Dimensional Virtual Calligraphy Simulation with Pen T...
Author: Pamela Moody
0 downloads 2 Views 2MB Size
Three Dimensional Virtual Calligraphy Simulation with Pen Tablet Akira Wada, Jungpil Shin

Three Dimensional Virtual Calligraphy Simulation with Pen Tablet Akira Wada, Jungpil Shin Graduate School of Computer Science and Engineering, The University of Aizu, Tsuruga, Ikki-machi, Aizuwakamatsu City, Fukushima 965-8580, Japan [email protected]

Abstract The purpose of this study is realization of 3D virtual calligraphy simulation that is able to do on computer with pen tablet sensuously. To do the simulation sensuously, we propose three models. The first model is a 3D virtual writing brush model. The second is drawing field model, and last is droplet model based on unique and artistic stroke drawn by writing brush. These models are able to transform to the most suitable shape by information acquired from pen tablet. This simulation is very comfortable system. Therefore, so many people of various ages were able to do simulation sensuously and gaily in school festival. Moreover, the result of usability inspection, we confirmed this simulation is applicative calligraphy simulation that user can use as accomplishment, hobbies, arts and so on usually and casually in daily life.

Keywords: Three Dimension, Virtual Calligraphy, Oriental Brush 1. Introduction In calligraphy, stroke drawings of writing brush are very unique and artistic because of unique shape of brush tip. In Japan, calligraphy is popular and familiar culture for many people as accomplishments, hobbies, healings, entertainments, and arts. Moreover, elementary and junior high school students are learning penmanship as national language. However, doing the calligraphy on a daily basis is difficult because to it requires many instruments (writing brush, Indian ink, paper, and so on), and keeping up of these instruments is very bothersome. In recent years, several methods are proposed to generate brush style typeface by computer. Some research proposed virtual calligraphy system by paying attention to dialogicality between user and system using real writing brush and special device[1],[4]. These system are able to express high-quality brush style typeface, though individual uses this system on a daily basis is difficult because of using the special device. In addition, some research approached mechanical simulation of calligraphy by focus on expression method of individual stroke to analyze the feature of writing brush dynamically[2],[3]. This system is also able to express high-quality bush style typeface, however because of unnatural way for drawing (e.g., drawing by pen tablet), this method is not enough to draw interactively. In this paper, we proposed three-dimension virtual calligraphy simulation that is able to do virtual calligraphy sensuously on computer by focus on the dialogicality between user and system. On this system, a pen tablet is used to do the simulation. The meaning of three-dimension is to express a virtual writing brush model on a display, and the model duplicates three dimensional movement of user’s pen. The aim of 3D virtual writing brush model is for user to be able to do the simulation sensuously and comfortable. To express unique and artistic stroke drawn by writing brush, we constructed droplet model based on the stroke. Using this model, user is able to express unique and artistic stroke by own handling. In this study, we aim to construct simple and comfortable simulation for all users.

2. Overview of Simulation The purpose of this thesis is doing the calligraphy simulation sensuously on computer. Consequently, simulation models are constructed by OpenGL to realize calligraphy visually. OpenGL is programming interface of 3D graphics product by Silicon Graphics Company. It operates very quickly and is able to

International Journal of Engineering and Industries (IJEI) Volume3, Number3, September 2012 doi : 10.4156/ijei.vol3.issue3.4

31

Three Dimensional Virtual Calligraphy Simulation with Pen Tablet Akira Wada, Jungpil Shin

draw high-precision 3D graphics. Using pen tablet is Intuos3 Graphics Tablet produced by WACOM Company. Acquired information is XY-coordinate, pressure, direction and altitude.

3. Simulation Models This simulation is configured by three models. The first is 3D virtual writing brush model. The second is drawing field model, and last is droplet model based on a stroke. In current system, each model is independent and doesn’t infect each other.

3.1 Three Dimension Virtual Writing Brush Model 3D virtual writing brush (VWB) is constructed by several parts. They are brush axis, brush head and tip of a writing brush (after that, we call the tip of a writing brush as brush ear). Moreover, brush ear is constructed by body and waist parts. Each part is constructed individually and is integrated as the one VWB. Therefore, each part is able to transform and rotation keeping a sense of unity as writing brush. Fig.1 is the 3D virtual writing brush model constructed in this study.

Fig.1 3D Virtual Writing Brush Model 3.1.1 Movement and Rotation of Three Dimensional VWB 3D VWB is able to move and rotate by information of pen tablet. While user operates the pen on the tablet, VWB duplicates the three dimensional movement of user’s pen by information of pen tablet (XY-coordinate, direction, and altitude) in real time. In this simulation, all expression is expressed on a display. Therefore user has to do all operation paying attention to the display on behalf of pen tablet at hand. Because of that, what user can confirm own operation by VWB visually on a display is very effective to do the simulation sensuously and comfortable. 3.1.2 Shape Change of Brush Ear In calligraphy, brush ear of writing brush is most important component. Brush ear is configured by a great deal of bristle, and its shape is changed so feely. The shape is very unique and variegation. In this study, shape change of brush ear is expressed by using pressure information acquired from pen tablet. Body of brush ear is able to transform to the most suitable shape at one by one pressure information. When brush ear shape is changing, waist of brush ear is fulcrum of transformation. Therefore, brush ear shape is able to change keeping a sense of unity as writing brush. Moreover, brush ear can move and rotate by altitude and direction information. Figure.2 is expressed the shape changing of brush ear. Left figure is standard form of brush ear. Center and right figure’s brush ear shape is changing by pressure. The shape change is expanded by increase of pressure.

3.2 Drawing Field In calligraphy, a large variety of paper for sizes or textures is used. However, paper of “HANSHI” (height : width = 330 : 240) is the most familiar paper for so many people. Therefore in this study, drawing field of “HANSHI” is adopted.

32

Three Dimensional Virtual Calligraphy Simulation with Pen Tablet Akira Wada, Jungpil Shin

Fig.2 Shape Changing of Brush Ear 3.2.1 Correspond to Pen Tablet This section correspond drawing field to input range of pen tablet. In the paper of “HANSHI”, height is longer than width, though in input range of pen tablet, width is longer than height. Thus, drawing field is corresponded to input range by rotating of pen tablet to 90 degrees counterclockwise direction. So in this simulation, X-coordinate is used as Y-coordinate, Y-coordinate is used as Xcoordinate. Therefore, input range of pen tablet is able to be used without change for drawing field. 3.2.2 Texture Expression of Japanese Paper In doing calligraphy simulation sensuously, visual texture is very important. And so, Japanese paper texture is expressed by putting graphic texture of Japanese paper on drawing field. The texture is prepared by scanning Japanese paper as raw picture, and correct color tone of the picture.Figure.3 is the drawing field constructed in Section 3. Size and texture of “HANSHI” is expressed.

Fig.3 Drawing Field

3.3 Expression of Stroke Unique and artistic stroke drawn by oriental writing brush is the most important feature of calligraphy. For example, stroke of “TOME”, “HANE” and “HARAI” is not able to be drawn by pen or pencil. Each stroke is appeared at the end of own stroke. “TOME” stroke is expressed when writing brush stops strictly at the end of stroke. “HANE” and “HARAI” strokes are expressed when writing brush bounces quickly at the end of stroke. In this section, we propose droplet model based on the stroke. Figure.4 shows each stroke feature.

33

Three Dimensional Virtual Calligraphy Simulation with Pen Tablet Akira Wada, Jungpil Shin

Fig.4 Stroke Features 3.3.1 Droplet Model Droplet Model (DP model) is basis model of stroke. A DP model is expressed on drawing field when user put pen on a point of tablet. And stroke is aggregation of DP models drawn back to back by user. Each DP model is drawn independently. And each model doesn’t infect each other, and the gap between models is not filled. Therefore, when user move pen like a shot, gap will be happen between models. However, it is rare that such a quick operation is performed on doing normally calligraphy, and to fill the gap between each model is not necessary. 3.3.2 Shape Variation of Droplet Model Size and shape of DP model is changed by pressure and altitude information. The size of DP model is expanded with pressure increase, and width of DP model is narrowed with altitude decrease. When pressure is max and brush axis is vertical to drawing field, the size and width of DP model is maximum. This shape changing of DP model is apparent on mechanical perspective. DP models are able to move, rotate and transform by information acquired from pen tablet. Each DP model is drawn on position coordinate of drawing field with user’s pen movement in real time. Moreover, each model changes the most suitable size and width by pressure and altitude, and is rotated by direction.Therefore in this simulation, stroke reflects a user’s individual characteristic of pen’s operation. Figure.5 is expressed DP model transformed by pen tablet information. 3.3.3 Color Shading of Droplet Model This simulation is performed on the computer. With that, we propose color shading of DP model by using advantage of digital. In real calligraphy, user has to operate complex procedure when change the color shade of stroke. And so in this simulation, color shade of DP model is changed by pen direction. Because, direction is easiest to operate information for user. Therefore, user can express color shading of stroke at choice so simply by changing pen direction arbitrarily. In addition, user can select depiction type of droplet model from next two types. They are dyed type and skeleton type, and dyed type is default type. When user selects the skeleton type of droplet model to do this simulation, user is able to observe shape changing, color shading, and concatenation of each droplet model. Therefore, depiction of skeleton droplet model helps user to draw a stroke which he wants. Figure.6 is expressed color shading of DP model by pen direction. In operation of general calligraphy, black color like deep Indian ink is used mainly.

34

Three Dimensional Virtual Calligraphy Simulation with Pen Tablet Akira Wada, Jungpil Shin

Fig. 5 Shape Variation of Droplet Model

Fig.6 Color Shading of Droplet Model

Fig.7 a Stroke Expressed Scratch Effect

3.3.4 Expression of Scratch Effect Scratch effect is called as “KASURE” in Japanese. This is expressed when ink quantity decreases rapidly and writing brush moves so quickly. Scratch effect of stroke is quite characteristic of calligraphy. In this simulation, simple scratch effect of stroke is expressed experimentally by reference to shape of writing brush when “KASURE” happened. Figure.7 is expressed simple scratch effect by this simulation in circle.

3.4 Function of Drawing Scene Playback In calligraphy, to draw a stroke which user wants, user has to pay attention to own brush movement. However, in many case, user will pay much attention to a drawing stroke when he is drawing the stroke. And so, user can’t observe own drawing scene without movie of video recording. There, in this simulation, drawing scene playback function is implemented. Playback is constructed by drawn strokes and virtual writing brush movement of user. So user can watch own drawing scene any time to observe droplet models, drown strokes, pen movement, and so on. When watching drawing scene playback, user is able to change playback speed and depiction type of droplet model freely. So, user can select a playback condition according to a point to pay much attention on own drawing way. Therefore, drawing scene playback function will help user to learn good drawing way to draw a stroke that he wants. Figure.8 describes depiction type of droplet model.

35

Three Dimensional Virtual Calligraphy Simulation with Pen Tablet Akira Wada, Jungpil Shin

Fig.8 Dyed type and Skeleton type of Droplet Model

4. Experiment In this section, the calligraphy simulation is conducted by some persons as configured in this study. We evaluate the result of simulation and discuss about it.

4.1 Result and Evaluation of Simulation Figure.9 and 10 are brush style typeface (after that, we call brush style typeface as brush character) drawn by this simulation. While a user was doing simulation, 3D VWB copes the three dimensional movement of user’s pen on a display in real time. In result, user could pay attention to the display and did simulation sensuously and comfortable. Moreover, because of there is 3D VWB on a display, and texture of Japanese paper is expressed on drawing field, most user could able to notice that this simulation is what to do. Figure.9 shows brush character “SHODO” meaning calligraphy drawn by semicursive style of penmanship. In this character, unique feature of writing brush “TOME”, “HANE” and “HARAI” ware expressed as the DP model changes to the most suitable shape. This brush character is expressed by deep black color shade Figure.10 shows brush character “DENSHA” meaning train drawn by user’s free operation of pen. This character is expressed complex color shading by user changed pen direction arbitrarily. This writing style is very comfortable because user can operate the pen so freely and gaily. Therefore, we consider this simulation is also able to be used healing, entertainment, art, and so on.

4.2 Discussion From the result of experiment, user is able to draw a stroke has many features. Moreover, user can express a stroke which has many colors easily. This characteristic is advantage compared with real calligraphy. In addition, it is confirmed that visual effect of this simulation suffices user to do calligraphy simulation sensuously and comfortable.

36

Three Dimensional Virtual Calligraphy Simulation with Pen Tablet Akira Wada, Jungpil Shin

Fig.9 Brush Character “Shodo”

Fig.10 Brush Character “Densha”

5. Usability Inspection of Simulation In this chapter, usability of this simulation is inspected. On doing usability inspection, this simulation system and pen-brush is used. Pen-brush is called as “Fude-pen” in Japanese. Pen-brush is at the middle writing between writing brush and pen and it is used to do calligraphy casually and usually. For example, pen-brush is used to write a letter, draw a picture, autograph of a wedding present, and so on. Its writing touch is similar small brush used to write autograph mainly. Fig.11 shows pen-brush. Right figure is tip of pen brush. This usability inspection is tested by six students. All of them know calligraphy, and three students are accustomed to do calligraphy, however other students are unused to do calligraphy.

Fig.11 Pen Brush

37

Three Dimensional Virtual Calligraphy Simulation with Pen Tablet Akira Wada, Jungpil Shin

5.1 Inspection Items and Methods Inspection items are as indicated below. Second and forth item are more important survey items. Routinely-availability means user can use this simulation usually and casually in daily life.    

Required time to get used to do the simulation Usability Routinely –availability Degree of Proficiency doing calligraphy

Usability is inspected by following steps: Step 1: To write “会津大学” 10 times by two kind of pen-brush Step 2: To practice calligraphy 30 minutes by this simulation Step 3: To write “会津大学” 10 times by two kind of pen-brush Step 4: To respond to questionnaire about this simulation On steps 1 and 3, two types of pen-brush are used and we pay attention to difference of each stroke written by each pen-brush, because many kinds of writing brush will be used in real calligraphy. The first pen-brush is constructed by short length and hard bristles, and writing touch is harsh. The second pen-brush is made up by long length and flexible bristles, and writing touch is fluidity. The second pen-brush is more expensive then the first. Fig.11 is the second pen brush. On step 2, user has to write a character “永” by this simulation after the “Eight Principles of Yong” that is the most famous calligraphy learning method. “Eight principle of Yong “is called as “永字八法” in Japanese. Its explains how to write the eight strokes common in Chinese characters and which are found all in the one character, yong (永, "forever", "permanence"). It is believed that the frequent practice of these principles as a beginning calligrapher could ensure beauty in one's writing. Degree of proficiency doing calligraphy is evaluated by comparing written characters between steps 1 and 3 on the basis of penmanship primary concept. In penmanship, expression of stroke features “TOME”, “HANE”, and “HARAI” (explained in section 3.3) and balance of character are very important components. And, other items are evaluated by result of questionnaire about this simulation. Figure.12 is expressed each stroke feature written by the second pen-brush

Fig.12 Stroke Features written by pen-brush

5.2 Degree of Proficiency Doing Calligraphy This section explains about the results of two students regarding the degree of calligraphy proficiency. On steps1 and 3, Left 5 characters are written by the first pen-brush, and right 5 characters are written by the second pen-brush. 5.2.1 Inspection Result of User A Step1: Fig.13 is result of step 1 written by user A using pen-brush. User A was unused to do the calligraphy. Thus he was bewildered by writing a character using pen-brush, but he wrote each

38

Three Dimensional Virtual Calligraphy Simulation with Pen Tablet Akira Wada, Jungpil Shin

character carefully. However, written characters are similar to a character written by pen, and they have few features of penmanship. Step2: Fig.14 expresses characters drawn by user A using this simulation. Left character is drawn at first, thus this character has few features of penmanship primary concept because user A was not used to draw brush character by pen tablet. Right character is drawn at last of simulation. This character has many good features. Good stroke of “TOME”, “HANE” and “HARAI” is drawn inside each circle. Moreover, constructive balance of character is improved totally.

Fig.13 Brush Characters are shown written by user A. Left 5 characters are written by the first pen brush, and right characters are written by the second.

Fig.14 Brush Character written by simulation on Step 2 Step 3: Fig.15 is result of step3. Each character is expressed features of penmanship. Fig.16 describes the difference of strokes between steps 1 and 3. Upper characters are drawn on step 1. User A is not able to draw “HARAI” well yet, however expression of “TOME” and “HARAI” are very well. Moreover, constructive balance of each character is improved comprehensively. Therefore, this simulation helps user A to be used to do calligraphy.

39

Three Dimensional Virtual Calligraphy Simulation with Pen Tablet Akira Wada, Jungpil Shin

Fig.15 Brush Characters written by User A on Step 3

Fig.16 Features Comparing of Each Step

5.2.2 Inspection Result of User B Step1: Fig.17 is user B’s result of step 1. User B is used to do calligraphy, thus he could express each stroke features “TOME”, “HANE”, and “HARAI”. Moreover, written characters are balanced well almost.

Fig.17 Brush Characters written by User B on Step 1 Step2: Fig.18 expresses characters drawn by user B using this simulation. Left character is drawn at first. In this time, user B was not accustomed to do simulation with pen tablet, thus written character has few stroke features. On the other hand, the right character written at last is balanced very well, and stroke features are expressed clearly. This means, user B understood techniques how to adjust pen operation to draw good strokes through doing this simulation. For example, he learned technique of pressure control to express “TOME” and “HARAI” clearly.

Fig.18 Brush Characters written by User B Fig.19 Compared characters with steps 1 and 3

5.3 Questionnaire Results about this Simulation Questionnaire items and result average for each item are as indicated below. Users answered to each questionnaire item by five-level rating for seven items and written style for five items. In this case, 1 is

40

Three Dimensional Virtual Calligraphy Simulation with Pen Tablet Akira Wada, Jungpil Shin

worst and 5 is best evaluation. Routinely-availability means user is able to use this simulation usually and casually in daily life.  Usability of simulation : 4.0  Routinely-availability : 4.7  Want to use again : 4.5  Amusement of doing simulation : 4.8  Actual feeling degree of proficiency doing calligraphy : 3.2 *1  Satisfaction degree of simulation time : 4.0  Satisfaction degree of simulation function : 3.5  Desired time to be used to use this simulation : 10 ~ 30 minutes  Desired time to become skillful to do calligraphy using this simulation : *Not less 1~2 hour  Desired function : *Smooth behavior of virtual writing brush, *Reduction of data dispersion around pen erectness *Graphical user interface (GUI) to clear display, watch a playback, etc  Which pen-brush is convenient : The second (expensive) pen-brush is useful to write character for all users.  What one has learned : *How to express stroke features *How to use pen-brush *Adjust pen pressure to express stroke features *To draw a character balanced well *To be used to do calligraphy

5.4 Comprehensive Evaluation In this chapter, usability inspection of simulation was tested by 6 students. The mainly inspection topics are handleability of this simulation and degree of proficiency doing calligraphy. From the result of section 5.2, it is confirmed that most users became skillful to draw a stroke by both this calligraphy simulation and pen-brush. Thus, they could express stroke features “TOME”, “HANE”, and “HARAI” partially at last of step 2 and 3. Moreover, on steps 2 and 3, written characters are improved balance clearly compared with at first. Therefore, we confirmed this calligraphy simulation is able to help users to be used to and become skillful to do calligraphy. From these results, this calligraphy simulation will be able to make a contribution proficiency doing calligraphy. Thus, most users became good to do calligraphy by pen-brush. However on *1 of questionnaire results, user’s actual feeling of proficiency doing calligraphy is low (3.2 point). This discrepancy will be happened by simulation time. 30 minutes will be enough to be used to use this simulation, but it is short of time to realize feeling to be got skillful to do calligraphy. Therefore, we are able to anticipate user will be realized the feeling to do this simulation 1~2 hours or more. Moreover, from the section 5.3, usability evaluation of simulation is low. Thus, we must improve this simulation system to be become usable for all users by reference to demand of users. And also, this simulation was performed by so many people in open campus of university. Most users could do the simulation comfortable and gaily by only basic explanation about the simulation. Any user did the simulation continuously come to some dozen times to be draw good character of picture. From difference in writing brush and pen, many users can not draw character or picture as user would like. However, we confirmed most user will be able to draw strokes arbitrarily to have a lot of practice. Therefore, we confirmed this simulation is able to be used as applicative calligraphy simulation.

6. Conclusion . In this thesis, we proposed three dimension virtual calligraphy simulation to do sensuously and comfortable with pen tablet. Through this paper, it is shown that users can do calligraphy sensuously and comfortable in daily life as accomplishments, hobbies, healings, arts, and so on by using this simulation. Therefore, the purpose of this paper is achieved. At this moment, there are two future works. The first is to improve each model of simulation. For example, shape variation of the brush ear is very simple. The movement of brush ear model needs to improve its realistic to express more unique and various shapes. The second is expression of scratch effect and diffusion effect. Diffusion effect is called as “NIJIMI”. Diffusion effect is expressed on drawing stroke by writing brush when writing

41

Three Dimensional Virtual Calligraphy Simulation with Pen Tablet Akira Wada, Jungpil Shin

brush absorbs ink excessively. Both features are most important characteristic of calligraphy. In this study, simple scratch effect is expressed. However real scratch effect and diffusion effect are more complex and compositive phenomenon. Therefore we need to study more and verify the phenomenon refer to preceding study and experiment, and try to express more realistic.

7. References [1] Hideyuki Shimada, Yasuhiro Shimada, Mitsuru Ohkura, “The development of Virtual Calligraphy system Using Writable Brush” , Journal of Information Processing Society of Japan (IPSJ), Vol.47, No.12, pp.3392-3401, 2006. [2] Suguru Saito, Masayuki Nakajima, “3-D Physical Writing model for Interactive Painting,” Journal of Information Processing Society of Japan (IPSJ), Vol.41, No.3, pp.608-615, 2000. [3] Suguru Saito, Masayuki Nakajima,” Brush Model of Painting Tool,” Journal of The Institute of Electronics, Information and Communication Engineers (IEICE), Vol.D-12-157, p330, 1999. [4] Chun, and Tai:Real-Time Painting with an Expressive Virtual Chinese Brush, IEEE Computer Graphics and Applications, Vol.24, No.5, pp.76-85, 2004. [5] Tomoaki Tsuchida, Xiangshi Ren, Jibin Yin, ”Tool of Pen Inclination and Direction,” Interaction of Information Processing Society of Japan (IPSJ) 2008. [6] Toshinori Yamasaki, Masahiro Yamamoto, Seiji inokuchi, “Calligraphy CAI System Using Writing Speed Analysis,” Journal of The Institute of Electronics, Information and Communication Engineers (IEICE), Vol.J70-D, No.11, pp.2071-2076, 1987. [7] Won-Du Chang, Jungpil Shin:Interactive Virtual Oriental Brush with Pen-Tablet System, Lecture Notes in Artificial Intelligence (Proceedings of the 11-th International Conference on KnowledgeBased Intelligent Information), pp. 387-394, 2007. [8] http://www.knowledgerush.com/kr/encyclopedia/Eight_Principles_of_Yong/ Eight Principles of Yong . [9] J. Shin, H. Sakoe, “Stroke Correspondence Search Method for Stroke-Order and Stroke-Number Free On-Line Character Recognition-Multilayer Cube Search-”, IEICE Trans. & Syst. , vol.J82-DII, No.22, pp.230-239, Feb. 1999. [10] A. Takeda, J. Shin, W.-D. Chang, “Modeling Scratchiness Effect of Oriental Writing Brush,” 2006 IEEE International Conference on Computer and Information Technology, pp.121-126, Sep. 2006. [11] N. Takesue, K. Mochida, A. Kitadai, M. Nakagawa, “A Handwriting-based Kanji Learning System Enabling Teachers to Designate Evaluation Points”, IPSJ SIG, 2005-CE-78, pp15-22, Feb. 2005. [12] J. Shin and A. Takeda, “Character Learning System Using Inter-stroke information,” Lecture Notes in Artificial Intelligence (Proceedings of the Eigth International Conference on Knowledge-Based Intelligent Information), pp. 165-174, Sep. 2004. [13] R. Saito, J. Shin, “Simulating Oriental Brush Character Considered with Brush Ahead Rotation,” The 2nd International Conference on the Frontiers of Information Technology, Application and Tools (FITAT2009), pp. 83-86, 2009.

42