Fernando, W. A. C., Canagarajah, C. N., & Bull, D. R. (2000). Fade, dissolve and wipe production in MPEG-2 compressed video. IEEE Transactions on Consumer Electronics, 46(3), 717 - 727. DOI: 10.1109/30.883437

Publisher's PDF, also known as Version of record

Link to published version (if available): 10.1109/30.883437 Link to publication record in Explore Bristol Research PDF-document

Copyright © 2000 IEEE. Reprinted from IEEE Transactions on Consumer Electronics. This material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way imply IEEE endorsement of any of the University of Bristol's products or services. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to [email protected]. By choosing to view this document, you agree to all provisions of the copyright laws protecting it.

University of Bristol - Explore Bristol Research General rights This document is made available in accordance with publisher policies. Please cite only the published version using the reference above. Full terms of use are available: http://www.bristol.ac.uk/pure/about/ebr-terms.html

~

Fernando et al.: Fade, Dissolve and Wipe Production in MPEG-2 Compressed Video

7 17

FADE, DISSOLVE AND WIPE PRODUCTION IN MPEG-2 COMPRESSED VIDEO W. A. C. Fernando, C. N. Canagarajah and D. R. Bull Image Communications Group, Centre for Communications Research, University of Bristol, Merchant Ventures Building, Woodland Road, Bristol BS8 1 UB, United Kingdom E-mail: [email protected]

ABSTRACT

video is also essential. Therefore, a typical desktop video editing system must first convert the compressed domain representation to a spatial domain representation and then perform the editing function on the spatial domain data. Then, output of the editing system must he recompressed. This increases the overall computational complexity of the editing process. In order to avoid the unnecessary decompression operations and compression processes, it is efficient to edit the image and video in the compressed format itself.

With the increase of digital technology in video production, several types of complex video special effects editing have begun to appear in video clips. In this paper a novel algorithm is proposed for fade-in, fade-out, dissolve and wipe video special effects editing in compressed video without full frame decompression and motion estimation. DCT coefficients are estimated and use these coefficients together with the existing motion vectors for these special effects editing in compressed domain. Results show that both Though video editing in spatial domain is well established, objective and subjective quality of the edited video in work on processing video effects in compressed domain has compressed domain closely follows the quality of the edited not received a lot of attention. Since, Discrete Cosine video in uncompressed video at the same hit rate. Transform (DCT) is chosen as the hasis function in compressions standards [ l , 2, 31, most of the compressed1. INTRODUCTION domain based processing methods were developed to deal with DCT domain processing of the data. The basic idea in Due to the increase in video products such as digital DCT-domain based processing is that the spatial domain cameras, camcorders and storage devices (DVDs), digital processing can be replaced by an equivalent processing of video editing is becoming increasingly popular. Video the DCT-domain representations. Therefore, most previous editing effects are needed to enhance the quality of the video approaches on video editing in compressed domain have production. Most video editing can he divided into two been performed in DCT domain. Previous approaches can be major categories: abrupt transitions and gradual transitions. divided into two categories: special effects editing for JPEG Gradual transitions include camera movements: panning, images and special effects for MPEG sequences. Shen et a1 tilting, zooming and video editing special effects: fade-in, [4] proposed inner-block transform method to perform fade-out, dissolving, wiping. Abrupt transition is the simplest regular geometric transformations for JPEG images. Shen et edit between two shots in which the transition is immediate al also developed fast algorithms for DCT domain between two frames. Special effects occur gradually over convolution [5].It was shown that pixel-wise multiplication multiple frames. Though, the number of possible video in the spatial-domain can be replaced by a convolution special effects is quite high in video production, most of function in the DCT domain. Smith et a1 [6] showed how the these special effects fall into fading, dissolving or wiping. algebraic operation of pixel-wise and scalar addition and During a fade, the intensity gradually decreases to, or multiplication, can he done in DCT compressed domain. increases from, a solid colour. In a dissolve, two shots, one increasing in intensity, and the other decreasing intensity, are Authors used these operations in JPEG images to implement two common video transformations: dissolving and subadditively mixed. Wipes are generated by translating a line titling. They argued that these scalar addition and across the frame in some direction, where the content on the multiplication could he implemented on quantised DCT two sides of the line belong to the two shots separated by the coefficients. However due to the non-linear behaviour of the edit. All these special effects are used to produce gradual mapping function, many problems were introduced with this transitions between two scenes. These video editing tools are scheme [6]. All these DCT domain based approaches have designed for spatial domain processing. been applied only for JPEG images. Shen have proposed The large channel bandwidth and memory requirements for DC-only fade-out operation for MPEG compressed video the transmission and storage of image and video necessitate [7]. This algorithm is proposed under the assumption that the use of video compression techniques [1,2,3]. Hence, the fade-out is viewed as a reduction of picture brightness. visual data in multimedia databases is expected to he stored However, this is a poor assumption of the actual fade-out mostly in the compressed form. Thus editing of compressed operation in video production.

Original manuscript received June 19,2000

0098 3063100 $10.00

2000 IEEE

Authorized licensed use limited to: UNIVERSITY OF BRISTOL. Downloaded on February 24, 2009 at 05:55 from IEEE Xplore. Restrictions apply.

718

IEEE Transactions on Consumer Electronics, Vol. 46, No. 3, AUGUST 2000

Currently, most existing video processing operations performed on the compressed sequences require motion estimation or full frame decomwession. To maximise the benefits from data compression without incurring extra computation and storage, it would be advantageous to develop processing algorithms that do not require decompression of the entire compressed data. Operations on compressed bit streams directly or with minimal decoding of relevant information will then eliminate the computational time necessary for full decompression and the extra storage needed for the decompressed results. In this paper we present a novel technique for fade-in, fade-out, dissolve and wipe editing in compressed video without full frame decompression and re-compression. Rest of the paper is organised as follows. In section 2 mathematical for fading, dissolving and wiping are presented, Section 3 discusses an overview for MPEG-2 video, a model for uncompressed video editor and the conventional compressed domain video editor, also describes the process of DCT coefficients extraction from compressed domain, Proposed scheme for video special effects editing is presented in section 4, Some experimental results are given in section 5. Finally, section 6 presents the conclusions and future work.

(f”(1,Y )

(L,t L)< n S L,

(3)

where, c i s the video signal level (solid colour), $ , ( x , y ) is the resultant video signal, f, ( x , y ) is picture F, g, ( x , y ) is Picture G* 4 is length Of sequence F, is the frame number, L is length of dissolving/fading sequence and L2is length of the total sequence,

2.2 Wiping Wiping is a transition from one scene to another wherein the new Scene is by a moving boundary. This moving boundary can be any geometric shape. However in practice this geometric shape is either a line or a set of lines. According to the geometric shape of this boundary, there are about 20-30 different moving boundaries used for wiping in as shown in videg production. Wiping can be Equation (4).

n iL,

f,,( x , Y)

2. MATHEMATICAL MODEL FOR VIDEO EDITING

L, < n < ( L + L , )

(4)

( L + L , )< n < L2

In this section mathematical models for fade-in, fade-out, dissolving and wiping are discussed.

2.1 DissolvinglFading In video editing and production, proportions of two or more picture signals- are simply added together so that the two pictures appear to merge on the output screen. Very often this process is used to move on from picture F to picture G. In this case, the proportions of the two signals are such that as the contribution of picture F changes from 100% to zero, the contribution of picture G changes from zero to 100%. This is called dissolving. When picture F is a solid color, it is called as fade-in and when picture G is a solid colour, it is known as fade-out. Therefore, dissolving, fade-in and fadeOut can be modelled as shown in Equations (11, Equation (2) and Equation (3) respectively.

.t;(1,Y ) s,,(*, Y ) =

1-

n-r,

O