RGSR – Rafael GPU Screen Recording

Developer Manual

Students: Assaf Akrabi, Asaf Porath Rafael Supervisors: Ortal Lev, Yossi Maoz Course Staff: Prof. Elad Michael

Index 1. General 1.1. Introduction 1.2. System Requirements 1.3. Program structure

2. Recorder 2.1. IDE 2.2. Methodology 2.3. User Interface

3. Decoder 3.1. IDE 3.2. Methodology 3.3. User interface

1. General 1.1. Introduction This is a screen capture program. The program is used to capture the screen to video. This program is unique for its low CPU use, since it’s using the GPU’s computing power to handle the computations, along with OpenGL techniques to capture the screen buffers. 1.2. System requirements - Operating system: 32 bit version of Windows XP SP3 or newer. - Processor: Intel Pentium Dual Core 1.8 GHz or newer - Memory: 2 GB of RAM or higher - GPU: CUDA capability 1.2 or higher 1.3. Program structure The program consists of two separate programs: 1. RGSR.exe – The recording software. From now we will refer to it as the “Recorder” 2. Decoder.exe – the offline diff decoding and video encoding software. From now on we will refer to it as the “Decoder”. As can be understood from their names, the Recorder is the online tool used for recording the screen into images and save them to disk. The Decoder is an offline tool used for putting the screen images back together and after that into a video.

2. Recorder 2.1. IDE The Recorder program was developed in C/C++ in Visual Studio environment. It is using OpenGL and CUDA SDKs. It uses the CUDA library which is automatically added by VS when creating a CUDA project. Additionally, GLEW library is used and needs to be included manually in the project properties. 2.2. Methodology o

Methods The programs main class is OpenGLContext Class. It holds all of the main operations required for capturing the screen frame buffer which holds the pixels status in the current screen. OpenGLContext Class methods: In order to initialize this class the constructor must get the window handler of the transparent window. After creation, this class instance performs most of the programs background actions, including: - Make/Unmake functions for setting the current active GL context. - Creating & binding the PBOs used later for capturing the framebuffer. - Reading the framebuffer into the active PBO. After using the class API for capturing two following frames into allocated buffers, a diff action is performed in order to extract only the changed pixels. This is a parallel action done using CUDA. Once a diff buffer is generated, it is copied to the RAM and is processed into COO representation, and then saved to disk. The capture and diff actions are done simultaneously, using 3 PBO buffers. The buffers are each read into and diffed from in their turn, as shown at the picture below:

A frame capture example: Asynchronic

Frame Buffer CUDA memory

1.Current read

3.Previous

2.Current Diff Buffer

The next frame will look like this:

Frame Buffer

Asynchronic

CUDA memory

1.Current

2.Previous

3.Current read

Diff Buffer

Notice that the numbers 1-3 represent the buffer id as they as swapping roles.

o

Actions

Upon program start: Creating transparent window. Getting its context for GL context. Creating PBOs (3 PBOs, all as ‘PACK’ buffer used for read). Creating ‘captured’ folder. Allocating memory for future buffers. Starting the timer for the capture function (80ms). 2.3. User Interface The Recorder’s UI is implemented at from of a tray-icon program. When right clicking the icon, an options menu will appear: Start Recording: Bind and read two initial buffers. Enabling the capture procedure: *First run also saves a full image. Bind and read into a “resting” PBO. Get cursor position (to be used for next frame) Use the other two PBOs data to create the diff file between their captured frames. cudaGraphicsGLRegisterBuffer cudaGraphicsMapResources cudaGraphicsResourceGetMappedPointer

After that we’ve got pointers to the buffers in CUDA memory, and now we can do a parallelized call to calculate the difference between them (diff). Copy the diffed frame into RAM Encode COO(coordinate, data) sparse matrix representation of the diffed frame for minimal size. Save the previous cursor position taken, and the encoded diffed frame to file.

Swap roles between PBOs (prev, curr, currRead) and cursor position variables. Stop Recording: Stops the capture function calls and creates a metadata file which includes the average time between frames for later use in creating the video Clear Recording: Executes a simple command line action to delete the files under the “captured” directory. Exit: Quits the program, freeing resources.

3. Decoder 3.1. IDE The decoder is a tool for offline extraction of the images captured back into video. It was developed in Eclipse environment, without emphasis on performance (different from the Recorder). The tool uses GDI+ library for extracting images and ffmpeg library for encoding video. 3.2. Methodology o

o

Frames First frame is saved as a full image, so its extraction process is easy – all that is done is to use the GDI+ library to save the data in jpeg format. The following images’ extraction is done in the following way – the next diff file is taken and within it are written the pixels that changed from the last frame. What’s done from this point is to take the last frames data and add the diff file to the changed pixels, and we receive the current frame, and then save it also in jpeg format using GDI+. Video The video encoding is done using ffmpeg library, taking the jpeg frames created and turning them into a video. The frames per second parameter is received from the metadata file from the capture phase.

3.3. User Interface The decoder involves a basic user interface. It is implemented as a cmd program, with a menu for choosing whether to decode the frame pictures, encode the video, and exit the program.