1160

JOURNAL OF MULTIMEDIA, VOL. 9, NO. 10, OCTOBER 2014

A New Design of Video Streaming Capture and Transmission System Xingyu Geng Information Engineer School of Hebei University of Technology, Tianjin 300401, China Email: [email protected]

Fan Yang*, Guofeng Pan, Zhe Kong, and Sha Wang Information Engineer School of Hebei University of Technology, Tianjin 300401, China *Corresponding author, Email: [email protected], [email protected], [email protected], [email protected]

Abstract—Aiming to solve some existing problems such as low video streaming rate, high cost and complex set-up, a new video streaming capture and transmission system based on servfox is proposed. This system uses servfox to invoke V4L APIs to capture video streaming in Linux operating system, and then transmits the data to a remote monitoring client through Internet which can display the video streaming. Remote client can be either a software or a web browser. The experimental results show that the image captured by this system is high-quality and the streaming rate is high. At the same time, the system possesses high reliability and low cost. Compared with existing methods, this system can be widely used in the process of video streaming capture and transmission. Keywords—Servfox; Embedded; Streaming; Network

I.

ARM;

Linux;

Video

INTRODUCTION

Over the past a few years, a great progress has been made in the field of video streaming capture [1-2]. The abilities of artificial labor have been enhanced after the emergence of network monitoring technology. According to human beings’ thought and intelligence, people can combine the characteristics of network monitoring to accomplish remote operation assignments and a few awkward works that maybe harmful to human beings, such as geological detection, remote monitoring, or rescues after disaster. Recently, some related achievements of video streaming capture have been made in China [3]. The capturing, processing and transmitting of video streaming data are three key procedures in network monitoring system, which can affect the user experience and the real-time property of system directly. However, considering the complicated techniques, it’s difficult to integrate or to exploit this technology. So it is not suitable for the existing video streaming techniques to be used on network monitoring system. Aiming to solve the problems above, this paper proposes a new embedded video streaming transmission system based on Servfox [4]. The new system can achieve a solution of low cost and high efficiency. This system consists of ARM embedded platform, embedded Linux operation system and GSPCA USB camera. Mul© 2014 ACADEMY PUBLISHER doi:10.4304/jmm.9.10.1160-1165

tithreading technology is used to improve the processing speed and efficiency, which means that the video capturing, the video processing, and the video transmitting will be conducted in different threads, so they can work rapidly and independently. USB protocol and TCP/IP protocol are used which are fast and easy to achieve. In order to achieve the frameworks of both C/S and B/S monitoring, both the client software and Web are adopted [5]. All the methods can provide the system better portability, stability and instantaneity. II.

SYSTEM S UMMARY

There are four sections in this new system-- section video capturing, section video processing, section video transmitting and section video monitoring. In section video capturing, GSPCA camera, USB hardware I/O port, and embedded ARM processor constitute to the platform [6]. In section video processing, the main function of this section is to depress and code the video data through C/C++ programming language. In section video transmitting, a transmission tunnel is established based on TCP/IP protocol to transmit the processed data in section video processing, and socket is used because of its high transmission quantity of data. The section video monitoring is built on remote monitoring PC to display and record the video data which are transmitted from former sections. The system has two modes to monitor—client software and Web browser. A brief frame diagram of this system is clearly seen from Fig. 1. III.

THEORIES AND CONSTRUCTION OF S YSTEM

A. Capture and Transmission Theory The V4L audio and video APIs provided by Linux kernel are invoked in Servfox to grab video data [7]. V4L is short for Video for Linux, which is a programmable interface provided by Linux kernel. The V4L APIs involve the switch of video devices, capturing or processing video data, etc. A list of key parameters is used in capturing as following, and the capturing flow diagram is clearly seen from Fig. 2. VIDIOC_S_FMT: set the format of video image VIDIOC_REQBUFS: allocate video buffer

JOURNAL OF MULTIMEDIA, VOL. 9, NO. 10, OCTOBER 2014

1161

Figure 1. The video streaming capture and transmission system

Figure 2. Flow diagram of V4L capture

VIDIOC_QUERYBUF: query the length, the offset address and status of video buffer VIDIOC_STREAMON: start video capture and save the captured data in video buffer. TCP/IP protocol is applied to transmit video data in this system [8]. It is suitable to use TCP/IP protocol to transmit video data, because TCP/IP is beneficial to transmit massive data consecutively and it can prevent such issues effectively from losing or jumping frames. During the procedure of transmitting, a new thread will be established to connect to a remote monitoring PC and to receive commands from it through sockets [9]. B. Transplantation to ARM Platform A driver free USB camera is utilized as video capture device in this system. Therefore, we need to add the UVC (USB video class) camera support in Linux kernel con-

© 2014 ACADEMY PUBLISHER

figuration before the transplantation of Servfox. The procedure can be simply displayed as following [10]: Device Drivers ---> Multimedia support ---> Video For Linux [*] Enable Video For Linux API 1 (DEPRECATED) (NEW) [*] Video capture adapters (NEW) ---> [*] V4L USB devices (NEW) ---> USB Video Class (UVC) [*] UVC input events device support (NEW) USB ZC0301 [P] webcam support (DEPRECATED) The running platform and the cross compiler should be designated when the Linux kernel is compiled. An example command is as following: make zImage ARCH=arm CROSS_COMPILE=arm-linux. Now, burn the compiled kernel binary file companied with U-Boot and Yaffs2 filesystem into the NandFlash chip on ARM platform, and then reboot the platform so that the system can identify the USB camera correctly [11]. Servfox doesn’t depend on any other libraries to run, but it is essential to configure the platform the Servfox will run for. Open the Makefile file which is located in the source code package of Servfox, and then replace the compiler value with arm-linux-gcc. A static compiling parameter is also recommended here. After compiling, the binary file is copied to Yaffs2 filesystem, and the executable right to the file needs to be added. C. Construction of Monitoring Platform Servfox running in ARM embedded platform can only achieve video capturing, video processing and video transmitting. As a result, a monitoring platform needs to be constructed on remote PC to display the video data from ARM platform. 1) Construction of Spcaview Spcaview is a monitoring client software package, including three parts—spcaview, spcaserv, and spacacat. The three parts separately paly the video streaming, con-

1162

JOURNAL OF MULTIMEDIA, VOL. 9, NO. 10, OCTOBER 2014

struct the streaming server, and grab the picture. Spcaview depends on SDL library when running on Linux, so it must be installed firstly before compiling the spcaview source code. 2) Construction of Web Server In this system, the video streaming can be monitored by web browser on remote PC. Therefore, a network server needs to be established at ARM platform. The system utilizes BOA server as its network server because of its high efficiency and transplantation. BOA is an embedded friendly single-task http server, which can run on Linux or Unix operation systems [12]. BOA is an open source project with high performance and security, and it also supports CGI. The compiled BOA binary file will be copied to Yaffs2 filesystem, then the server should be configured, and the executable right should also be added to it. IV.

INTRODUCTION OF EXPERIMENTAL P LATFORM

The system central processor utilizes Samsung S3C6410 SoC, of which the frequency is up to 667MHz; 256MB DDR RAM with 32bit bandwidth; 2GB NandFlash, which is suitable for Yaffs2 filesystem, and prevents data from losing when power is down; 4.3’’ touchscreen LCD with HVGA resolution, leading to nice UI experience; DM9000 10/100M self-adaptive Ethernet network chip with standard RJ-45 network interface; RS-232 serial port interface; High speed USB 2.0 interface; ZC301P USB network camera, with VGA level image capturing [13]. Embedded software platform: embedded Linux kernel 2.6.36.2; Yaffs2 root filesystem; Qt 4.8.5 GUI library. Remote monitoring platform on PC: Ubuntu 12.04.2 LTS operation system; Linux kernel 3.2; SDL library 1.2; Qt 4.8.5 GUI library; Google Chrome 31 web browser.

V.

EXPERIMENTAL RESULTS AND ANALYSIS

A. Experiment Procedure The USB camera needs to be plugged into the high speed USB interface on embedded platform, and the serial cable needs to be plugged into RS-232 interface on embedded platform with connection with PC on the other side to send commands. The Ethernet cable needs to be plugged into RJ-45 interface on embedded platform with connection to PC on the other side. The running command of Servfox on embedded platform is as below (assume the USB camera device node is video2, and network listening port is 8080): ./servfox -d /dev/video2 -s 640x480 -w 8080 –g If the system starts regularly, some messages will be printed as below: servfox version: 1.1.2 date: 10:15:2013 (C) [email protected] Waiting .... for connection. CTrl_C to stop !!!! There are two methods to monitor the video streaming data on PC: spcaview client software and web browser. 1. spcaview client software: running command is as below (assume the IP address of PC is 192.168.1.10; the IP address of embedded platform is 192.168.1.11; both network ports of PC and embedded platform are 8080): ./spcaview –g -s 640x480 -w 192.168.1.1:8080 If the client is regularly started, some messages will be printed on embedded platform as below: Got connection from 192.168.1.10 At the same time, a monitoring window will emerged on PC, and the video streaming will be displayed in real-time. A series of screenshots of monitoring window by client software is shown from Fig. 3.

Figure 3. Display of Client Software

Figure 4. Display of Web Browser

© 2014 ACADEMY PUBLISHER

JOURNAL OF MULTIMEDIA, VOL. 9, NO. 10, OCTOBER 2014

2. web browser: this system supports web browser monitoring in real-time. First, BOA server must be ran on embedded platform. The running command is as below: ./boa & Open the Google Chrome web browser on PC, and enter the following website on the browser. http://192.168.1.11/http-java-applet/index.html Now, a series of screenshots of monitoring window by web browser is shown from Fig. 4. B. Experimental Results Now making a series of contrasts experiments between video streaming system based on Servfox proposed by this paper (method Servfox), video streaming system based on OpenCV (method OpenCV) and H.264 (method H.264) [14]. OpenCV is an open source computer vision library, which provides a vast of video capturing and processing APIs. H.264 is a popular video depression standard, which is widely used in network video transmission. During experiments, the hardware platform of these three methods is definitely same. Besides, the network condition, the targets to be captured, and monitoring method on PC are definitely same too. Method Servfox utilizes Servfox 1.1.3; method OpenCV utilizes OpenCV 2.3 library to program; method H.264 utilizes standard H.264 library to compile. Average experimental data is computed after 50 times repetitive experiments. The experimental procedures of method OpenCV and method H.264 refer to a few of existing papers [15]. This paper will take the analyses from following aspects: video rate, most fluent resolution, complexity of system platform construction, occupied ratio of system hardware resource, and starting time of system. 1. The primary index of capturing and transmitting of video streaming is video rate. Only when video rate is high enough can take analysis on target objective. Generally speaking, human eyes suppose the video is fluent when the video rate reaches 24fps or higher. Servfox which adopts JPEG video streaming standard and multithreading technique can reach a 30fps video rate. TABLE I.

1163

the largest extent can reach 140%. Method H.264 takes the second place, and method OpenCV ranks the last. The results can be clearly seen from Table I and Fig. 5. 2. The key index of capturing and transmitting of video streaming is the quality of video signal. A higher quality transmitting system will be more convenient to carry out follow-ups, such as image processing. The experimental results show that method Servfox can transmit video signals of level VGA (640x480) clearly, as same as method H.264. Meanwhile, method OpenCV can only transmit video signals of level QVGA. The results can be clearly seen from Table II. TABLE II.

Methods servfox OpenCV H.264

Video rate(fps) 29 12 21

Figure 5. Comparison of Video Rate

The experimental results show that method Servfox hold a large advantage over the other two methods, and

© 2014 ACADEMY PUBLISHER

Most fluent resolution 640x480 320x240 640x480

3. Unlike desktop system, the embedded system is short of both hardware and software resources. As a consequence, not every running libraries or other running environments can be deployed at the same time on embedded system. Therefore it is a common and essential way to set up the running environments before using embedded system, which is also the most difficult procedure during embedded transplanting. Therefore, the complexity of system platform construction is one of the normal standards to evaluate the availability of one system. Servfox can run regularly without any other running environment, leading an easier procedure of system platform construction. The experimental results show that method Servfox need no more running environment. In other words, method Servfox is easy to transplant on embedded platform. On the other hand, both the method OpenCV and method H.264 need external running libraries to run regularly. The results can be clearly seen from Table III. TABLE III.

COMPARISON OF PROCESS OF SYSTEM BUILDING Methods servfox OpenCV H.264

COMPARISON OF VIDEO RATE

Methods Servfox OpenCV H.264

COMPARISON OF MOST SMOOTH RESOLUTION

TABLE IV. Methods servfox OpenCV H.264

Depended environments none OpenCV library H.264 library COMPARISON OF RESOURCE USAGE

CPU usage ratio (%) 3 90 4

RAM usage ratio (KB) 17492 28236 18336

4. Compared with PC, embedded platform is resources deficiency, as the processor speed or memory volume is far lower than PC’s. Therefore, the platform resources occupied ratio is an important factor to evaluate an embedded system solution. The experimental results display that method Servfox occupies the lowest both the CPU and RAM resources when running. Method H.264 also occupies low, but not good enough. On account of no hardware acceleration, method OpenCV occupies a huge resources ratio when

1164

JOURNAL OF MULTIMEDIA, VOL. 9, NO. 10, OCTOBER 2014

running. The results can be clearly seen from Table IV and Fig. 6.

Linux operation system with high extensibility is applied, which makes it possible to fuse other sensors data together to proceed remote controls. This system achieves two modes of monitoring framework—B/S and C/S with web technique. A series of contrastive experiments in video capturing and streaming has been made in this paper. The experimental results show that the method Servfox owns better stability, efficiency and capacity of real time. Consequently, the system proposed by this paper is suitable for a video streaming system. REFERENCES

Figure 6. Comparison of Resource usage

5. Starting time can’t be ignored. This indicates the level of code optimization and code efficiency, which is more important on a deficient embedded platform. The experimental results conduct that method Servfox cost the lowest time to start. Method H.264 takes the second place, and method OpenCV ranks last. The results can be clearly seen from TABLE.V and Fig. 7. TABLE V.

COMPARISON OF STARTING TIME

Methods Servfox OpenCV H.264

Starting time(s) 0.8 6.7 2.1

Figure 7. Comparison of Starting Time

VI.

CONCLUSIONS

In this paper, a new network video streaming capturing and transmitting system has been introduced. It can achieve high-rate and real time video capturing, video processing and video transmitting. The system includes four sections—capturing, processing, transmitting and monitoring. The system utilizes Servfox—an advanced video capturing method, to capture high quality video signal, and at the same time this method has a high video rate and a good resources occupied ratio. Embedded

© 2014 ACADEMY PUBLISHER

[1] Guo Jianguang and Fu Xiuhui. “Research on realizing net-robot Tele-operation”. Chinese Journal of Science Instrument, vol. 26, pp. 477-478. August 2005 [2] Hu Ping and Fu Xiuhui. “Robotic teleoperation based on web”. Chinese Journal of Science Instrument, vol. 26, pp. 479-480, August 2005 [3] Xiaoming Liu, Lian Pan. “The Study of Target Tracking Based on ARM Embedded Platform.” Journal of Computers, vol. 7, pp. 2015-2023, August 2012. [4] Daode Zhang, Yurong Pan, Xingyu Hu. “Design of High-Speed Parallel Data Interface Based on ARM &FPGA.” Journal of Computers, vol. 7, pp. 804-809. March 2012 [5] Gao Junxiang, Du Haiqing. “Design of Greenhouse Surveillance System Based on Embedded Web Server Technology.” Elsevier, vol. 23, pp. 374-379. May 2011 [6] Bo-Sheng Wu, Chen-Chiung Hsieh, Chia-Chen Lee. “A Distance Computer Vision Assisted Yoga Learning System.” Journal of Computers, vol. 6, pp. 2382-2388. November 2011 [7] J Weerachai, P Siam and K Narawith. “USB Security Camera Software for Linux.” Elsevier, vol. 8, pp. 171-176, August 2011 [8] Mark Kraeling and Andrew McKay. “Linux for Embedded Systems.” Elsevier, vol. 20, pp. 921-959, August 2013 [9] Zheng Wei and Wang Yuechao. “Design of Internet-based tele-robot soccer system.” ROBOT, vol. 28, pp. 255-263, March 2006 [10] Zheng Wei and Wang Yuechao. “On human-robot interaction in the network-based multi-robot teleoperation system.” ROBOT, vol. 29, pp. 513-519, June 2007 [11] Weigong Zhang, Suran Di, Yongxiang Zhang and Liming Li. “Image Capture and Display Based on Embedded Linux.” Elsevier, vol. 25, pp. 1141-1148, July 2012 [12] Liu Wenfeng, Li Chengyuan and Li Shanping. “Research on embedded Linux operating system.” Journal of Zhejiang University (Engineering Science), vol. 38, pp. 447-452, April 2004 [13] Jing Li and Weidong Hao. “Research and Design of Embedded Network Video Monitoring System Based on Linux.” Computer Science and Software Engineering, 2008 International Conference, pp. 1310-1313, December 2008 [14] Chen Liwei, Yang Jianhua, Cao Xiaohuan and He Ning. “Indoor environmental monitoring system in the framework of Internet of things.” Journal of University of Electronic Science and Technology of China, vol. 41, pp265-268, February 2012 [15] Luo Yong, Shi Li, Zhou Liangjie, Fan Hongkun and Yang Chun. “Design of new intelligent monitoring and control system for animal robot.” Chinese Journal of Science Instrument, vol. 33, pp. 2429-2435, November 2012

JOURNAL OF MULTIMEDIA, VOL. 9, NO. 10, OCTOBER 2014

1165

Xingyu Geng, born in Hebei, China in the year of 1990. Post graduate student mayor in circuits and systems in Hebei University of technology China.

Zhe Kong, born in Hebei, China in the year of 1989. Post graduate student mayor in circuits and systems in Hebei University of technology China.

Fan Yang, born in Jilin, China in the year of 1964. Doctor mayor in circuits and systems in Hebei University of technology China.

Sha Wang, born in Hebei, China in the year of 1988. Post graduate student mayor in circuits and systems in Hebei University of technology China.

Guofeng Pan, born in Neimenggu, China in the year of 1968. Doctor Mayor in circuits and systems in Hebei University of technology China.

© 2014 ACADEMY PUBLISHER