3 Senior Design Project Spring 2012

GORT II ECE-492/3 Senior Design Project Spring 2012 Electrical and Computer Engineering Department Volgenau School of Engineering George Mason Univers...
Author: Lenard Henry
2 downloads 2 Views 637KB Size
GORT II ECE-492/3 Senior Design Project Spring 2012 Electrical and Computer Engineering Department Volgenau School of Engineering George Mason University Fairfax, VA

Team members: Faculty Supervisor:

Jay Elliott, Pavel Liudvih, Scott Pierce, and David Shaffer Prof. Michael Young

Abstract: GORT (GeoMobile Omni-directional Robotic Teleprescence) is a user controlled robot that allows the user full interaction with its surrounding environment. GORT incorporates custom two-way audio and video telepresence software, advanced microcontroller acceleration, braking and safety controls of a three wheel omni-directional base, and an on/off user controlled video screen which shows the face of the user. Unlike the current set-up of a conference call, GORT allows the user to navigate through his/her environment with omni-directional wheels mounted on a triangular base. There are a total of three cameras on board so that the user can see obstacles behind, below, and in front of GORT. The forward facing front camera mounted above the video screen allows the user to drive GORT remotely and interact with people in the GORT environment. The communication between GORT and the user needs to be uninterrupted. An isolated network has been established on the GMU campus to verify the advantages of bandwidth and continued connection on a 900MHz network.

1. Introduction It is impossible to be in more than one place at a time. Most solutions to this problem have involved developing faster modes of transportation and communication. GORT is proposed to help to bridge this gap by providing a way for users to control a robotic avatar that is in a remote location. By using the GORT telepresence device, a user will be able to explore and interact with an environment far away from their present location. The goal of this project was to redesign the GORT I system (developed by the ECE 493 Spring’11 team), improve and enhance its functionality, and integrate all components into a working system. Added functionality included: (i) smooth control movements, (ii) obstacle avoidance capability, (iii) noise cancellation, (iv) continuous video stream, and (v) in-sync audio and video feed. Many of the difficulties of using Wi-Fi for networking were avoided by broadcasting on the 900 MHz frequency using an isolated network for the GORT system. 2. System hardware architecture Figure 1 shows the system hardware architecture. The user’s station controller and router are connected to a computer with a monitor and speakers to output the data from GORT; a web camera and a microphone to send the audio/video to GORT; and a router to packetize the data and prepare for transmission. The router is further connected to the broadcasting antenna which enables wireless communication with GORT. The GORT station has a similar router for receiving/transmitting the information, and an Ethernet switch to separate the video/audio signal for the GORT laptop from the control signal for the microcontroller. The laptop is naturally connected with the monitor, speakers, and a microphone for videoconferencing purposes. Motors receive commands from the microcontroller via H-bridges. Proximity sensors feed the microcontroller the data necessary for its safe operation. All power to the GORT components is supplied by the 12V DC battery.

Figure 1: System hardware architecture User commands are input through the Xbox controller, connected to a user’s laptop with a built-in camera and microphone. The laptop is then connected to the MicroTik router RB433 which handles data packetizing and sends/receives it over the 900MHz wireless channel via an L-COM Yagi antenna. The GORT station has a MicroTik RB411 router performing similar functions to the router on the user’s station. The data is then transmitted to the Ethernet switch which sends the videoconferencing signal to the on-board DELL M1530 laptop and control signals to the Arduino Mega2560 microcontroller.

Both the user and the laptop on GORT are running AVSpeediConf software for audio/video signal processing. For videoconferencing purposes, the laptop is connected to the directional shotgun microphone which enhances the speech quality of the speaker in front of the GORT. Other components include a high resolution main Logitech webcam pointing at the speaker; a lower resolution rearview Logitech webcam; and a wide angle steering Logitech webcam elevated behind and above the base for better monitoring. The purpose of the steering camera was to provide additional control when driving GORT through narrow openings such as through door frames. The Hi-fi audio system installed on GORT enhances the audio sound from the user and can be used to produce the horn sound to alert people about the approaching robot. A monitor displays the user’s video and is located at approximately 6” to produce a human-like feel for the speaker. Arduino Mega2560 is at the heart of the movement controls (Figure 2). It interprets the signals received from the user and drives FR801-001 motors accordingly, by sending the Pulse Width Modulated signal via H-bridges. At the same time, the microcontroller monitors the environment around GORT through the array of Ultrasonic HC-SR04 distance sensors and can overwrite the user’s control if it results in unsafe behavior. Each motor drives one omnidirectional wheel. The triangular base with these wheels allows for extra mobility and stability on uneven surfaces. The translation of steering signals onto the omidirectional GORT motorized platform is shown in Figure 3.

Figure 2: GORT II station design

Figure 3: Omi-directional motion in GORT II (for forward=50 and rotate left=25)

GORT II includes several circuits designed and integrated into a panel shown in Figure 4. It involves over 30 components and 100+ connections.

Figure 4: Panel schematics (30 components and 100+ connections) Experimenting with GORT motion control, it was decided that the Freescale H-bridges would be replaced with Simple-H bridges with built-in current sensors. The later feature in conjunction with the proper coding helped to ensure that the motors would not overdraw the current past the peak value (see Experimentation section). Two 20 CFM 50mm fans were added to the H-bridges driving two main wheels to increase the peak current from 20A to 25A. 3. GORT II portability For transportation purposes, it was desired for GORT to be able to be disassembled without any special tools in a relatively short period of time. A series of quick disconnects comprising of mechanical as well as electrical were incorporated into the design. The wires from two webcams, the shotgun microphone, and the monitor power supply were run through a 25 pin D-sub connector. It was determined in the final design that GORT could be disassembled in 3 sections with the top, middle, and bottom sectionsdisassembled in less than 2 minutes. Putting GORT together and starting it takes about 3 minutes.

4. Network and Video Conferencing User input is obtained using the Microsoft Xinput API to read the state of the buttons and analog sticks on the Xbox 360 controller. From there, the data is sent over a simple protocol using TCP where it is encapsulated in command structures. The first byte of each structure contains the type which includes either a movement or a button press. A movement command is followed by three bytes. The first two are a vector corresponding to the left analog stick. The third byte is a scalar indication of how far along the x-axis the right analog stick is pressed. Button commands consist of a single byte, which indicate whether the button is pressed or not.

Figure 5: Quick releases assure quick disassembly

Figure 6: Network Diagram The open-source Gstreamer multimedia library was used for processing audio and video by loading elementsand connecting them to create pipelines. These elements were individual programs which accept streams as inputs through “sink pads”, perform some operation on them and then output them through “source pads”. The Gstreamer was chosen because it had elements which implement Real-Time Transport Protocol (RTP), a streaming protocol that ensures that the audio and video streams do not become out of phase with each other. Although perfect performance was observed on the Linux version of Gstreamer, several elements on the Windows version of Gstreamer were prone to glitches. It was decided that GORT's onboard laptop would run the Linux operating system to minimize potential problems, and that GORT's user application would run Windows. The pipeline created was specifically designed to work with Windows. The pipeline works by combining video and audio streams with the rtpbin element, and sending them over UDP from the server to the client. The video is taken from the webcams, converted to a format that is compatible with the Gstreamer's video encoders, and then encoded as an H263p stream. From there, the rtph263ppay element prepares the stream to be sent over RTP by adding timestamps and other data. The audio stream works similarly, except it uses the speex codec for streaming from the user to GORT and the MP3 codec for streaming from GORT

to the user. The reason for using two different codecs is that on Windows, the Gstreamer's MP3 encoder and its speex decoder do not work, but the MP3 decoder and the speex encoder do. The video client is the exact inverse of the video server. GORT uses three video servers to stream videos to the user and one video client to accept video from the user. The user application uses three video clients to accept video from GORT and one video server to send video to GORT. 5. Microcontroller-based Control Systems The state diagram in Figure 7 shows three different movement modes for GORT.

Figure 7: State Diagram of movement operations

The first mode called theManual with Safety Mode, attempts to safeguard GORT from damaging itself or getting into a situation that is difficult to get out of. In this mode, if GORT begins to slowly veer towards a wall that is on its left hand side, GORT attempts to auto remedy the situation by adjusting the steering to veer slightly away from the wall until it considers the wall to be a safe distance away. The second mode called the Manual Safety Override Mode, overrides the safety features that protect GORT, allowing the pilot to move the robot uninhibited, even if that means crashing into a wall. The third mode is Straight Line Mode designed to be used in a hall, or while GORT is close to a straight object that continues on for some time. When GORT goes into this mode collects data on how far it is from the object, and maintains that distance while going forward without the pilot telling GORT any further instructions until it stops. Figure 8 shows the flowchart for the microcontroller source code.

6. Experimentation The GORT II project resulted in a highly complicated system with different modules which had to work with each other flawlessly over extended periods of time and in real-time. The system gradually evolved from initial design through phases of system development and integration where unexpected surprises required the project team to enhance or develop better solutions. Major problems were related to the GORT motion system and GORT realtime networking. Six distance sensors were mounted on GORT (2 on front, and 2 on each side), and there were6 wires going to the microcontroller that would allow the team to trigger each distance sensor individually. An additional 6 wires from each distance sensor’s echo pin were also wired, and they were routed to an Arduino external interrupt. After extensive testing, it was found that this proved to be a robust system that allowed the microcontroller to process other data while waiting for an echo to come back from the distance sensors. This became very important when other systems began to be added to the microcontroller, such as the H-bridge overheat protection and acceleration. What was eventually understood, was that after some time of dealing with microcontroller freezing, the Arduino Ethernet shield software library had a bug that was being inflicted due to these interrupts that would randomly cause the Microcontroller to freeze. Because of the way the Arduino Ethernet library was coded, it used SPI interrupts to communicate with the Ethernet shield. If there were any other interrupts while the microcontroller was communicating with the Ethernet shield, it would cause the Arduino to freeze. After this discovering, the team had to remove all interrupts from the microcontroller software and find other means of implementing various systems without interrupts. Eventually, the pulseIn() function was decided upon (which was

not ideal), however the team used a timeout so that the function never took longer than 4 ms. The function is called approximately every 16 ms at such a time that it would never interfere with another, more important microcontroller process. Some of the other systems involved an acceleration system which was found to be very robust. An extra function that is called every 16 ms was also added to the microcontroller. The microcontroller keeps track of the current speed as well as the goal speed. Every time the function is called it is first established whether or not the device was accelerating, or decelerating. Then based on this information, the speed is either increased or decreased. This function corresponded to several hundred lines of code. Through testing, it was we determined that the process took anywhere from 600 microseconds to 1 millisecond to complete.

Figure 8: Microcontroller Code Diagram Because of the initial H-bridges blowing, the teamdecided to add an H-bridge protection system that monitors the current going through each H-bridge. Initially, if the H-bridge exceeded 25 A, then the acceleration system would be used to protect GORT from overheating. The current speed would be set to 0, and GORT would be allowed to accelerate back to full speed until it was overdrawing current again, in which case its current speed was reset to 0. This few hundred milliseconds were enough to keep the H-bridges from overheating. After testing with GORT at full weight, it was quickly found that this approach was not sufficient. During acceleration, GORT motors would exceed 25A, and sometimes go as high as 38 A. Because of this, the system was changed to allow surge currents of up to 45 A for less than 3 seconds. If the current surged for more than 3 seconds, GORT would shut down and not move for 5 seconds. Occasionally GORT would still have some trouble with surge currents for longer than 3 seconds, however it was determined that this would be a rare scenario. A solution to this problem would be to purchase more expensive H-bridges that are capable of exceeding 25 A continuous current. The final system added to GORT was a wall following system using the distance sensors. It incorporated the use of 2 Proportional Integral Derivative (PID) loops, one for the front 2 wheels, and one for the back wheel. When GORT was near a flat wall with no contours, it could easily follow it. What became more difficult was when GORT would pass by a door. When a very sudden change of 5+ cm would occur, GORT would be able to identify that the wall

had most likely changed, that this change in the wall could be ignored, and that no adjustment was needed. Anything less than 5 cm could be attributed to GORT oscillation, or distance sensor inconsistency. 7. Conclusions GORT II has performed seamlessly during the final presentation of senior design projects of the ECE department. Furthermore, it was put on demonstration at the IEEE engineering project showcase during the Student Activities Conference in Columbus, OH (5hrs of demo). GORT II was also a major attraction during the ECE 493 senior design presentation day when it moved among people and interacted with them in real-time.