Wearable Sensors Project Proposal Josh Handley Rosy Logioia Gouri Shintri Clay Smith

February 9, 2004 CPSC 483

1

Table of Contents 1. Introduction a. Problem Background b. Needs Statement c. Goals and Objectives

3 3 3 4

2. Method of Solution a. Literature and Technical Survey b. Design Constraints and Feasibility c. Evaluation of Alternative Solutions d. Statement of Work i. Proposed Design ii. Approach for Design Validation iii. Economic Analysis and Budget iv. Schedule of Tasks: Pert and Gantt Charts e. Project Management and Team Work f. Societal, Safety, and Environmental Analysis g. Appendices i. Bibliography ii. CV/Qualifications of Team Members iii. Pert and Gantt Charts iv. Product Datasheets

2

4 4 5 5 6 6 8 8 8 8 10 12 12 14 18 21

Introduction Problem Background Wearable computers and affective computing is becoming one of the most interesting and complicated topics of computer human interaction. It involves various fields of study, such as computer science, psychology and physiology. Even though the literature on the subject is still at its primal stages, many projects are underway to discover the link between psychological states and physiological responses of the human body. The main concept is to decipher if our brain processes an emotion before the physiological response or vice versa. In other words, the question is if the emotion is experienced before the physiological change or if the physiological response is translated into an emotion by the brain. Psychologists first looked into the subject and many theories were formulated, many of which clash with each other. Now, thanks to the advances in technology, computer scientists have decided to throw themselves into the mix to try to solve the mystery. At the beginning, the main concern for computing was to create instruments that would correctly measure the physiological changes in a body. For that reason computer scientists collaborated and borrowed from the medical field. They used various instruments, such as electromyogram, or respiratory sensor. Eventually, with time, those instruments became small enough so that more than one sensor could be attached on a person to measure their physiological changes. These are the most recent studies that, NASA and the MIT Labs for instance, have been embarking on. However, the subjects of the experiments are usually covered with multiple wires to register every little change that occurs on their bodies while they are performing a series of psychological and physiological tasks. The need to make those sensors less intrusive as possible has given affective computing a new goal: wireless wearable sensors. The ultimate goal of this technology is to monitor the physiological and emotional state of the human body and keep track of the data via a personal area network accessible to medical personnel. There should be continuous and fast flow of information so the proper care can be given in case of accidents, diseases, or psychological evaluation. Needs Statement In some of the international conferences on wearable computers, many advantages and disadvantages of affective computing have been emerging in the past few years. In particular, there seems to be an issue about the level of comfort of the wearable computers. Some studies have concentrated on creating sensors that could be inserted in pieces of clothing and finding presentable fabrics that can hide the wires and sensors. Others have stressed the incorporation of the sensors into everyday gadgets, such as backpacks or watches. All the projects related to this topic seem to have in common the problem of adding stress and the feeling of uneasiness during experiments. In other words, the subjects undergoing the experiments are attached with multiple sensors, which are connected through multiple wires to bulky encoders and

3

microcontrollers, which then attach to a computer through huge connection units. If the goal is to analyze the emotional and physical levels of the human body and their response under certain situation, just the idea of being hooked up to all those wires is enough to create distress and discourage the participation in such experiments, much less the need to ever want to incorporate that kind of technology in every day life. It is for this reason that we want to concentrate our efforts in creating a wireless device that is not invasive, easy to use and integrate into various objects such as watches or jewelry. Goals and Objectives The goal of our project is to design a device that will measure the blood volume pulse and the skin conductance of an individual who is undergoing a series of tasks. These measurements have to be transmitted wirelessly to a computer which will graph the information in real time. The tasks for the experiment will be divided into physical and physiological: from continuous light physical activities to more “brain involved” activities. The data collected by the device will be sent wirelessly to a computer which will display them in real time through a computer graphical interface. Our main concern is how we can transmit the data in the most efficient way. Some of the solutions we have considered use transreceivers, microcontrollers, and already assembled wireless tools, such as Bluetooth.

Method of Solution Literature and Technical Survey For our project we researched what is already being done in the field. One of the first applications of wearable computers was done by Nasa to create a lightweight ambulatory physiological monitor system to be used during the Spacelab-J mission (STS-47) in space. The apparatus straps on your waist with a belt containing amplifiers, A/D converter, battery and microcontroller. Wires come off of it to connect to the chest, arms and headband, and other parts of the body. All the information are then available on a wrist watch unit which are visible on liquid crystal display. The device measures blood volume, respiration, conductance and electrocardiography. More information can be found at: www.nasatech.com Another source of research was provided by the project Rosalind Picard is working on at the MIT Media Laboratory, along with other colleagues. We concentrated our attention on Affective Jewelry. They created an earring to measure blood volume pressure and a ring and bracelet and a shoe to measure skin conductance. We took inspiration from those projects and decide to utilize the same idea but make it a wireless system. More information can be found at: http://affect.media.mit.edu/AC_research/projects/affective_jewelry.html http://www-white.media.mit.edu/tech-reports/TR-483/

4

We also checked out projects abroad and found a consortium of companies founded by the European Commission. Various companies belong to the consortium each with a different area of expertise. For instance, WEALTHY’s main concern is with creating wearable computers embedded in clothing, or MOBIlearn, which has an interest in business oriented devices. For more information you can check their websites at: http://www.mobilearn.org/index.php http://www.wealthy-ist.com/index.php?action=show_main_page Another interesting site was the Wearable Group at Carnegie Mellon, a research team for the Carnegie institute of Technology. They currently have projects both in hardware and software, such as the Itsy/Cue computer, a palm size computer to Arm Linux. They have a specific wearable group collaborating with Intel and Darpa to design wearable devices for almost every part of the body and to study the flow movement of the human body in space and reproduce it on a computer. Their site is: http://www.wearablegroup.org/ Design Constraints and Feasibility Many companies have produced sensors such as the blood volume pulse sensor and the galvanic skin conductance sensor. The real challenge is getting rid of the wires that almost always accompany them. To make a sensor wireless each sensor’s signal must be distinguishable from any other signal. This means that the data must be transmitted on a unique frequency or be transmitted with an identifier. The other problem in designing a wireless sensor is making it low power and low current, so that a small battery, for instance the size of a watch battery, will be sufficient to supply the power needed. Another constraint is to make the entire transmitting system as small as possible. We know that this project is very feasible because Bluetooth has already produced such a product. Evaluation of Alternative Solutions For cost effective reasons, we researched to find out if it is better to buy the sensors or to make the sensors. It would be significantly cheaper to make the sensors. However, the cons outweigh the pros in this case. First, a significant amount of time would be spent researching and building the sensors. When testing the sensors, we would have to apply an electric shock which can be extremely dangerous if not done properly. Finally, the instruments need to be calibrated so measurements are based on some standard. Thought Technology Limited has offered to provide the sensors to the team at half price since this is an educational project. These sensors have been tested and calibrated and will be ready for use when shipped. The most secure and consequently probably the most expensive way to transmit the data is using a Bluetooth Development kit, produced by Mistral. The chip that is developed and produced by Mistral is about an inch and a half in size, which meets our size requirements. However, the chip is most likely expensive.

5

Unfortunately, the sales person is on vacation and could not be reached for further information on prices and chip specifications. Most likely, by using this solution a microcontroller will be needed to transmit the data. Only one transmitter chip and one receiver chip would be needed. A second alternative solution to this problem is to transmit the data without using a microcontroller. This is feasible by using a transmitter and receiver produce by the company Laipac. The TLP & RLP 434A are simple transmitters and receivers that transmit using radio frequencies at 315 MHz, 418 MHz, and 433.92 MHz. To transmit the data, an encoder and a decoder would be needed, such as the HT12E and HT12D. The problem with this solution is that there would need to be three transmitters and three receivers. This causes strain on our transmitting size constraints. However, it is a very simple solution. With the microcontrollers, there is the option of using one microcontroller at the receiving end or a microcontroller at the sending and receiving ends. In the case of using one microcontroller, data is constantly read from all sensors and transmitted wirelessly using separate TLP/RLP 434A transmitter/receiver pairs for each sensor. The data received from each receiver is read into the microcontroller’s flash memory. The data is then read into the computer. In the case of using two microcontrollers, the sensors are polled at regular intervals, and the converted digital signal is stored in the sending microcontroller’s flash RAM. After the data is sent wirelessly, it is read into the flash of the receiving microcontroller, and then into the computer. The advantage of the first design is that it only uses a microcontroller on the receiving end of the wireless connection. This makes the footprint of the worn device slightly smaller and thus less obtrusive. It is also marginally less expensive, since only one microcontroller is needed rather than two. There are several disadvantages to this design as well. First of all, it is not necessary to constantly transfer data. Receiving a constant stream of data only serves to multiply the timing issues that we will have to deal with. To get useful information on the GUI side, we only need to poll the sensors every few milliseconds. If we do not use a microcontroller, we will only be able to use the TLP/RLP 434A transmitter/receiver. This pair can only operate on three frequencies. Since we cannot combine signals from various sensors onto one transmitter without a microcontroller, we would only be able to have a maximum of three sensors. We will interface with the computer using the parallel port via Inpout32.dll. Inpout32.dll is a dynamic link library that may be used to easily control the operation of the parallel port of a windows computer. The extra speed provided by USB is not necessary for a project with a data transfer rate of at most several kB/s. Statement of Work Proposed Design – Our proposed design is to use one Laipac transmission chip, the TLP 434A, and three Laipac receivers, the RLP 434A, as well as a single encoder, HT12E, and three HT12D decoders. The transmitter and receivers are

6

approximately $10 each and the encoder and decoders are approximately $2.50. In order to accomplish this, we will use a microcontroller on both the transmitting and receiving ends. A microcontroller will be needed to set the address bits of the encoder and set the transmission bit. By choosing a small microcontroller, the size constraints can be met while keeping the cost low. The slightly extra cost of using two microcontrollers is justified by the added flexibility provided by the microcontroller on the receiving end. There are 8 address bits which allows there to be 256 unique addresses which allows for greater production of the product without interference from two devices that have the same addresses. An A/D converter, already embedded into the microcontroller, will be needed to translate the signal from analog to digital. The schematics are pictured below. Figures 1 and 2 show the physiological sensors that will be bought from Thought Technology Limited. Figure 1. Blood Volume Pulse Sensor

Figure 2. Skin Conductance Sensor

Laipac has published an application circuit which uses the TLP & RLP 434A and HT12E & HT12D. This validates that these parts are compatible with each other. The data sheet of the HT12E encoder states that it will transmit when the trigger pin, TE, is set. The microcontroller can set the address bits and tell the A/D converter to set the data bits, and then trigger the encoder to transmit the data. Figure 3 shows how the transceivers work in conjunction with microcontrollers. Figure 3.

7

Approach for Design Validation – Many factors went into decision of finalizing our design. After gathering information about feasibility, costs, and communication between devices, we opted to start from a simple design, prototype it, and, after evaluation, if we feel confident about it we could expend it and improve it. First of all, we agreed on buying the sensors to measure the galvanic skin conductance and the blood volume pulse. This decision came about for timing, economical, and liability reasons. Even though the two sensors will be expensive we felt most of our time should be dedicated to the wireless communication part of the project; moreover, as previously mentioned the skin conductance involves a low voltage, which if not calibrated correctly, could provide uneasiness for the subject undergoing the experiment. Next, we had the most complicated part to decide on: how to implement the wireless connection. Suggestions came from the TA to utilize the Bluetooth kit, a wireless device, to connect to the USB port. Even though it is harder to interface, in the long run, it would be really useful to expand the technology, for instance by adding extra sensors. Ultimately, we found out the cost of the kit was way over the budget at our disposal. Also, we had to take into consideration not only the expendability, but also the sizes of the various objects we will be using to take into account the wearability of our project. We found that using transmitters and receivers would help the size of our prototype even if we needed a few of them, and we had to incorporate decoders and encoders in our design. We were able to find a microcontroller that includes an analog/digital converter, so that we can avoid using additional space, and it also provides better reliability in the transmittance of data to the computer. All of these components fit in the parallel port on the receiver end of our project, and allows for the design to expand adding extra sensors. Overall, we believe our design takes into consideration the size constraints imposed by the project, the reliability of capturing and transmitting data, the usability of the device and the cost effectiveness of choosing certain products and still leaving space for extra or unexpected expenses. Economic Analysis and Budget – The following tables show the budget for the different options.

Wearable Sensors Budget Sensors BVP Sensor Skin Conductance Sensor

1 1

$125.00 $125.00 TOTAL

8

$125.00 $125.00 $250.00

Radio Frequency Part TLP 434A Transmitter RLP 434A Receiver HT12E Encoder HT12D Decoder C8051F015 Microchip

Quantity 2 4 2 4 1

Price $10.00 $10.00 $2.30 $2.30 $22.44

Total $20.00 $40.00 $4.60 $9.20 $22.44 TOTAL

$96.24

BlueTooth Part C8051F015 Microchip Mistral Bluetooth Development Kit

Quantity 1 1

Price $22.44 $15,000.00 TOTAL

Total $22.44 $15,000.00 $15,022.44

Solution Totals Radio Frequency Radio Frequency Parts Sensors Miscellaneous

$96.24 $250.00 $100.00

TOTAL BlueTooth BlueTooth Parts Sensors Miscellaneous

$446.24

$15,022.44 $250.00 $100.00 TOTAL

$15,372.44

If we follow the Radio Frequency solution, versus the BlueTooth solution, it will be a savings of almost $15,000. Schedule of tasks: Pert and Gantt Charts – These charts are provided on pages 16 and 17, respectively. Project Management and Team Work There were four clear cut parts to this project. Each person was assigned to a part of the project so the maximum amount of information could be researched. The following are the assignments made to each person on the team: 1. Clay: wireless components 2. Josh: microcontrollers 3. Gouri: sensors

9

4. Rosy: computer human interaction In general, the four tasks will be the individual objectives. Each person will be expected to become the expert of that part of the project. However, all team members will be educated in every aspect of the project. After reviewing the curriculum vitae and qualifications, it was decided that Clay and Gouri have a better understanding of the hardware aspect of the project, whereas Josh and Rosy are more familiar with the software. So, while Clay and Gouri supervise the construction of the prototype, Josh and Rosy will supervise the GUI design. Clay, Gouri, and Josh will manage the testing of the final prototype, and Rosy will design the tasks to be done by the test subjects. All members of the group will take part in the actual human testing process and analysis of the results. All will collaborate to complete any written assignments such as the final and biweekly reports. The design plan involves: • identifying the needs of the projects (wearable computers) • establishing requirements (bvp, skin conductance, wireless data transmission) • providing alternative designs (trans-receiver or wireless chip); • building prototype • evaluating and testing prototype These are the broad tasks that need to be completed for the project. The Pert and Gantt charts further describe the management of the project. Societal, Safety, and Environmental Analysis Wearable computers are still in the prototype stage. Yet, computing companies are putting financial and technological efforts into them because a market is opening up in various areas where computers are not easily used. We already mentioned the benefits it would have in the medical field: doctors could have at their disposal not only current and detailed information about a patient, but also their medical and family history to act in a preventive way. Another area of public interest would be the use of wearable computers by law enforcement and emergency services. It would definitely make it easier to gather information and dispatch them in real time for crime or accident scenes and to communicate with emergency workers and hospitals in a more effective way. As we realized in the recent world events, in case of a war, wearable computers would be extremely useful. The advantages would be to help monitor the position of soldiers and monitor their health state in case of injuries, and also provide better and faster communications with the command units. Another not so obvious sector that could make use of wearable computers is the industry. Some companies such as Bell Telephone Company in Canada tried out the integration of wearable computers in their customer service support by providing their technicians with hand free computers. The trial was a success in the sense that the computers were small enough not to interfere with the technicians job and cut

10

the time of answering customers’ calls by fifty minutes. (Sleeth C., Wilson L., 2002). In terms of safety, we had to consider two phases: the construction of the prototype and the type of experiments. We made the decision to buy the sensors for blood volume pulse and skin conductance to avoid building them ourselves and running into possible problems of running a voltage, though minimal, through people’s skin and not doing it in the right way. To avoid liability we decided that spending almost half of our budget was in our best interest. The other issue where safety comes into play is the physical data tracking of the experiments. Since it takes only minimal changes in our body to see a difference in the blood pressure and conductance we will design a physical experiment of about thirty minutes consisting of very light activities, such as jogging in place and sitting. We will pick three subjects whose age is in the twenties, thirties, forties and more. Therefore, we will have to slightly adapt the physical exercise if anybody feels uncomfortable with the activities. We also took into consideration the simple fact that we have to swap the devices between various people, and we came to the conclusion that the best thing to do would be to swipe the device, specifically, the cuffs that will go around the fingers with sanitary cloths. After 50 uses, the pads will need to be replaced if testing needs to be accurate. As for the environment of our experiment, we decided it will be done in the classroom and consequently, the prototype will be built to be utilized indoors and at a distance of not more than 50 to 100 feet away from a computer. In the phases of our project we will keep in mind the six attributes a wearable computer should have according to Steve Mann (1998), a leading researcher in wearable devices: 1. 2. 3. 4. 5. 6.

not monopolizing of the user’s attention not restrictive to the user observable by the user controllable by the user attentive to the environment communicative to others

If we follow these attributes, we will be going in the correct direction to making wearable computers actually wearable.

11

Appendices Bibliography Beyond Logic. http://www.beyondlogic.org/. Beyond Logic contains general information about interfacing with the USB and parallel ports. Logix4U. http://www.logix4u.net/ This site includes tutorials on interfacing with parallel ports in general, using inpout32.dll to interface with the parallel port, and information on creating dlls for windows. Silicon Laboratories. http://www.silabs.com/ Silicon Laboratories sells small multi-purpose microcontrollers and related products needed to interface with them. We are using Silicon Laboratories to purchase two C8051F015 mixed-signal microcontrollers. Affective Computing Research Areas. http://affect.media.mit.edu/AC_research/ MIT Media Laboratory has several projects involving affective computing. The most pertinent of these is the affective jewelry research project. They have developed jewelry which measures human physiological signals. Laipac. http://www.laipac.com Laipac sells GPS and wireless technology. We are using Laipac to purchase TLP/RLP 434A wireless transmitters and receivers. Golledge Electronics Frequency Control Products. http://www.golledge.com. Golledge sells frequency control products such as oscillators and crystals. We are using Golledge to purchase a GXO-U102 oscillator. Axelson, Jan. Parallel Port Complete: Programming, Interfacing & Using the PC'S Parallel Printer Port. Independent Publishers Group. This book is a reference for using the parallel port. In addition, it includes several examples of projects done using the parallel port, including one that interfaces with analog sensors. Picard, Rosalind. Affective computing. MIT Press. Cambridge, Massachussets. 1997. This book includes much information on the theory of affective computing and the conditions needed for it to be successfully put into practice. Techniques in Psychophysiology. Edited by Irene Martin and Peter Venables . John Wiley & Sons, c1980. QP360T4. This book includes techniques for psychophysiological measurement of human subjects. It is useful in determining the operation of the sensors.

12

Fundamentals of wearable computers and augumented reality. edited by Woodrow Barfield and Thomas Caudell. Electronic resource. Lawrence Erlbaum Associates. Mahwah, NJ. 2000. This book examines the ideas and technologies necessary to more closely bond humans and computers by using wearable computers. 7th International Symposium on Wearable Computers. IEEE online resource. October 21-23, 2003. Topics at this symposium included applications of wearable computing, hardware involved with wearable computing, human interfaces, social implications, and the future of wearable computing.

13

Josh Handley Permanent Address 128 Darlington Huntsville, AL 35801

Local Address PO Box 5787 College Station, TX 77844 (979) 847-0861 [email protected]

Education

Texas A&M University College Station, TX Bachelor of Science, Computer Engineering(CPSC) Fall 2000 to Present Expected Graduation: Spring 2004 3.846 cumulative GPR Technical Electives :Artificial Intelligence, Networks, Graphics

Projects

For my Microcomputer Systems Design course, I created a system which used an EB63 evaluation board – with MMLite installed – and the SOAP communication protocol to display text messages to an LCD. The demonstration of this included a Win32 Tetris clone which displayed various game statistics to the LCD.

Experience

Nektar Therapeutics Summer 2003 IT Intern I designed and implemented a chemical procedure documentation database - using MS SQL Server – and an application to modify it – using Visual Basic. In addition, I preformed various network and personal computer hardware troubleshooting activities.

Skills

Software Knowledge:

Proficient in Microsoft Operating Systems, MS Office, MS Visual Studio Basic knowledge of Xilinx ISE, MS SQL Server Programming Knowledge:

Proficient in C, C++, Java, Visual Basic, MIPS32 Assembly, Verilog Basic knowledge of SQL

Honors

Lechner Fellowship Scholarship, Director’s Excellence Award, Member of Tau Beta Pi, Member of Upsilon Pi Epsilon

14

Rosy Logioia – CPSC 483 Section 502 Computer Engineering -- CECN e-mail: [email protected] phone: 979-691-6371 ELECTIVES: CPSC 310: Database CPSC 436: Computer Human Interaction CPSC 463: Networking INTERESTS: I would like to learn more about and become more familiar with computer human interaction; general web-related topics (servers, designing pages, etc.); networking; hardware. SKILLS: C, C++, little JAVA, SQL, Verilog, Mips

Spring 2004 CLASS SCHEDULE: Monday 9:35-10:50 10:20-11:10 3:55-5:10 4:10-7:00

Tuesday CPSC 436

KINE 198

Thursday CPSC 436

KINE 198 CPSC 463

CPSC 483

CPSC 463 CPSC 483

GPA: 2.67 PROJECT PICKS: 1) 2) 3) 4)

Wednesday

Robot learning + tactile whiskers. Navigate a maze. Hack Furby. Coff-e-mail.

15

Gouri Mallikarjun Shintri P.O. Box 4178 Escondido Dr. College Station, TX 77844 77083 (832) 443-5169 6139

6619 Houston, TX (281) [email protected]

______________________________________________________________________________________ _______

Education

GPA 3.3/4.0 Technical Electives: Database Systems, Structured Programming in Ada

Experience 5/02 to Present

Skills

Temporary Part-time Intern. Hewlett-Packard Company Within the past year, I have led a team of 9 in testing products which involve exposure to networking, computer architecture, and different operating systems. I have also had the opportunity to be involved in project management projects that deal with customer interaction and budget forecasting. Currently, I am developing troubleshooting and management skills as part of a mentoring activity while continuing with the assignments listed above. Programming Languages: C, Java Operating Systems: Worked with all Windows, most RedHat

Linux, SLES 7.0, Netware 5.0/5.1/6.0 Preference able to

I would consider myself a hardware person because I have been understand it better than software. However, majority of my practical experiences have been with software.

Top 4 Picks Sensor

1.Coff-e-mail, 2.Facial Tracking, 3.Circadian Circuits, 4.Wearable

Class Schedule

CPSC 483 CPSC 463 CPSC 485

MW 410-500 TR 320-530 TR 355-510 To Be Announced

16

CLAYTON SMITH 5914 CR 216 Richards, Texas 77873 979-450-1714 [email protected]

Education

Relevant Experience

Texas A&M University, College Station, Texas B.S. in Computer Engineering GPR 2.9

Graduation May 2004

Technical Electives: • Computer Graphics • Independent Study on teaching students to add multimedia to Java applications • Networking Web Page Designer • Multi-layer three tier architecture • Users over the internet dynamically create, design, and edit a web page • Project included Java applets, servlets, and mySQL database GAMA Advertising Inc. • Approached advertising firm to integrate program into practice • Designed to track sales and automatically produce reports • Implemented using Java Swing GUI • Independent project that is currently in testing phase IP Communications •

Other Experience

Software Base

•

IT Technician Summer and Winter of 2000 Developed the Network Operations Control Center (NOCC) department web page Updated hardware, software, and trouble shot user problems

Union Grove Baptist Church August 2002-Current Youth Director • Developed the following skills: leadership, budget management, fund raising, public speaking, personal preparation, and planning events Rumours Coffee House and Deli September 2001Current Student Supervisor • Promoted to student supervisor in under one year • Managed staff of ten personnel • Managed money Extensive programming experience • C++ and Java. Experience • Rational Rose, Verilog, Dreamweaver, Adobe Photoshop, Microsoft Office and many others

17